iptables –set-mark – Route diferent ports through different interfaces
Short story,
3 interfaces, eth0 (LAN), eth1(ADSL), eth2(4G).
eth0 -> eth1: Works
(ports 80, 443, 4070) eth0 -> eth2: Doesn’t happen
This is a graphical representation of the idea:
Port 80 & 443 via eth2
ant the rest via eth1
Netscheme:
eth0: -ip 10.0.0.1 -net 10.0.0.0/8 -gw 10.0.0.1 (the servers own intf) eth1: -ip 192.168.1.74 -net 192.168.1.0/24 -gw 192.168.1.254 eth2: -ip 192.168.1.91 -net 192.168.0.0/24 -gw 192.168.0.1
This new script reroutes 22 and 4070 to the proper table, I think.
However, after getting to that table, it doesn’t get rerouted to eth2.
This script works, except for 22 and 4070!
(Port 80 is un-commented and it works but via eth1 which is wrong.)
modprobe iptable_nat modprobe ip_conntrack echo "1" > /proc/sys/net/ipv4/ip_forward iptables -P INPUT ACCEPT iptables -F INPUT iptables -P OUTPUT ACCEPT iptables -F OUTPUT iptables -P FORWARD DROP iptables -F FORWARD iptables -F PREROUTING iptables -t nat -F iptables -t mangle -F iptables -F # This next line restores any issues trying to connect to something # if you get weird ACK packets when trying to connect (at least i did)! iptables -t mangle -A PREROUTING -p tcp -j CONNMARK --restore-mark ip route flush table main iptables -A PREROUTING -i eth0 -t mangle -p tcp --dport 22 -j MARK --set-mark 1 ### iptables -A PREROUTING -i eth0 -t mangle -p tcp --dport 80 -j MARK --set-mark 1 iptables -A PREROUTING -i eth0 -t mangle -p tcp --dport 4070 -j MARK --set-mark 1 ## Setup routes # LAN route add -net 10.0.0.0 netmask 255.0.0.0 dev eth0 # ADSL route add -net 192.168.1.0 netmask 255.255.255.0 dev eth1 # 4G (Only accessible if marking packages with x01 route add -net 192.168.0.0 netmask 255.255.255.0 dev eth2 # Default via ADSL ## -- Does the same as ip route below? route add default gw 192.168.1.254 echo "201 eth2.out" >> /etc/iproute2/rt_tables ip rule add fwmark 1 table eth2.out ip route add default via 192.168.0.1 dev eth2 table eth2.out ip route add default via 192.168.1.254 dev eth1 ## Setup forwards # From 4G to LAN iptables -A FORWARD -i eth2 -o eth0 -m state --state ESTABLISHED,RELATED,NEW -j ACCEPT # From ADSL to LAN iptables -A FORWARD -i eth1 -o eth0 -m state --state ESTABLISHED,RELATED,NEW -j ACCEPT # From LAN to ADSL (Default route out) # - Note: If marked packages is sent to ADSL they will be mangled and rerouted to 4G iptables -A FORWARD -i eth0 -o eth1 -j ACCEPT iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE iptables -t nat -A POSTROUTING -o eth2 -j MASQUERADE
Old script:
Ignore everything below unless you're interested in retracing my steps!!
I’ve created a router.sh script to setup my environment in case I do something bad.
I’ve got 3 ports which I want to send to a 4G connection and the rest via a landline ADSL connection.
To do this, I’ve instructed iptables to mangle packages on the default route and send them via my 4G interface if the –dport == 443 | 80 | 4070
However, this doesn’t work; I’m still getting routed through my landline no matter what.
This is what my script looks like:
#!/bin/bash ## routing tables # wireless = 4G via eth2 # adsl = adsl via eth1 modprobe iptable_nat modprobe ip_conntrack echo "1" > /proc/sys/net/ipv4/ip_forward iptables -P INPUT ACCEPT iptables -F INPUT iptables -P OUTPUT ACCEPT iptables -F OUTPUT iptables -P FORWARD DROP iptables -F FORWARD iptables -t nat -F ip route flush table main ip route flush table wireless ip route flush table adsl ## Setup routing tables # ADSL ip route add table adsl to 192.168.1.0/24 dev eth1 # 4G ip route add table wireless to 192.168.0.0 dev eth2 ip rule add fwmark 0x1 table wireless ## Setup routes # LAN route add -net 10.0.0.0 netmask 255.0.0.0 dev eth0 # ADSL route add -net 192.168.1.0 netmask 255.255.255.0 dev eth1 # 4G (Only accessible if marking packages with x01 route add -net 192.168.0.0 netmask 255.255.255.0 dev eth2 # Default via ADSL route add default gw 192.168.1.254 ## Forward ports into the LAN iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 80 -j DNAT --to 10.0.0.3:80 ## Lets mark all packets we want for 4G forward # HTTPS iptables -A OUTPUT -t mangle -o eth1 -p tcp --dport 443 -j MARK --set-mark 1 # HTTP iptables -A OUTPUT -t mangle -o eth1 -p tcp --dport 80 -j MARK --set-mark 1 # Spotify iptables -A OUTPUT -t mangle -o eth1 -p tcp --dport 4070 -j MARK --set-mark 1 ## Setup forwards # From 4G to LAN iptables -A FORWARD -i eth2 -o eth0 -m state --state ESTABLISHED,RELATED,NEW -j ACCEPT # From ADSL to LAN iptables -A FORWARD -i eth1 -o eth0 -m state --state ESTABLISHED,RELATED -j ACCEPT # From LAN to ADSL (Default route out) # - Note: If marked packages is sent to ADSL they will be mangled and rerouted to 4G iptables -A FORWARD -i eth0 -o eth1 -j ACCEPT iptables -A FORWARD -j LOG #iptables --table nat --append POSTROUTING --out-interface eth2 --jump SNAT --to-source "192.168.1.74" iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE
I’ve also tried to add these 3 to the bottomg of the script:
iptables -t nat -A POSTROUTING -o eth2 -p tcp --dport 80 -j SNAT --to "192.168.0.91" iptables -t nat -A POSTROUTING -o eth2 -p tcp --dport 443 -j SNAT --to "192.168.0.91" iptables -t nat -A POSTROUTING -o eth2 -p tcp --dport 4070 -j SNAT --to "192.168.0.91"
Also tried without success:
iptables -A PREROUTING -t mangle -i eth0 -p tcp --dport 80 -j MARK --set-mark 1
Last but not least, tried:
## Lets mark all packets we want for 4G forward # HTTPS iptables -A POSTROUTING -t mangle -o eth1 -p tcp --dport 443 -j MARK --set-mark 1 # HTTP iptables -A POSTROUTING -t mangle -o eth1 -p tcp --dport 80 -j MARK --set-mark 1 # Spotify iptables -A POSTROUTING -t mangle -o eth1 -p tcp --dport 4070 -j MARK --set-mark 1
The routing works, I can browse the web, listen to music and what-not, but I’m doing it through the wrong interface.
I’ve Googled around for a long while now and found bits and pieces to understand what I’m doing and why I’m doing it. I could do traffic shaping via tc but if it’s possible via marking packages in iptables it would help me a long way.
My guess is that I’m doing the order wrong on the different rules, mainly the MASQUERADE part? or if that should even be there?
Can someone explain how to DNAT port say, tcp:80 from an external interface (either one or BOTH protocols) to a internal 10.0.0.0 address space?
Outputs:
<a href="https://getridbug.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="790b16160d39371c0d1b0b101d1e1c">[email protected]</a>:~# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface<br> 0.0.0.0 192.168.1.254 0.0.0.0 UG 0 0 0 eth1<br> 10.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 eth0<br> 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2<br> 192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
<a href="https://getridbug.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="17657878635759726375657e737072">[email protected]</a>:~# ifconfig eth0 Link encap:Ethernet HWaddr 00:0c:29:7e:9e:4e inet addr:10.0.0.1 Bcast:10.255.255.255 Mask:255.0.0.0 eth1 Link encap:Ethernet HWaddr 00:0c:29:7e:9e:58 inet addr:192.168.1.74 Bcast:192.168.1.255 Mask:255.255.255.0 eth2 Link encap:Ethernet HWaddr 00:0c:29:7e:9e:62 inet addr:192.168.0.91 Bcast:192.168.0.255 Mask:255.255.255.0
Followed these instructions:
output-traffic-on-different-interfaces-based-on-destination-por
iptables-forward-specific-port-to-specific-nic
Among a few other related threads.
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
BatchyX already give some very good explanation about iptables and routing, so I will exercise my laziness and go directly to script.
It should NAT all traffic to port 80,443,22,4070 through 192.168.0.91. All the rest will NAT through 192.168.1.254.
I re-do my testing and end up following this guide. What is missing in that guide is the last 3 lines in my script. Which I found out from another port, but I lost track of that link.
It is a tested working script.
Need Default Route
One thing I did not put in the script is setting up the default route. It should be
route add default gw 192.168.1.254
When you do
route -n
, it should be the only default route (Dest:0.0.0.0)0.0.0.0 192.168.1.254 0.0.0.0 UG 0 0 0 eth1
fw-router.sh
# Reset/Flush iptables iptables -F iptables -X iptables -t nat -F iptables -t nat -X iptables -t mangle -F iptables -t mangle -X iptables -P INPUT ACCEPT iptables -P FORWARD ACCEPT iptables -P OUTPUT ACCEPT #Reset/Flush/Setup IP Route (table 4) ip route flush table 4 ip route show table main | grep -Ev ^default | while read ROUTE ; do ip route add table 4 $ROUTE ; done ip route add table 4 default via 192.168.0.1 #Mark Packet with matching D.Port iptables -t mangle -A PREROUTING -p tcp --dport 22 -s 10.0.0.0/24 -j MARK --set-mark 4 iptables -t mangle -A PREROUTING -p tcp --dport 80 -s 10.0.0.0/24 -j MARK --set-mark 4 iptables -t mangle -A PREROUTING -p tcp --dport 443 -s 10.0.0.0/24 -j MARK --set-mark 4 iptables -t mangle -A PREROUTING -p tcp --dport 4070 -s 10.0.0.0/24 -j MARK --set-mark 4 #SNAT Rules iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 192.168.1.74 iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to-source 192.168.0.91 #IP Route ip rule add fwmark 4 table 4 ip route flush cache #IP Stack #This is the missing part from the guide echo 1 > /proc/sys/net/ipv4/ip_forward for f in /proc/sys/net/ipv4/conf/*/rp_filter ; do echo 0 > $f ; done echo 0 > /proc/sys/net/ipv4/route/flush
PS1: In short, MASQUERADE
does not work (in most case, and definitely in your case) for NAT with multiple external IPs that need some kind of load balancing or need DNAT to handle incoming traffic. You need SNAT
for direction control.
PS2: Pure iptables is not sufficient.
Method 2
Note: I only have considered the first script, ignoring the old one.
- You don’t need to modprobe netfilter modules by hand with current iptables. This is only necessary for custom connection trackers.
- Don’t mix up
route
andip route
. This is pure evil. Just useip
everywhere and forget aboutifconfig
androute
-
/etc/iproute2/rt_tables
is not reset across reboots. Appending the same entry over and over is not a good idea, you only need to do it once. Remember thatrt_tables
just define name aliases to numeric values, it does not change any configuration. -
Now for
iptables
:
In yourFORWARD
chain, you drop packets coming from LAN to 4G. This is bad. theFORWARD
hook is used after routing is done. At this point, all policy routing is done, and it is already known whether the packet should be sent to 4G or ADSL. There is no rerouting done inFORWARD
or afterFORWARD
(well, technically, rerouting can be done afterPOSTROUTING
in severe cases, but back to the point).
Now for your routing: Remember that Ubuntu enables reverse path filtering by default. Reverse path filtering works as follow: When the kernel receives a packet (may it be forwarded or not) from an interface A, it will invert the source address and the destination address, and check if the resulting packet should be routed through interface A. If it isn’t, the packet is dropped as a address spoofing attempt.
For packets received from eth0
, this is not a problem. For packets received from eth1
, this is also not a problem, because when reversing the source IP address and the destination IP address, the kernel will it the default route in table main
. For packets received from eth2
, which you do not mark, this is a problem, because the kernel will hit the default route in table main
, and consider that these packets should have been received from eth1
. The easiest solution is to disable reverse path filtering on eth1:
sysctl -w net.ipv4.conf.eth1.rp_filter=0
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0