Issue With NAT and Multiple WAN Interfaces _ Vyatta

22/4/2014

Issue with NAT and multiple WAN interfaces | Vyatta.org Community

Vyatta.com (http://vyatta.com/) Support Center (http://www2.vyatta.com/support)

Vyatta.org Community

|

Search

GETTING STARTED

SUPPORT & SERVICES

DOWNLOADS

COM M UNITY

DOCUM ENTATION

CONTRIBUTE

FORUM S

Issue with NAT and multiple WAN interfaces

Log in or register to post comments

Last post

11 posts / 0 new

Fri, 12/17/2010 06:52

#1

Singularity Issue with NAT and multiple WAN interfaces Hello, I am having some difficulty implementing a simple NAT setup with multiple WAN interfaces. The setup consists of desktop PC's on a 10.1.100.0/16 network and VoIP servers on a 10.1.20.0/16 network. These access the internet through a Vyatta installation with a single LAN interface eth0 (10.1.30.1/16). The Vyatta server is configured with one LAN interface and 4x ADSL PPPOE interfaces (pppoe0-pppoe3). Currently all traffic is NAT'ed through pppoe3, which is set up as Vyatta's default route. What I need to do is: 1) Use Source NAT to selectively route outbound traffic from the LAN through different WAN interfaces depending on the IP address of the internal PC. This will allow me to send the (10.1.20.0/16) VoIP traffic out a different WAN interface to general internet (10.1.100.0/16) traffic. 2) Perform Destination NAT on incoming WAN packets based on interface and destination port. This will allow me to send incoming VoIP packets from any WAN interface to a VoIP server on the LAN. At the moment I'm running into problems: http://www.vyatta.org/node/4675

1/17

22/4/2014


1) When using Source NAT, any rule that does not use the current default route/device seems to be ignored. For some reason Vyatta is just skipping over the rule. I have no idea why this is happening, but if I change the outbound interface and outgoing address to that of default route (pppoe3), the rule works. For example, rule 5 in my configuration never shows any hits in the statistics when pppoe3 is set as the default route, all traffic hits rule 10 instead, which is using the default route interface. 2) When using Destination NAT to redirect traffic from a WAN interface which is not the default route, I end up with an asynchronous routing scenario. For example, rule 1070 in my configuration collects data from external clients coming in on pppoe0, port 4569 (IAX2). It then sends it to internal address 10.1.20.5 (VoIP Server) - This works correctly, and data can be seen arriving at the server. However, when 10.1.20.5 replies, data can be seen leaving the VoIP server and arriving at Vyatta. It is then for some reason sent out the default route interface (pppoe3) instead of the originating WAN interface (pppoe0). Any ideas why this is happening? Isn't Vyatta supposed to maintain session state and send things back out the interface from which they came? When I check the packets leaving pppoe3, they actually have the correct source address of pppoe0, so its like the NAT side of things is working, its just getting screwed up during routing. Here is my sanitized current configuration: interfaces { ethernet eth0 { address 10.1.30.1/16 duplex auto hwid 00:0c:29:64:a1:43 smp_affinity auto speed auto } ethernet eth1 { duplex auto hwid 00:0c:29:64:a1:6b pppoe 0 { defaultroute none mtu 1492 nameserver auto password password userid adsl_voip } smp_affinity auto speed auto } ethernet eth2 { duplex auto hwid 00:0c:29:64:a1:4d pppoe 1 { defaultroute none http://www.vyatta.org/node/4675

2/17

22/4/2014


mtu 1492 nameserver auto password password userid adsl_inet2 } smp_affinity auto speed auto } ethernet eth3 { duplex auto hwid 00:0c:29:64:a1:57 pppoe 2 { defaultroute none mtu 1492 nameserver auto password password userid adsl_vpn } smp_affinity auto speed auto } ethernet eth4 { duplex auto hwid 00:0c:29:64:a1:61 pppoe 3 { defaultroute auto mtu 1492 nameserver auto password password trafficpolicy { out VoIPPriority } userid adsl_inet } smp_affinity auto speed auto } loopback lo { } } protocols { static { interfaceroute 58.1.1.1/32 { nexthopinterface pppoe2 { } } interfaceroute 58.2.2.2/32 { nexthopinterface pppoe2 { } } interfaceroute 124.1.1.1/32 { nexthopinterface pppoe2 { } } interfaceroute 202.2.2.2/32 { nexthopinterface pppoe2 { } } } } service { nat { rule 5 { description "NAT for VoIP Servers" outboundinterface pppoe0 http://www.vyatta.org/node/4675

3/17

22/4/2014


outsideaddress { address 58.3.3.3 } source { address 10.1.0.0/16 } type source } rule 10 { description "General Outbound NAT" outboundinterface pppoe3 outsideaddress { address 150.1.1.1 } source { address 10.1.0.0/16 } type source } rule 15 { description "NAT for VoIP Servers" outboundinterface pppoe0 outsideaddress { address 58.3.3.3 } source { address 10.1.20.0/24 } type source } rule 1000 { description "Incomming IMAP" destination { address 150.1.1.1 port 143 } inboundinterface pppoe3 insideaddress { address 10.1.10.6 } protocol tcp source { address 0.0.0.0/0 } type destination } rule 1010 { destination { address 202.1.1.1 port 3000 } inboundinterface pppoe2 insideaddress { address 10.1.11.5 port 3389 } protocol udp type destination } rule 1020 { description RulesforVoIP destination { port 50605090 } inboundinterface pppoe3 http://www.vyatta.org/node/4675

4/17

22/4/2014


insideaddress { address 10.1.20.1 } protocol tcp type destination } rule 1030 { destination { port 50605090 } inboundinterface pppoe3 insideaddress { address 10.1.20.1 } protocol udp type destination } rule 1040 { destination { port 3000031000 } inboundinterface pppoe3 insideaddress { address 10.1.20.1 } protocol udp type destination } rule 1050 { destination { port 2021 } inboundinterface pppoe3 insideaddress { address 10.1.20.1 } protocol tcp type destination } rule 1060 { destination { address 150.1.1.1 port 85 } inboundinterface pppoe3 insideaddress { address 10.1.115.188 } protocol tcp source { address 0.0.0.0/0 } type destination } rule 1070 { destination { address 58.3.3.3 port 4569 } inboundinterface pppoe0 insideaddress { address 10.1.20.5 } protocol udp source { http://www.vyatta.org/node/4675

5/17

22/4/2014


address 0.0.0.0/0 } type destination } } ssh { port 22 protocolversion 2 } } system { hostname vyatta.server.com login { banner { prelogin "Vyatta Server " } user vyatta { authentication { encryptedpassword XXXXXXXXXXXXXXXXXXXXXX plaintextpassword "" } level admin } } ntpserver 0.vyatta.pool.ntp.org package { autosync 1 repository community { components main distribution stable password "" url http://packages.vyatta.com/vyatta (http://pack

ages.vyatta.com/vyatta) username "" } } syslog { global { facility all { level notice } facility protocols { level debug } } } timezone Australia/Sydney } trafficpolicy { shaper VoIPPriority { bandwidth 650kbit class 10 { bandwidth 60% burst 15k ceiling 100% description "VoIP RTP traffic" match VOIPRTP { ip { dscp 46 } } queuetype fairqueue } class 20 { bandwidth 5% http://www.vyatta.org/node/4675

6/17

22/4/2014


burst 15k ceiling 100% description "VoIP SIP Traffic" match VOIPSIP { ip { dscp 26 } } queuetype fairqueue } default { bandwidth 15% burst 15k ceiling 100% queuetype fairqueue } description "QoS policy to decrease VoIP latency" } } vpn { ipsec { espgroup p2 { compression disable lifetime 1800 mode tunnel pfs enable proposal 1 { encryption aes128 hash sha1 } } ikegroup p1 { lifetime 7200 proposal 1 { dhgroup 2 encryption aes128 hash sha1 } } ipsecinterfaces { interface pppoe2 } logging { logmodes all } sitetosite { peer 58.1.1.1 { authentication { mode presharedsecret presharedsecret REALPSKGOESHERE } ikegroup p1 localip 202.1.1.1 tunnel 1 { allownatnetworks disable allowpublicnetworks disable espgroup p2 localsubnet 10.1.0.0/16 remotesubnet 10.4.0.0/16 } } peer 58.2.2.2 { authentication { mode presharedsecret presharedsecret REALPSKGOESHERE http://www.vyatta.org/node/4675

7/17

22/4/2014


} ikegroup p1 localip 202.1.1.1 tunnel 1 { allownatnetworks disable allowpublicnetworks disable espgroup p2 localsubnet 10.1.0.0/16 remotesubnet 10.2.0.0/16 } } peer 124.1.1.1 { authentication { mode presharedsecret presharedsecret REALPSKGOESHERE } ikegroup p1 localip 202.1.1.1 tunnel 1 { allownatnetworks disable allowpublicnetworks disable espgroup p2 localsubnet 10.1.0.0/16 remotesubnet 10.3.0.0/16 } } peer 202.2.2.2 { authentication { mode rsa rsakeyname Wollongongkey } ikegroup p1 localip 202.1.1.1 tunnel 1 { allownatnetworks disable allowpublicnetworks disable espgroup p2 localsubnet 10.1.0.0/16 remotesubnet 10.4.0.0/16 } } } } rsakeys { rsakeyname Wollongongkey { rsakey REALRSAKEYGOESHERE } } }

/* Warning: Do not remove the following line. */ /* === vyattaconfigversion: "cluster@1:conntracksync@1:dhcpre lay@1:dhcpserver@4:firewall@3:ipsec@2:nat@3:qos@1:quagga@2:syste m@3:vrrp@1:wanloadbalance@2:webgui@1:webproxy@1" === */

Result of "show interfaces": vyatta@io:~$ show interfaces Interface IP Address State Link Description eth0 10.1.30.1/16 up up eth1 up up eth2 up up eth3 up up http://www.vyatta.org/node/4675

8/17

22/4/2014


eth4 up up lo 127.0.0.1/8 up up lo ::1/128 up up pppoe0 58.3.3.3 up up pppoe1 202.3.3.3 up up pppoe2 202.1.1.1 up up pppoe3 150.1.1.1 up up vyatta@io:~$

IP Tables Info: IPTABLES Filter Chain Details Chain INPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 951K 337M VYATTA_POST_FW_HOOK all * * 0.0.0.0/ 0 0.0.0.0/0 Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 300K 133M VYATTA_POST_FW_HOOK all * * 0.0.0.0/ 0 0.0.0.0/0 Chain OUTPUT (policy ACCEPT 22952 packets, 6031K bytes) pkts bytes target prot opt in out source destination Chain VYATTA_POST_FW_HOOK (2 references) pkts bytes target prot opt in out source destination 1251K 471M ACCEPT all * * 0.0.0.0/0 0.0.0.0/0 Nat Chain Details Chain PREROUTING (policy ACCEPT 12901 packets, 926K bytes) pkts bytes target prot opt in out source destination 13084 936K VYATTA_PRE_DNAT_HOOK all * * 0.0.0.0 /0 0.0.0.0/0 178 9030 DNAT tcp pppoe3 * 0.0.0.0/0 150.1.1.1 tcp dpt:143 /* NAT1000 */ to:10.1.10.6 0 0 DNAT udp pppoe2 * 0.0.0.0/0 202.1.1.1 udp dpt:3000 /* NAT1010 */ to:10.1.11.5:3389 0 0 DNAT tcp pppoe3 * 0.0.0.0/0 0.0.0.0/0 multiport dports 5060:5090 /* NAT1020 */ to:10.1.20.1 1 649 DNAT udp pppoe3 * 0.0.0.0/0 0.0.0.0/0 multiport dports 5060:5090 /* NAT1030 */ to:10.1.20.1 0 0 DNAT udp pppoe3 * 0.0.0.0/0 0.0.0.0/0 multiport dports 30000:31000 /* NAT1040 * / to:10.1.20.1 0 0 DNAT tcp pppoe3 * 0.0.0.0/0 0.0.0.0/0 multiport dports 20:21 /* NAT1050 */ to:1 0.1.20.1 http://www.vyatta.org/node/4675

9/17

22/4/2014


0 0 DNAT tcp pppoe3 * 0.0.0.0/0 150.1.1.1 tcp dpt:85 /* NAT1060 */ to:10.1.115.188 4 215 DNAT udp pppoe0 * 0.0.0.0/0 58.3.3.3 udp dpt:4569 /* NAT1070 */ to:10.1.20.5 Chain POSTROUTING (policy ACCEPT 7741 packets, 565K bytes) pkts bytes target prot opt in out source destination 14591 1002K VYATTA_PRE_SNAT_HOOK all * * 0.0.0.0 /0 0.0.0.0/0 0 0 SNAT all * pppoe0 10.1.0.0/16 0.0.0.0/0 /* NAT5 */ to:58.3.3.3 6850 436K SNAT all * pppoe3 10.1.0.0/16 0.0.0.0/0 /* NAT10 */ to:150.1.1.1 0 0 SNAT all * pppoe0 10.1.20.0/24 0.0.0.0/0 /* NAT15 */ to:58.3.3.3 Chain OUTPUT (policy ACCEPT 2487 packets, 178K bytes) pkts bytes target prot opt in out source destination Chain VYATTA_PRE_DNAT_HOOK (1 references) pkts bytes target prot opt in out source destination 13084 936K RETURN all * * 0.0.0.0/0 0.0.0.0/0 Chain VYATTA_PRE_SNAT_HOOK (1 references) pkts bytes target prot opt in out source destination 14591 1002K RETURN all * * 0.0.0.0/0 0.0.0.0/0 Mangle Chain Details Chain PREROUTING (policy ACCEPT 95443 packets, 30M bytes) pkts bytes target prot opt in out source destination Chain INPUT (policy ACCEPT 80297 packets, 28M bytes) pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 15144 packets, 1994K bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 3464 packets, 1139K bytes) pkts bytes target prot opt in out source destination Chain POSTROUTING (policy ACCEPT 18608 packets, 3133K bytes) pkts bytes target prot opt in out source destination Raw Chain Details Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 1260K 474M VYATTA_PRE_CT_PREROUTING_HOOK all * * 0.0.0.0/0 0.0.0.0/0 1260K 474M NAT_CONNTRACK all * * 0.0.0.0/0 http://www.vyatta.org/node/4675

10/17

22/4/2014


0.0.0.0/0 0 0 NOTRACK all * * 0.0.0.0/0 0.0.0.0/0 Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 23203 6124K VYATTA_PRE_CT_OUTPUT_HOOK all * * 0. 0.0.0/0 0.0.0.0/0 23203 6124K NAT_CONNTRACK all * * 0.0.0.0/0 0.0.0.0/0 0 0 NOTRACK all * * 0.0.0.0/0 0.0.0.0/0 Chain NAT_CONNTRACK (2 references) pkts bytes target prot opt in out source destination 1284K 480M ACCEPT all * * 0.0.0.0/0 0.0.0.0/0 Chain VYATTA_PRE_CT_OUTPUT_HOOK (1 references) pkts bytes target prot opt in out source destination 23203 6124K RETURN all * * 0.0.0.0/0 0.0.0.0/0 Chain VYATTA_PRE_CT_PREROUTING_HOOK (1 references) pkts bytes target prot opt in out source destination 1260K 474M RETURN all * * 0.0.0.0/0 0.0.0.0/0

If anyone can offer suggestions as to what I'm doing wrong and how to go about fixing these issues, it would be greatly appreciated. Thanks, Jeff Top

or to post comments

Tue, 12/21/2010 09:00

Permalink

Singularity Issue with NAT and multiple WAN interfaces As per another post I found on this forum, I have changed my config to use load balancing rules with a single interface per rule to replace my source NAT rules... This appears to be working... However I'm still having issues with destination NAT. Does anyone have any ideas about why I'd be having an asymmetric routing issue when using multiple WAN interfaces with DNAT? Using rule 1070 from my config (see previous post) as an example, I can see packets come in pppoe0 from a remote address on port 4569. http://www.vyatta.org/node/4675

11/17

22/4/2014


These are correctly NAT'ed to 10.1.20.5 which sees the request from the remote address and sends a response back through the Vyatta router. Vyatta then receives this request and correctly changes the source address of the packet to that of pppoe0. It is at this point where we run into problems - Vyatta now takes the packet and sends it out any one of the WAN interfaces pppoe0, pppoe1, pppoe2 or pppoe3, the interface picked seems to be random. How can I FORCE Vyatta to choose the correct interface pppoe0? Surely I can't be the only one having this sort of problem...? I've gotta be going wrong somewhere configuration wise, yet as far as I can see what I've done looks correct... Is there some sort of known Vyatta limitation at work here? Does anyone have a successful multiple WAN setup with destination NAT config they could share? Thanks, Jeff Top

or to post comments

Tue, 12/21/2010 09:17

Permalink

Tijz Issue with NAT and multiple WAN interfaces Hi Jeff, No you're not the only one with this problem.. I'm having the same kind of problem, sadly though, no resolution as I was just coming here to post it myself. Instead of ppoe interfaces I just use two ethernet interfaces connected to two different ADSL routers. I'm also having trouble using DNAT. As you describe, only outside traffic targeting the "main" WAN interface is working correctly, traffic targeting the other interface get's natted ok but any return traffic is routed over the "main" WAN interface... I also fiddled with load balancing, but as far as I can see it forces you to use masquerade nat, in which case any SNAT rule does not work (As I understand it masquerade translated ANY AND ALL traffic with the IP of the interface it is masquerading on). So.. you're not alone in this, hopefully someone else can point us in a right direction... http://www.vyatta.org/node/4675

12/17

22/4/2014


Tijs Top

or to post comments

Permalink

Tue, 12/21/2010 15:22

bjbrock Not the answer but the reason... Routing decisions are made prior to SNAT while DNAT operates on the packets before routing. While this isn't the answer to your question it is the logic behind what is happening. ie. You cannot use NAT rules to route. Top

or to post comments

Permalink

Tue, 12/21/2010 23:05

kherona Issue with NAT and multiple WAN interfaces Hi Friend, According to Vyatta NAT documentation, the type of NAT should be masquerade in order to gain the benefit of using multiple interfaces. in Vyatta you have 3 types of NAT: source (based on source IP of incoming packets). destination (based on destination IP of incoming packets). masquerade (based on the interface the packets going to leave). (you need to make source-based-routing in order for this to work). Top

or to post comments

Wed, 12/22/2010 04:32

Permalink

Tijz Issue with NAT and multiple WAN interfaces Hi all, thanks for the replies http://www.vyatta.org/node/4675

13/17

22/4/2014


But none of these post explain if it should be possible. In theory it should though. Because in my case I want to replace our Draytek Vigor 3300 hardware appliance with a vyatta virtual appliance. The Draytek has no issues with SNAT using both interfaces. Surely, if Draytek can do it, vyatta can. I hope.... Top

or to post comments

Wed, 12/22/2010 09:05

Permalink

Singularity Issue with NAT and multiple WAN interfaces Ok, so after going over Vyatta and Netfilter / iproute2 documentation, it has become apparent that this is actually a problem that requires source based routing, something that Vyatta does not yet support through the CLI. That means that to get this to work, you will need to get your hands dirty and edit the routing table directly to add in the required source based rules. For example, to get around the problems I was having the following needs to be set up for rule 1070: This sets up four separate routing tables, one per interface. Each table has its interface as the default route sudo ip route add default dev pppoe0 table 1 sudo ip route add default dev pppoe1 table 2 sudo ip route add default dev pppoe2 table 3 sudo ip route add default dev pppoe3 table 4

Once this is done, we need to setup routing rules to send traffic to the correct routing table. The following will send packets coming from internal device 10.1.20.5 out through table 1, which has pppoe0 as its default route device. Please note that for this to work, any DNAT / SNAT rules in use MUST use the specified interface (in this case pppoe0). sudo ip rule add from 10.1.20.5 table 1

Unfortunately since this hasn't been done through the Vyatta CLI, the changes aren't persistent and will disappear on reboot. To fix this, you need to add anything you do you the file /etc/rc.d/rc.local - Here's an http://www.vyatta.org/node/4675

14/17

22/4/2014


example of the one I am testing with atm... echo "Setting up custom routing tables..." #Set up tables /sbin/ip route add default dev pppoe0 table 1 /sbin/ip route add default dev pppoe1 table 2 /sbin/ip route add default dev pppoe2 table 3 /sbin/ip route add default dev pppoe3 table 4 #Local LAN Internet traffic to go out pppoe3 /sbin/ip rule add from 10.1.100.0/24 table 4 #Email and other general server traffic to go out pppoe3 /sbin/ip rule add from 10.1.10.0/24 table 4 #All VoIP Server traffic to go out pppoe0 /sbin/ip rule add from 10.1.20.0/24 table 1

YMMV, but hopefully that helps you out a bit Tijz, has been driving me crazy for about a week now . . . Can any Vyatta employee's shed some light as to when sourced based routing will be available through the CLI? Being the one of the most requested features, It would sure make a lot of peoples lives easier. Can we expect it in the next release? Thanks, Jeff Top

or to post comments

Permalink

Fri, 12/24/2010 04:03

Tijz Issue with NAT and multiple WAN interfaces Hi Jeff, thanks for sharing your research on this! Will try it, allthough it makes the configuration not very portable which might become a problem... Tijs Top

http://www.vyatta.org/node/4675

or to post comments

15/17

22/4/2014


Permalink

Sat, 12/25/2010 05:48

Tijz Issue with NAT and multiple WAN interfaces So, I just set this up. Yes I know, it's christmas.. i had a few hours to kill before going to diner :) I immediately ran into trouble, as this source based routing does not solve my problem entirely. In the good old draytek I DNAT-ed port 25 from both WAN interface to my Exchange server. So that in case one of the WAN connections broke down, mail could be delivered through the other WAN connection. Using source based routing however (the way I understand it at the moment) I route EVERY traffic from one particular source, say my Exchange server, through a specific interface (probably different then the default gateway). So I am still not able to DNAT prt 25 from both WAN connections to one internal IP. Or am I missing something? Top

or to post comments

Sat, 12/25/2010 21:43

Permalink

Singularity Issue with NAT and multiple WAN interfaces Hi Tijz, That situation makes things a little more difficult, but its still possible. What you need to do instead is set up iptables to MARK packets when they enter the router based on address / interface, then route based on packet marking instead. I've done this with a CentOS install before and once set up it does work well. An example is explained pretty well in this blog post: http://nerdboys.com/2006/05/05/conning-the-mark-multiwanconnections-using-iptables-mark-connmark-and-iproute2/ (http://nerdboys.com/2006/05/05/conning-the-mark-multiwan-connectionsusing-iptables-mark-connmark-and-iproute2/)

Some other sites you might find useful: http://lartc.org/howto/lartc.netfilter.html http://www.vyatta.org/node/4675

16/17

22/4/2014


(http://lartc.org/howto/lartc.netfilter.html)

http://www.linuxtopia.org/Linux_Firewall_iptables/x4368.html (http://www.linuxtopia.org/Linux_Firewall_iptables/x4368.html)

Top

or to post comments

Permalink

Mon, 12/27/2010 00:51

Tijz Issue with NAT and multiple WAN interfaces Ok, thanks! will look into that Top

or to post comments

Vyatta.com (http://www.vyatta.com) | Support Center (http://www2.vyatta.com/support) | Vyatta University (http://www.vyatta.com/support/training) | Contact Us (http://www.vyatta.com/company/contact)

© 2012 Vyatta Inc. - All rights reserved. Vyatta and the Vyatta logo are registered trademarks of Vyatta, Inc. Other marks are trademarks of their respective holders.

http://www.vyatta.org/node/4675

17/17

Issue With NAT and Multiple WAN Interfaces _ Vyatta

Recommend Documents