Cisco 2851 NAT Load Balancing Problem (is solved!)

[Anton Pestov]

How to configure NAT load-balancing over two ISPs on Cisco 2851.

System image file is "flash:c2800nm-adventerprisek9-mz.151-4.M.bin"

My config (users behind the router/PAT unable to acccess to Internet, I don't known why?!!):

interface FastEthernet0/1/0.1101
 description *** INET: ISP1 ***
 bandwidth 100000
 encapsulation dot1Q 1101
 ip address 1.1.1.2 255.255.252
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 ip flow ingress
 ip flow egress
 ip nat outside
 ip virtual-reassembly in
 service-policy output POLICY-100M
 !
interface FastEthernet0/1/0.1103
 description *** INET: ISP2 ***
 bandwidth 50000
 encapsulation dot1Q 1103
 ip address 2.2.2.2 255.255.255.252
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 ip flow ingress
 ip flow egress
 ip nat outside
 ip virtual-reassembly in
 service-policy output POLICY-50M
!

interface GigabitEthernet0/0.12
 description *** LAN ***
 encapsulation dot1Q 12
 ip address 172.18.253.233 255.255.255.252
 ip pim sparse-mode
 ip nat inside
 ip virtual-reassembly in
 no cdp enable

!

ip route 0.0.0.0 0.0.0.0 1.1.1.1

ip route 0.0.0.0 0.0.0.0 2.2.2.1

!

ip nat pool POOL-ISP1 1.1.1.2 1.1.1.2 prefix-length 30
ip nat pool POOL-ISP2 2.2.2.2 2.2.2.2 prefix-length 30
ip nat inside source route-map RMAP-NAT-ISP1 pool POOL-ISP1 overload
ip nat inside source route-map RMAP-NAT-ISP2 pool POOL-ISP2 overload

!

route-map RMAP-NAT-ISP1 permit 10
 match ip address ACL-NAT
 match interface FastEthernet0/1/0.1101

oute-map RMAP-NAT-ISP2 permit 10
 match ip address ACL-NAT
 match interface FastEthernet0/1/0.1103
!

ip access-list extended ACL-NAT
 deny   ip object-group OBJNET-PRIVATE object-group OBJNET-PRIVATE
 permit ip object-group OBJNET-SR any
!

If "no ip route 0.0.0.0 0.0.0.0 2.2.2.1" the situation doesn't change, there is no Internet access.

...and working config (only through one ISP only ):

!<interface config all the same, see above>

!

ip route 0.0.0.0 0.0.0.0 1.1.1.1

!

ip nat pool POOL-ISP1 1.1.1.2 1.1.1.2 prefix-length 30
ip nat pool POOL-ISP2 2.2.2.2 2.2.2.2 prefix-length 30

ip nat inside source list ACL-NAT pool POOL-ISP1 overload

!

ip access-list extended ACL-NAT
 deny   ip object-group OBJNET-PRIVATE object-group OBJNET-PRIVATE
 permit ip object-group OBJNET-SR any
!

If  add "ip route 0.0.0.0 0.0.0.0 2.2.2.1" the situation change, there is no Internet access!.

What is wrong in first config?! How to do:

- if both ISPs avail.., then use both ISP for NAT for all inside hosts

- if one of ISPs unavail.., then use second ISP for NAT for all inside hosts

[Ricardo Munioz Cabero]

Config seems fine to me.

Perhaps the issue is in the ISP routers or above, they need the routes to your public IP addresses as well.

Also, remember there are several ways for the router to load balance the traffic: process-switching, fast-switching and CEF, I'm not sure but I think 2800 routers use fast-switching.

That means that if you start sendng a lot of packets to a remote node from only one host, all of those will go through the same route.

Farther, NAT doesn't make a new translation for every packet, it also uses a cache for packets which has already been translated, so if you start a download using the public address of ISP1 for the first packet, then all of the next packets (from the same flow) will use the same IP address, even if routed through another ISP.

I know this doesn't explain why ISP2 is not working, still, I hope it helps.

Regards.

[Anton Pestov]

Thanks for comments

I see, that router use ip CEF, but nevertheless it doesn't solve a problem. 

This is bug? There is no solutions?

[Ricardo Munioz Cabero]

I don´t think it´s a bug, just keep calm y perform a regular t-shoot procedure. As I said before, may be the problem is in the upstream routers.

You should check what the router does with some packets, specially with the ones that seems to be discarded. Check if they are being routed (the packets counter should increase in the interface), then check if they are being NATed.

You could also use identical ACLs (as you are doing) but with different names, so you can check if the counters increase in each of them.

Good luck.

[Anton Pestov]

1. I configured identical ACL-NAT's and reconfigured Route-Map's:

ip route 0.0.0.0 0.0.0.0 1.1.1.1

ip route 0.0.0.0 0.0.0.0 2.2.2.1

!

ip nat inside source route-map RMAP-NAT-ISP1 pool POOL-ISP1 overload
ip nat inside source route-map RMAP-NAT-ISP2 pool POOL-ISP2 overload

!

route-map RMAP-NAT-ISP1 permit 10
 match ip address ACL-NAT1
 match interface FastEthernet0/1/0.1101

oute-map RMAP-NAT-ISP2 permit 10
 match ip address ACL-NAT2
 match interface FastEthernet0/1/0.1103
!

The same problem - no Internet access, even if "no ip route 0.0.0.0 0.0.0.0 2.2.2.1" !

2. I configured identical ACL-NAT's and reconfigured ip nat translations:

ip route 0.0.0.0 0.0.0.0 1.1.1.1

ip route 0.0.0.0 0.0.0.0 2.2.2.1

!

ip nat inside source list ACL-NAT1 pool POOL-ISP1 overload
ip nat inside source list ACL-NAT2 pool POOL-ISP2 overload

!

The same problem - no Internet access, but if "no ip route 0.0.0.0 0.0.0.0 2.2.2.1" that everything works (but only through one provider)

[Ricardo Munioz Cabero]

My friend...

The idea of doing that was not intended to solve the problem, but to allow you to troubleshoot with more granular information...

I told you to check counters of the interfaces packets, ACLs matches, etc. There are commands to do that:

show interfaces

show ip access-list

sho ip nat translations

... etc.

you should already know how to use them and their options.

[Anton Pestov]

Router for translation of addresses only the first record "ip nat inside source list ..." is selected, even if there is the second record is configured. In ios configuration they are sorted in alphabetical order, and always only the first record is used for NAT.

[LukaszTJB]

Hello Anton,

ip nat source list only works with one ip nat outside interface. You have to work with route-maps if you want to use NAT over multiple WAN-links.

Try to use the interface statement in the ip nat config. I don't see any reason to work with IP-Pools here if you have only one WAN IP per Interface.

ip nat inside source route-map RMAP-NAT-ISP1 interface FastEthernet0/1/0.1101 overload

ip nat inside source route-map RMAP-NAT-ISP2 interface FastEthernet0/1/0.1103 overload

The route-maps you posted above look fine.

Are you sure the ACL 'ACL-NAT1' and 'ACL-NAT2' contains the correct IP Ranges of your clients? You can also work with only one ACL for both route-maps if the IP-Ranges are the same.

If you want to use both links equally you have to configure two default routes with the same metric. Otherwise you' ll use only one of your links.

[Anton Pestov]

Hello Lukasz

ACL-NAT1 and ACL-NAT2 are completely identical. And any of route-map's in general doesn't work (even if interface ip statement in ip nat configured, instead of pools):

"

1. I configured identical ACL-NAT's and reconfigured Route-Map's:

ip route 0.0.0.0 0.0.0.0 1.1.1.1

ip route 0.0.0.0 0.0.0.0 2.2.2.1

!

ip nat inside source route-map RMAP-NAT-ISP1 pool POOL-ISP1 overload
ip nat inside source route-map RMAP-NAT-ISP2 pool POOL-ISP2 overload

!

route-map RMAP-NAT-ISP1 permit 10
 match ip address ACL-NAT1
 match interface FastEthernet0/1/0.1101

oute-map RMAP-NAT-ISP2 permit 10
 match ip address ACL-NAT2
 match interface FastEthernet0/1/0.1103
!

The same problem - no Internet access, even if "no ip route 0.0.0.0 0.0.0.0 2.2.2.1" !

"

Actually I found out the following... balancing of default routes incorrectly works. All traffic goes through ISP1, even if two default gateways configured! My tests:

I run 'traceroute' with source ip 1.1.1.2 (interface from ISP1 subnet) to 8.8.8.8 - OK!

I run 'traceroute' with source ip 2.2.2.2 (interface from ISP2 subnet) to 8.8.8.8 - receive A! A! A! from 1.1.1.1. 

(PBR & ZBF doesn't configured)

[Ricardo Munioz Cabero]

Since both NAT ACLs are identical... you could just stop using them and have just the "match interface" statement within each route-map

Also, I repeat... check the counters of the several "show" commands you have available, don't just wait for a configuration change to suddenly work.

Below some verification commands you could use:

show route-map

show ip interface brief
!- Be sure the interface pointing to ISP2 is un up/up

show ip nat translations

[Anton Pestov]

The all problems is solved after update IOS from bugged c2800nm-adventerprisek9-mz.151-4.M.bin to lastest c2800nm-adventerprisek9-mz.151-4.M10.bin !

Сolleagues, thanks for the help!

[Ricardo Munioz Cabero]

Great to know that!

Still, it's a little weird since 'M' IOS are supposed to be more reliable.

[Ricardo Munioz Cabero]

My friend...

The idea of doing that was not intended to solve the problem, but to allow you to troubleshoot with more granular information...

I told you to check counters of the interfaces packets, ACLs matches, etc. There are commands to do that:

show interfaces

show ip access-list

sho ip nat translations

... etc.

you should already know how to use them and their options.