NAT, Policy Based Routing, Multiple ISPs

[mnleblanc]

Hello everyone,

I have a 3640 router with 4 fastethernet interfaces:

fa1/0 is my LAN facing, inside nat

fa2/0 is my main WAN / ISP connection to AT&T, outside nat, connects to a 2821

fa3/0 is my second WAN / ISP connection to Comcast Business, outside nat, connects to an SMC broadband gateway

fa3/1 is my third WAN / ISP connection to Comcast Residential, outside nat, connects to an 871W and then a broadband modem

Everything works fine through fa2/0.

I am experimenting with policy based routing, simply using the source network as a match in the route-map, to push certain traffic to either fa3/0 or fa3/1.  The PBR works fine.  I can ping the gateway addresses on the SMC and 871W routers from a LAN side host (10.1.5.1) coming in through fa1/0, where I am applying the IP policy.

The problem develops when I try to actually connect to the Internet through fa3/0--->SMC or fa3/1--->871W.  If I try to ping Yahoo.com from the inside host, (10.1.5.1) the request times out on the host and the router debug ip nat only shows the outgoing translation.  However, a Wireshark capture shows the returning echo reply with the proper return addresses (mac and IP) of the 3640 fa3/0 or 3/1 interface.  I have also tried a debug ip nat detailed and debug ip packet, but I am not seeing (or at least not recognizing) the problem.

I have static routes to the internal networks on both the SMC and the 871W.

It is not an access list issue, as far as I can see.

I have attached the running-config from the 3640 and some host command line output.

The problem seems "textbook", but I been looking around for a while and can't seem to find a document that addresses this issue specifically enough.

Thanks in advance for any help or guidance,

Mark

[Jon Marshall · Accepted solution]

mnleblanc wrote:
Hello everyone,
I have a 3640 router with 4 fastethernet interfaces:
fa1/0 is my LAN facing, inside nat
fa2/0 is my main WAN / ISP connection to AT&T, outside nat, connects to a 2821
fa3/0 is my second WAN / ISP connection to Comcast Business, outside nat, connects to an SMC broadband gateway
fa3/1 is my third WAN / ISP connection to Comcast Residential, outside nat, connects to an 871W and then a broadband modem
Everything works fine through fa2/0.
I am experimenting with policy based routing, simply using the source network as a match in the route-map, to push certain traffic to either fa3/0 or fa3/1.  The PBR works fine.  I can ping the gateway addresses on the SMC and 871W routers from a LAN side host (10.1.5.1) coming in through fa1/0, where I am applying the IP policy.
The problem develops when I try to actually connect to the Internet through fa3/0--->SMC or fa3/1--->871W.  If I try to ping Yahoo.com from the inside host, (10.1.5.1) the request times out on the host and the router debug ip nat only shows the outgoing translation.  However, a Wireshark capture shows the returning echo reply with the proper return addresses (mac and IP) of the 3640 fa3/0 or 3/1 interface.  I have also tried a debug ip nat detailed and debug ip packet, but I am not seeing (or at least not recognizing) the problem.
I have static routes to the internal networks on both the SMC and the 871W.
It is not an access list issue, as far as I can see.
I have attached the running-config from the 3640 and some host command line output.
The problem seems "textbook", but I been looking around for a while and can't seem to find a document that addresses this issue specifically enough.
Thanks in advance for any help or guidance,
Mark

Mark

Could you clarify something for me. I appreciate you see a return reply from google but -

fa2/0 has a public IP ie. 12.x.x.194 so any private addressing such as 10.1.5.1 will be natted to the 12.x.x.x194 address. This is a routable address on the Internet so all works fine as the traffic can be routed back to you.

But fa3/0 and fa3/1 are using 192.168.x.x addressing which is not routable on the Internet so where is the NAT taking place for those links. If there is no NAT further upstream that changes the 192.168.x.x address to a public address then you will not be able to access the Internet over those links.

Jon

[mnleblanc]

Jon,

Thanks for your response.

fa3/0 sends traffic to a Comcast Business Class SMC gateway (integrated broadband modem and router) which nats to a static 75.x.x.126

fa3/1 sends traffic to a 871W that is connected to a Comcast residential broadband modem and nats to a dynamic public IP.

I have fa3/0 and fa3/1 nat so that they appear as hosts on SMC and 871W inside LANs.

Mark

[Jon Marshall · Accepted solution]

mnleblanc wrote:
Jon,
Thanks for your response.
fa3/0 sends traffic to a Comcast Business Class SMC gateway (integrated broadband modem and router) which nats to a static 75.x.x.126
fa3/1 sends traffic to a 871W that is connected to a Comcast residential broadband modem and nats to a dynamic public IP.
I have fa3/0 and fa3/1 nat so that they appear as hosts on SMC and 871W inside LANs.
Mark

Mark

Can you try something out for me.

On the the fa3/0 interface which is where 10.1.5.x is policy routed can you remove the "ip verify unicast reverse-path" and test again.

Jon

[mnleblanc]

Jon,

That was it!  I have complete connectivity through fa3/0 and fa3/1 now.  Thanks for helping me with this issue.

Mark