ASA 5510 Dual Internet connections - Deny reverse Path Check

[Lee Dress]

we currently have an internet connection for our main office via verizon fiber on our ASA 5510

all of our internet traffic and VPN Tunnels to our remote offices use this path.

we have acquired a second internet interface via comcast fiber.

I followed the instructions to setup a secondary ISP here:

http://www.cisco.com/c/en/us/support/docs/security/pix-500-series-security-appliances/70559-pix-dual-isp.html

my routes now look like this:

0.0.0.0 0.0.0.0 via verizon metric 1

0.0.0.0 0.0.0.0 via comcast metric 254

the problem I get is when I enable the interface connected to Comcast, half of my VPN Tunnels drop, and I get a ton of "Deny reverse path check on interface verizon"

I figured the metrics would take care of the issue.

LEt me know if anyone can help.

[Marvin Rhoads]

Are the two ISPs using separate IP address ranges? "Deny reverse path check" usually means that you have asymmetric routing (stateful connections with conversation replies coming back a different path than they went out).

[Lee Dress]

yes, the 2 ISPs are using different IP Ranges.

Verizon starts with a 60,

Comcast starts with a 50.

according to the document I followed, the 2 different metrics on the static routes should have eliminated things trying to follow different routes.

[Marvin Rhoads]

OK. You're right it should take care of the outbound routing. It's return traffic that the error message is complaining about.

Did you add a nat statement for the Comcast interface?

[Lee Dress]

it's definitely inbound traffic, not sure it's return traffic.

there are no NATs on the interface.  Just wondering why traffic is trying to come in via that interface. everything in my external network is pointing to the verizon IP.  I mean why would a site to site tunnel try to go through a different interface when it's been given a specific address to tunnel to?

[Marvin Rhoads]

Yes, inbound is more accurate. "Why " is a good question.

Are you using provider-assigned IP addressing in both cases? Is there any reason why Comcast would know to advertise your Verizon address block (the only reason I can think why Verizon-address-bound traffic should come in via that interface).

It's disruptive to your operations but if there's any way to get a capture while the error happens it would assist in troubleshooting.

[Lee Dress]

I think it's something in the ASA itself. the Verizon path is complaining about the reverse path.  I don't want to turn off reverse path verify because that will open me up to spoofing.  I do have enable traffic between 2 interfaces at same security level.  maybe that would do it?

[Lee Dress]

Would it help if I put in static routes to where my remote ASAs are?

Maybe that will eliminate the RPFs

[Marvin Rhoads]

It's worth a try.

Among static routes, longest prefix is chosen so /32 host routes are as specific as you can get. If it's the incoming traffic though, that's harder to manipulate.

[Lee Dress]

I think I finally heard what you were trying to say. 

the inbound connections were using the internet's path of least resistance, so my tunnels were trying to come in through the wrong gateway. on my remote offices that use comcast, the route to my ASA (on a verizon network) through comcast was less hops, so they came in through my comcast router.

the key here was to change the static routes on the remote ASAs to tell them the way to my internet address for the tunnel was through the gateway of that subnet, not through comcast's choice of routes.

Thank you for your help Marvin!