Inbound NAT issue with PBR on ASA

[tato386]

I have an ASA (9.6.3) with two interfaces connected to the Internet.  The ASA default route is pointing to ISP A and I have PAT and NAT using ISP A working fine.  I have a route-map using PBR that sets default next hop for certain clients to ISP B.  For the clients using ISP B I also have PAT and NAT setup.  PAT works fine and NAT works fine for _outbound_ traffic but I cannot get any inbound services to work. 

Test show that it is not a problem with rules or NAT because if I add a static route on the ASA that uses ISP B for a particular Internet IP the inbound works.  So I guess I need to add something else for NAT/PBR to work but I am not sure what.  Any ideas?

Thanks
Diego

[tato386]

I just ha the opportunity to try this on an ASA5515 running 9.12.3.9 and it worked like a charm.

Thank you CG!

Diego

[dbogdan]

good to hear. As you know it worked outbound, but not inbound for me. Wht did you do differently?

[tato386]

Not exactly my case.

My outbound was always working via PBR.  My issue was inbound and the floating static fixed it.

[dbogdan]

understood. Yes it works outbound and if you initiate from inside. Outside-in didn't work for me at initiation. The packet just dropped. thanks!

[chris-goulder]

The reason for this lays with the ASA connection table - once a tcp connection tuple is in the connection table that is used for applicable flows until the flow ends or the entry times out.

PBR is used to establish a flow using different routing to that in the general routing table, but once the flow is established and the entry in the connection table created, subsequent packets are handled just by the path identified in the connection table. PBR is not used once there is an entry in the connection table for a traffic flow

Also it is worth noting that typically when configuring PBR you are defining route policy for inside initiated traffic flows heading outside and that not have any bearing on outside initiated inbound traffic flows trying to head inside

So what is likely happening in your 'outside initiated traffic did not get through' case is along the lines of: -

1. tcp syn packet arrives at an outside i/f (but not the one holding the default route)

2. this allowed by the acl in place

3. this initiates the creation of an entry in the connection table for the tcp src|dst flow

4. syn packet forwarded to the inside host

5. inside host sends a syn/ack packet back to the source via the ASA

6. ASA checks its connection table and finds an entry for the tcp flow (entry added at 3. above)

7. As a connection table entry exists it is used and any PBR config does not come into play

8. ASA tries to forward the syn/ack reply packet out of the outside interface the flow came in on (connection table information)

9. this fails as there no applicable route in the ASA route table that would handle the flow on that i/f and the packet gets dropped

What you are doing by putting a floating static default route on your 'other' outside i/f is fixing the problem at step 9. in the above list. The ASA will see the floating default route on the interface and thus will be able to L3 forward it out of the outside interface the flow came in on

This behaviour can also crop up in asymmetric routing cases, whether PBR involved or not. The classic symptom being you can ping something but you cant get a tcp connection to establish. Depending on the scenario involved sometimes a floating static can help, other times you have to use a tcp state bypass policy (kludge) to fix it - but overall, unless of course you have no other option, the best solution is to try and fix/avoid the asymmetric routing in the first place as even if you find some config that will work it will likely break again later when something innocuous on the far side of your network changes

-Chris

[tato386]

Chris, you sure you are a "beginner"? ;)

[chris-goulder]

Well if Cisco say's so it must be true... :-)

[dbogdan]

Agreed.  I have abandoned using PBR as I was doing a test with placing an sd-wan router on a seperate interface  and pushing traffic out that door.  inbound traffic was not accepted as explained without actually creating static routes thus replacing the PBR idea altogether.  Thanks for your input on this.

---

@chris-goulder wrote:

The reason for this lays with the ASA connection table - once a tcp connection tuple is in the connection table that is used for applicable flows until the flow ends or the entry times out.

PBR is used to establish a flow using different routing to that in the general routing table, but once the flow is established and the entry in the connection table created, subsequent packets are handled just by the path identified in the connection table. PBR is not used once there is an entry in the connection table for a traffic flow

Also it is worth noting that typically when configuring PBR you are defining route policy for inside initiated traffic flows heading outside and that not have any bearing on outside initiated inbound traffic flows trying to head inside

So what is likely happening in your 'outside initiated traffic did not get through' case is along the lines of: -

1. tcp syn packet arrives at an outside i/f (but not the one holding the default route)

2. this allowed by the acl in place

3. this initiates the creation of an entry in the connection table for the tcp src|dst flow

4. syn packet forwarded to the inside host

5. inside host sends a syn/ack packet back to the source via the ASA

6. ASA checks its connection table and finds an entry for the tcp flow (entry added at 3. above)

7. As a connection table entry exists it is used and any PBR config does not come into play

8. ASA tries to forward the syn/ack reply packet out of the outside interface the flow came in on (connection table information)

9. this fails as there no applicable route in the ASA route table that would handle the flow on that i/f and the packet gets dropped

What you are doing by putting a floating static default route on your 'other' outside i/f is fixing the problem at step 9. in the above list. The ASA will see the floating default route on the interface and thus will be able to L3 forward it out of the outside interface the flow came in on

This behaviour can also crop up in asymmetric routing cases, whether PBR involved or not. The classic symptom being you can ping something but you cant get a tcp connection to establish. Depending on the scenario involved sometimes a floating static can help, other times you have to use a tcp state bypass policy (kludge) to fix it - but overall, unless of course you have no other option, the best solution is to try and fix/avoid the asymmetric routing in the first place as even if you find some config that will work it will likely break again later when something innocuous on the far side of your network changes

-Chris

---