02-03-2017 12:45 AM - edited 03-12-2019 01:52 AM
Hi All,
I recently posted a question here on SIP traffic not being able to go outside when a route map was used. Because this traffic was initially going through a different interface it could've hit a bug CSCuv00272 (as described in https://supportforums.cisco.com/discussion/13213941/cisco-asa-94-default-routing-route-mapnat)
However on this occasion, with a similar setup, there is no existing connection there to trigger the bug (new connection setup for the current flow in question).
We have a connection to an email security provider out on port 25. When our email server initiates a connection from the inside out to the email security provider on port 25, the test passes. When we get the email security provider to initiate a connection from their end and test if they can reach our email servers on port 25, it seems the ASA allows the connection through, but on the return traffic going back outside it has an issue with routing.
"Routing failed to locate next hop for TCP from INSIDE:x.x.x.x/25 to OUTSIDE2-SUBINTERFACE:x.x.x.x/3406"
The route-map is doing this:
route-map ROUTE-MAP, permit, sequence 20
Match clauses:
ip address (access-lists): Email-to-ISP-1
Set clauses:
ip next-hop X.X.X.X
route-map ROUTE-MAP, permit, sequence 30
Match clauses:
ip address (access-lists): Email-to-ISP-2
Set clauses:
ip next-hop X.X.X.X
The above two next hops are on the outside2 interface, however ISP1 and ISP2 are on two different subinterfaces.
NAT config:
object network Email-Server-1
nat (INSIDE,OUTSIDE-SUBINTERFACE1) static interface
object network Email-Server-2
nat (INSIDE,OUTSIDE-SUBINTERFACE2) static interface
Using a debug to see how the route map is working I can see in blue an example in the debug output for a source IP address that correctly matched the ACL and decided the correct next hop/outgoing interface. In red I can see the route map not allowing traffic for a different source IP through as expected, because that flow didn't match the ACL in the route-map:
pbr: policy based route lookup called for x.x.x.x/12121 to x.x.x.x/443 proto 6 sub_proto 0 received on interface INSIDE
pbr: First matching rule from ACL(17)
pbr: route map ROUTE-MAP, sequence 30, permit; proceed with policy routing
pbr: evaluating next-hop x.x.x.x
pbr: policy based routing applied; egress_ifc = OUTSIDE-SUBINTERFACE-1 : next_hop = x.x.x.x
pbr: policy based route lookup called for x.x.x.x/23467 to x.x.x.x/53 proto 17 sub_proto 0 received on interface INSIDE
pbr: no route policy found; skip to normal route lookup
pbr: policy based route lookup called for x.x.x.x/4443 to x.x.x.x/42310 proto 6 sub_proto 0 received on interface INSIDE
pbr: no route policy found; skip to normal route lookup
Packet tracer shows that when our email security provider initiates a connection to our email servers on port 24, the NAT process is working fine, the above debugs seem to suggest the route map is working fine, but routing still cannot 'locate' the next hop.
If I put a static route to reach the email security provider out one of the outside subinterfaces, this works perfectly. But the point of the route-map is to get traffic from one email server to go through via a different ISP and another email server to go through a different ISP. The next hop gateway is directly connected on the subnet of the outside subinterfaces.
Is there any other debugs that could help us determine what logic the ASA is using and subsequently failing to determine the next hop?
Solved! Go to Solution.
02-05-2017 08:42 PM
I thought I had replied back earlier, bit looks like it did not go through. The route just add's another route into the ASP routing table so that the ASA can find a next-hop to the destination interface based on the existing connection. PBR does not help in determining next hop if it for traffic matching an existing connection.
All other traffic should hit default route as only that will show up in the global routing table.
02-03-2017 01:23 AM
Just to add, i believe the route map debug logs which show the route map working as expected is for connections initiated from the inside.
I'm not sure if a connection is initiated from the outside, then on it's return traffic, is this something which the route-map will be called upon? Or is the route-map only for connections initiated from the inside?
02-03-2017 05:42 AM
I believe that this is a manifestation of the same bug. When you initiate from out to in, there is a connection built for the SYN traffic and that goes through correctly. When the return SYN-ACK packet comes in, it hits the PBR - but since there is already a connection for this, it does not use PBR to determine the route, but tries to look at the routing table. If there is no route, you will see this issue.
I guess one option you can do is to create a default route to your secondary outside interface with a higher metric ( so that it does not affect your existing default route).
route OUTSIDE2-SUBINTERFACE 0 0 <next-hop> 240
This way, when it matches the connection, there is a route present in the ASP routing table to send it to the secondary interface.
02-03-2017 05:56 AM
Hi Rahul,
Thanks for your response!
Before I try this -
Having lets say 3 default routes, meaning 1 can be the main global one, then one for ISP1 and one for ISP2.
Will that default route still come into use because of it's higher metric?
In that case does it just function as another route?
02-03-2017 06:06 AM
also, just referring to connection status, when we try with a test laptop, with the same error:
ASA5545-X# show conn detail address X.X.X.X
7112 in use, 16666 most used
Flags: A - awaiting inside ACK to SYN, a - awaiting outside ACK to SYN,
B - initial SYN from outside, b - TCP state-bypass or nailed,
C - CTIQBE media, c - cluster centralized,
D - DNS, d - dump, E - outside back connection, e - semi-distributed,
F - outside FIN, f - inside FIN,
G - group, g - MGCP, H - H.323, h - H.225.0, I - inbound data,
i - incomplete, J - GTP, j - GTP data, K - GTP t3-response
k - Skinny media, M - SMTP data, m - SIP media, n - GUP
O - outbound data, P - inside back connection,
q - SQL*Net data, R - outside acknowledged FIN,
R - UDP SUNRPC, r - inside acknowledged FIN, S - awaiting inside SYN,
s - awaiting outside SYN, T - SIP, t - SIP transient, U - up,
V - VPN orphan, W - WAAS,
w - secondary domain backup,
X - inspected by service module,
x - per session, Y - director stub flow, y - backup stub flow,
Z - Scansafe redirection, z - forwarding stub flow
TCP OUTSIDE-SUBINTERFACE-1: X.X.X.X/1816 INSIDE: X.X.X.X/25,
flags aB , idle 1s, uptime 4s, timeout 30s, bytes 0
ASA5545-X#
Then, once the timeout values get to this:
flags aB , idle 5s, uptime 17s, timeout 30s, bytes 0
The connection disappears
02-05-2017 08:42 PM
I thought I had replied back earlier, bit looks like it did not go through. The route just add's another route into the ASP routing table so that the ASA can find a next-hop to the destination interface based on the existing connection. PBR does not help in determining next hop if it for traffic matching an existing connection.
All other traffic should hit default route as only that will show up in the global routing table.
02-06-2017 10:02 AM
Hi Rahul,
Thanks for your help, this worked out very well!
Just wondering in a failure scenario where the original default gateway next hop is unreachable via the original outside interface, will the additional default routes entered be elevated to the global routing table?
Thanks!
02-06-2017 10:07 AM
Since these are static routes, the original route will only go away from the routing table if the interface is down. To bring down the original default gateway when next-hop is not reachable, you would have to use SLA monitoring and tracking to remove it from the routing table.
http://www.cisco.com/c/en/us/support/docs/security/asa-5500-x-series-next-generation-firewalls/118962-configure-asa-00.html
07-04-2020 10:06 AM
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide