cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
3333
Views
5
Helpful
8
Replies

Cisco ASA 5545-X 9.4 - Return SMTP traffic "routing failed to locate next hop for TCP"

Mike Pennycook
Level 1
Level 1

Hi All, 

I recently posted a question here on SIP traffic not being able to go outside when a route map was used. Because this traffic was initially going through a different interface it could've hit a bug CSCuv00272 (as described in https://supportforums.cisco.com/discussion/13213941/cisco-asa-94-default-routing-route-mapnat)

However on this occasion, with a similar setup, there is no existing connection there to trigger the bug (new connection setup for the current flow in question). 

We have a connection to an email security provider out on port 25. When our email server initiates a connection from the inside out to the email security provider on port 25, the test passes. When we get the email security provider to initiate a connection from their end and test if they can reach our email servers on port 25, it seems the ASA allows the connection through, but on the return traffic going back outside it has an issue with routing. 

"Routing failed to locate next hop for TCP from INSIDE:x.x.x.x/25 to OUTSIDE2-SUBINTERFACE:x.x.x.x/3406"

The route-map is doing this:

route-map ROUTE-MAP, permit, sequence 20

  Match clauses:

    ip address (access-lists): Email-to-ISP-1

  Set clauses:

    ip next-hop X.X.X.X

route-map ROUTE-MAP, permit, sequence 30

  Match clauses:

    ip address (access-lists): Email-to-ISP-2

  Set clauses:

    ip next-hop X.X.X.X

The above two next hops are on the outside2 interface, however ISP1 and ISP2 are on two different subinterfaces. 

NAT config:

object network Email-Server-1

nat (INSIDE,OUTSIDE-SUBINTERFACE1) static interface

object network Email-Server-2

nat (INSIDE,OUTSIDE-SUBINTERFACE2) static interface

Using a debug to see how the route map is working I can see in blue an example in the debug output for a source IP address that correctly matched the ACL and decided the correct next hop/outgoing interface. In red I can see the route map not allowing traffic for a different source IP through as expected, because that flow didn't match the ACL in the route-map:

pbr: policy based route lookup called for x.x.x.x/12121 to x.x.x.x/443 proto 6 sub_proto 0 received on interface INSIDE

pbr: First matching rule from ACL(17)

pbr: route map ROUTE-MAP, sequence 30, permit; proceed with policy routing

pbr: evaluating next-hop x.x.x.x

pbr: policy based routing applied; egress_ifc = OUTSIDE-SUBINTERFACE-1 : next_hop = x.x.x.x

pbr: policy based route lookup called for x.x.x.x/23467 to x.x.x.x/53 proto 17 sub_proto 0 received on interface INSIDE

pbr: no route policy found; skip to normal route lookup

pbr: policy based route lookup called for x.x.x.x/4443 to x.x.x.x/42310 proto 6 sub_proto 0 received on interface INSIDE

pbr: no route policy found; skip to normal route lookup

Packet tracer shows that when our email security provider initiates a connection to our email servers on port 24, the NAT process is working fine, the above debugs seem to suggest the route map is working fine, but routing still cannot 'locate' the next hop. 

If I put a static route to reach the email security provider out one of the outside subinterfaces, this works perfectly. But the point of the route-map is to get traffic from one email server to go through via a different ISP and another email server to go through a different ISP. The next hop gateway is directly connected on the subnet of the outside subinterfaces. 

Is there any other debugs that could help us determine what logic the ASA is using and subsequently failing to determine the next hop?

1 Accepted Solution

Accepted Solutions

I thought I had replied back earlier, bit looks like it did not go through. The route just add's another route into the ASP routing table so that the ASA can find a next-hop to the destination interface based on the existing connection. PBR does not help in determining next hop if it for traffic matching an existing connection.

All other traffic should hit default route as only that will show up in the global routing table.

View solution in original post

8 Replies 8

Mike Pennycook
Level 1
Level 1

Just to add, i believe the route map debug logs which show the route map working as expected is for connections initiated from the inside. 

I'm not sure if a connection is initiated from the outside, then on it's return traffic, is this something which the route-map will be called upon? Or is the route-map only for connections initiated from the inside?

I believe that this is a manifestation of the same bug. When you initiate from out to in, there is a connection built for the SYN traffic and that goes through correctly. When the return SYN-ACK packet comes in, it hits the PBR - but since there is already a connection for this, it does not use PBR to determine the route, but tries to look at the routing table. If there is no route, you will see this issue.

I guess one option you can do is to create a default route to your secondary outside interface with a higher metric ( so that it does not affect your existing default route).

route OUTSIDE2-SUBINTERFACE 0 0 <next-hop> 240

This way, when it matches the connection, there is a route present in the ASP routing table to send it to the secondary interface.

Hi Rahul, 

Thanks for your response!

Before I try this - 

Having lets say 3 default routes, meaning 1 can be the main global one, then one for ISP1 and one for ISP2. 

Will that default route still come into use because of it's higher metric?

In that case does it just function as another route?

also, just referring to connection status, when we try with a test laptop, with the same error:

ASA5545-X# show conn detail address X.X.X.X
7112 in use, 16666 most used
Flags: A - awaiting inside ACK to SYN, a - awaiting outside ACK to SYN,
B - initial SYN from outside, b - TCP state-bypass or nailed,
C - CTIQBE media, c - cluster centralized,
D - DNS, d - dump, E - outside back connection, e - semi-distributed,
F - outside FIN, f - inside FIN,
G - group, g - MGCP, H - H.323, h - H.225.0, I - inbound data,
i - incomplete, J - GTP, j - GTP data, K - GTP t3-response
k - Skinny media, M - SMTP data, m - SIP media, n - GUP
O - outbound data, P - inside back connection,
q - SQL*Net data, R - outside acknowledged FIN,
R - UDP SUNRPC, r - inside acknowledged FIN, S - awaiting inside SYN,
s - awaiting outside SYN, T - SIP, t - SIP transient, U - up,
V - VPN orphan, W - WAAS,
w - secondary domain backup,
X - inspected by service module,
x - per session, Y - director stub flow, y - backup stub flow,
Z - Scansafe redirection, z - forwarding stub flow

TCP OUTSIDE-SUBINTERFACE-1: X.X.X.X/1816 INSIDE: X.X.X.X/25,
flags aB , idle 1s, uptime 4s, timeout 30s, bytes 0

ASA5545-X#

Then, once the timeout values get to this:

flags aB , idle 5s, uptime 17s, timeout 30s, bytes 0

The connection disappears 

I thought I had replied back earlier, bit looks like it did not go through. The route just add's another route into the ASP routing table so that the ASA can find a next-hop to the destination interface based on the existing connection. PBR does not help in determining next hop if it for traffic matching an existing connection.

All other traffic should hit default route as only that will show up in the global routing table.

Hi Rahul, 

Thanks for your help, this worked out very well!

Just wondering in a failure scenario where the original default gateway next hop is unreachable via the original outside interface, will the additional default routes entered be elevated to the global routing table?

Thanks!

Since these are static routes, the original route will only go away from the routing table if the interface is down. To bring down the original default gateway when next-hop is not reachable, you would have to use SLA monitoring and tracking to remove it from the routing table.

http://www.cisco.com/c/en/us/support/docs/security/asa-5500-x-series-next-generation-firewalls/118962-configure-asa-00.html

Thank you Rahul.
I was really going crazy with troubleshooting until I read your post. Same kind of situation. Connection from inside out where going through second wan interface thanks to the PBR and connections from outside in kept failing.
I just add the 2de default route and voilà !!!
I can not thank you enough.
F.
Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: