Routing failed to locate next hop - ASA 5505 l2l VPN

[andy_4578]

We have a 5505 (Soon to be replaced with two 5515-x) firewall with two l2l vpns.

Were trying to allow one remote site traffic to flow through to the other remote site but the syslog shows "

				10.5.25.4	1	172.16.10.10	0	Routing failed to locate next hop for ICMP from outside:10.5.25.4/1 to inside:172.16.10.10/0
Config is below : ASA Version 8.4(3) names ! interface Ethernet0/0 switchport access vlan 2 speed 100 duplex full ! interface Ethernet0/1 ! interface Ethernet0/2 ! interface Ethernet0/3 ! interface Ethernet0/4 ! interface Ethernet0/5 ! interface Ethernet0/6 <--- More ---> ! interface Ethernet0/7 switchport access vlan 10 ! interface Vlan1 nameif inside security-level 100 allow-ssc-mgmt ip address 10.5.19.254 255.255.255.0 ! interface Vlan2 description WIMAX Interface nameif outside security-level 0 ip address x.247.x.18 255.255.255.248 ! ftp mode passive clock timezone GMT 1 same-security-traffic permit inter-interface same-security-traffic permit intra-interface object network obj_any subnet 0.0.0.0 0.0.0.0 object network guestwifi subnet 10.1.110.0 255.255.255.0 <--- More ---> object network NETWORK_OBJ_10.5.19.0_24 subnet 10.5.19.0 255.255.255.0 object network NETWORK_OBJ_10.5.31.0_24 subnet 10.5.31.0 255.255.255.0 object network NETWORK_OBJ_172.16.0.0_16 subnet 172.16.0.0 255.255.0.0 object network DS365-Cloud subnet 172.16.10.0 255.255.255.0 description DS365-Cloud object network Inside-network-16 subnet 10.5.0.0 255.255.0.0 object network atanta subnet 10.5.16.0 255.255.255.0 description atanta object network guest_dyn_nat subnet 10.5.29.0 255.255.255.0 object network NETWORK_OBJ_172.16.254.0_25 subnet 172.16.254.0 255.255.255.128 object network NETWORK_OBJ_10.5.16.0_20 subnet 10.5.16.0 255.255.240.0 object network NETWORK_OBJ_10.5.16.0_26 subnet 10.5.16.0 255.255.255.192 object network LDAP_DC7 host 10.5.21.1 <--- More ---> description LDAP object network c2si range 10.5.21.180 10.5.21.200 object network NETWORK_OBJ_10.5.25.0_24 subnet 10.5.25.0 255.255.255.0 object-group network rfc1918 network-object 192.168.0.0 255.255.0.0 network-object 172.16.0.0 255.255.240.0 network-object 10.0.0.0 255.0.0.0 object-group network DM_INLINE_NETWORK_1 network-object 10.5.19.0 255.255.255.0 network-object 10.5.20.0 255.255.254.0 network-object 10.5.22.0 255.255.255.0 network-object 10.5.30.0 255.255.255.0 network-object 192.168.100.0 255.255.255.0 object-group network Sure_Signal network-object x.183.x.128 255.255.255.192 network-object host x.183.133.177 network-object host x.183.133.178 network-object host x.183.133.179 network-object host x.183.133.181 network-object host x.183.133.182 object-group network LDAP_source_networks network-object 135.196.24.192 255.255.255.240 <--- More ---> network-object 195.130.x.0 255.255.255.0 network-object x.2.3.128 255.255.255.192 network-object 213.235.63.64 255.255.255.192 network-object 91.220.42.0 255.255.255.0 network-object 94.x.240.0 255.255.255.0 network-object 94.x.x.0 255.255.255.0 object-group network c2si_Allow network-object host 10.5.16.1 network-object host 10.5.21.1 network-object object c2si object-group network DM_INLINE_NETWORK_2 network-object 10.5.20.0 255.255.254.0 network-object 10.5.21.0 255.255.255.0 network-object 10.5.22.0 255.255.255.0 network-object 10.5.29.0 255.255.255.0 network-object object NETWORK_OBJ_10.5.19.0_24 object-group network DM_INLINE_NETWORK_3 network-object 10.5.19.0 255.255.255.0 network-object 10.5.20.0 255.255.254.0 network-object 10.5.21.0 255.255.255.0 network-object 10.5.22.0 255.255.255.0 network-object object atanta object-group network DM_INLINE_NETWORK_4 network-object 10.5.20.0 255.255.254.0 <--- More ---> network-object 10.5.21.0 255.255.255.0 network-object 10.5.22.0 255.255.255.0 network-object 10.5.23.0 255.255.255.0 network-object 10.5.30.0 255.255.255.0 network-object object NETWORK_OBJ_10.5.19.0_24 network-object object atanta network-object object DS365-Cloud access-list inside_access_in extended permit tcp any object-group Sure_Signal eq 50 access-list inside_access_in extended permit tcp any object-group Sure_Signal eq pptp access-list inside_access_in extended permit gre any object-group Sure_Signal access-list inside_access_in extended permit udp any object-group Sure_Signal eq ntp access-list inside_access_in extended permit icmp any object-group Sure_Signal echo access-list inside_access_in extended permit udp any object-group Sure_Signal eq 50 access-list inside_access_in extended permit udp any object-group Sure_Signal eq 4500 access-list inside_access_in extended permit udp any object-group Sure_Signal eq isakmp access-list inside_access_in extended permit ip any any access-list clientvpn extended permit ip 10.5.0.0 255.255.0.0 10.5.30.0 255.255.255.0 access-list BerkeleyAdmin-clientvpn extended permit ip 10.5.0.0 255.255.0.0 10.5.30.0 255.255.255.0 access-list BerkeleyUser-clientvpn extended permit ip 10.5.21.0 255.255.255.0 10.5.30.0 255.255.255.0 access-list outside_cryptomap extended permit ip object Inside-network-16 10.5.25.0 255.255.255.0 access-list guest_access_in extended permit ip 10.5.29.0 255.255.255.0 any access-list state_bypass extended permit tcp 192.168.100.0 255.255.255.0 10.5.30.0 255.255.255.0 log access-list state_bypass extended permit tcp 10.5.30.0 255.255.255.0 192.168.100.0 255.255.255.0 log access-list state_bypass extended permit tcp 10.5.29.0 255.255.255.0 10.5.30.0 255.255.255.0 log <--- More ---> access-list state_bypass extended permit tcp 10.5.30.0 255.255.255.0 10.5.29.0 255.255.255.0 log access-list outside_access_in extended permit icmp any any access-list outside_cryptomap_1 extended permit ip 10.5.16.0 255.255.240.0 10.5.16.0 255.255.255.192 access-list global_access extended permit tcp object-group LDAP_source_networks host 10.5.21.1 eq ldap access-list outside_cryptomap_2 extended permit ip 10.5.0.0 255.255.0.0 object DS365-Cloud access-list outside_cryptomap_3 extended permit ip object-group DM_INLINE_NETWORK_4 10.5.25.0 255.255.255.0 pager lines 24 logging enable logging buffer-size 100000 logging console debugging logging buffered debugging logging asdm informational mtu inside 1500 mtu outside 1500 ip local pool clientvpn 10.5.30.1-10.5.30.100 ip local pool VPN_IP_Pool 172.16.254.1-172.16.254.100 mask 255.255.255.0 no failover icmp unreachable rate-limit 1 burst-size 1 icmp permit any inside icmp permit any outside no asdm history enable arp timeout 14400 nat (inside,outside) source static rfc1918 rfc1918 destination static rfc1918 rfc1918 nat (inside,outside) source static NETWORK_OBJ_10.5.19.0_24 NETWORK_OBJ_10.5.19.0_24 destination static NETWORK_OBJ_10.5.31.0_24 NETWORK_OBJ_10.5.31.0_24 no-proxy-arp route-lookup <--- More ---> nat (inside,outside) source static NETWORK_OBJ_10.5.19.0_24 NETWORK_OBJ_10.5.19.0_24 destination static NETWORK_OBJ_10.5.19.0_24 NETWORK_OBJ_10.5.19.0_24 no-proxy-arp route-lookup nat (inside,outside) source static Inside-network-16 Inside-network-16 destination static DS365-Cloud DS365-Cloud no-proxy-arp route-lookup nat (inside,outside) source static DM_INLINE_NETWORK_1 DM_INLINE_NETWORK_1 destination static NETWORK_OBJ_172.16.254.0_25 NETWORK_OBJ_172.16.254.0_25 no-proxy-arp route-lookup nat (inside,outside) source static NETWORK_OBJ_10.5.16.0_20 NETWORK_OBJ_10.5.16.0_20 destination static NETWORK_OBJ_10.5.16.0_26 NETWORK_OBJ_10.5.16.0_26 no-proxy-arp route-lookup nat (inside,outside) source static c2si_Allow c2si_Allow destination static NETWORK_OBJ_172.16.254.0_25 NETWORK_OBJ_172.16.254.0_25 no-proxy-arp route-lookup nat (inside,outside) source static atanta atanta destination static NETWORK_OBJ_10.5.25.0_24 NETWORK_OBJ_10.5.25.0_24 no-proxy-arp route-lookup nat (inside,outside) source static DS365-Cloud DS365-Cloud destination static NETWORK_OBJ_10.5.25.0_24 NETWORK_OBJ_10.5.25.0_24 no-proxy-arp route-lookup nat (inside,outside) source static DM_INLINE_NETWORK_2 DM_INLINE_NETWORK_2 destination static NETWORK_OBJ_10.5.25.0_24 NETWORK_OBJ_10.5.25.0_24 no-proxy-arp route-lookup nat (inside,outside) source static NETWORK_OBJ_10.5.25.0_24 NETWORK_OBJ_10.5.25.0_24 destination static DS365-Cloud DS365-Cloud no-proxy-arp route-lookup nat (inside,outside) source static DM_INLINE_NETWORK_3 DM_INLINE_NETWORK_3 destination static DS365-Cloud DS365-Cloud no-proxy-arp route-lookup nat (inside,outside) source static Inside-network-16 Inside-network-16 destination static NETWORK_OBJ_10.5.25.0_24 NETWORK_OBJ_10.5.25.0_24 no-proxy-arp route-lookup nat (inside,outside) source static DM_INLINE_NETWORK_4 DM_INLINE_NETWORK_4 destination static NETWORK_OBJ_10.5.25.0_24 NETWORK_OBJ_10.5.25.0_24 no-proxy-arp route-lookup ! object network obj_any nat (inside,outside) dynamic interface object network LDAP_DC7 nat (inside,outside) static 194.247.x.19 service tcp ldap ldap access-group inside_access_in in interface inside access-group outside_access_in in interface outside access-group global_access global ! router eigrp 143 no auto-summary network 10.5.19.0 255.255.255.0 <--- More ---> network 10.5.29.0 255.255.255.0 network 10.5.30.0 255.255.255.0 redistribute static ! route outside 0.0.0.0 0.0.0.0 194.247.x.17 1 track 1 route inside 10.5.16.0 255.255.255.0 10.5.19.252 1 timeout xlate 3:00:00 timeout pat-xlate 0:00:30 timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02 timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00 timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00 timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute timeout tcp-proxy-reassembly 0:01:00 timeout floating-conn 0:00:00 dynamic-access-policy-record DfltAccessPolicy aaa-server group protocol radius aaa-server group (inside) host 10.5.21.1 key ***** aaa-server group (inside) host 10.5.16.1 key ***** user-identity default-domain LOCAL aaa authentication ssh console LOCAL aaa authentication telnet console LOCAL http server enable <--- More ---> http 192.168.1.0 255.255.255.0 inside http 10.5.16.0 255.255.240.0 inside no snmp-server location no snmp-server contact snmp-server enable traps snmp authentication linkup linkdown coldstart warmstart sysopt connection tcpmss 1350 sla monitor 1 type echo protocol ipIcmpEcho 8.8.4.4 interface outside sla monitor schedule 1 life forever start-time now crypto ipsec ikev1 transform-set strong-comp esp-aes-256 esp-sha-hmac crypto ipsec ikev1 transform-set strong esp-aes-256 esp-sha-hmac crypto ipsec ikev1 transform-set ESP-AES-128-SHA esp-aes esp-sha-hmac crypto ipsec ikev1 transform-set ESP-AES-128-MD5 esp-aes esp-md5-hmac crypto ipsec ikev1 transform-set ESP-AES-192-SHA esp-aes-192 esp-sha-hmac crypto ipsec ikev1 transform-set ESP-AES-192-MD5 esp-aes-192 esp-md5-hmac crypto ipsec ikev1 transform-set ESP-AES-256-SHA esp-aes-256 esp-sha-hmac crypto ipsec ikev1 transform-set ESP-AES-256-MD5 esp-aes-256 esp-md5-hmac crypto ipsec ikev1 transform-set ESP-3DES-SHA esp-3des esp-sha-hmac crypto ipsec ikev1 transform-set ESP-3DES-MD5 esp-3des esp-md5-hmac crypto ipsec ikev1 transform-set ESP-DES-SHA esp-des esp-sha-hmac crypto ipsec ikev1 transform-set ESP-DES-MD5 esp-des esp-md5-hmac crypto ipsec ikev2 ipsec-proposal strong protocol esp encryption aes-256 protocol esp integrity sha-1 <--- More ---> crypto ipsec ikev2 ipsec-proposal AES256 protocol esp encryption aes-256 protocol esp integrity sha-1 md5 crypto ipsec ikev2 ipsec-proposal AES192 protocol esp encryption aes-192 protocol esp integrity sha-1 md5 crypto ipsec ikev2 ipsec-proposal AES protocol esp encryption aes protocol esp integrity sha-1 md5 crypto ipsec ikev2 ipsec-proposal 3DES protocol esp encryption 3des protocol esp integrity sha-1 md5 crypto ipsec ikev2 ipsec-proposal DES protocol esp encryption des protocol esp integrity sha-1 md5 crypto dynamic-map dyn1 1 set ikev1 transform-set strong crypto map outside 1 match address outside_cryptomap_1 crypto map outside 1 set pfs crypto map outside 1 set peer 83.x.172.68 crypto map outside 1 set ikev1 transform-set ESP-AES-256-SHA crypto map outside 1 set ikev2 ipsec-proposal AES256 crypto map outside 2 match address outside_cryptomap_3 crypto map outside 2 set peer 23.100.x.177 crypto map outside 2 set ikev1 transform-set ESP-AES-128-SHA ESP-AES-128-MD5 ESP-AES-192-SHA ESP-AES-192-MD5 ESP-AES-256-SHA ESP-AES-256-MD5 <--- More ---> crypto map outside 2 set ikev2 ipsec-proposal strong AES256 AES192 AES crypto map outside 2 set security-association lifetime kilobytes 102400000 crypto map outside 3 match address outside_cryptomap_2 crypto map outside 3 set pfs crypto map outside 3 set peer 91.x.3.39 crypto map outside 3 set ikev1 transform-set ESP-3DES-SHA crypto map outside 3 set ikev2 ipsec-proposal 3DES crypto map outside 100 ipsec-isakmp dynamic dyn1 crypto map outside interface outside crypto ca trustpoint _SmartCallHome_ServerCA crl configure crypto ikev2 policy 1 encryption aes-256 integrity sha group 2 prf sha lifetime seconds 86400 crypto ikev2 policy 10 encryption aes-192 integrity sha group 5 2 prf sha lifetime seconds 86400 crypto ikev2 policy 20 encryption aes integrity sha group 5 2 prf sha lifetime seconds 86400 crypto ikev2 policy 30 encryption 3des integrity sha group 5 2 prf sha lifetime seconds 86400 crypto ikev2 policy 40 encryption des integrity sha group 5 2 prf sha lifetime seconds 86400 crypto ikev2 enable outside crypto ikev1 enable outside crypto ikev1 policy 1 authentication pre-share encryption aes-256 hash sha group 2 lifetime 28800 crypto ikev1 policy 2 authentication pre-share encryption 3des hash sha group 2 lifetime 86400 ! track 1 rtr 1 reachability telnet 10.5.16.0 255.255.240.0 inside telnet timeout 5 ssh 83.x.x.90 255.255.255.255 outside ssh timeout 5 console timeout 0 dhcpd auto_config outside ! dhcprelay server 10.5.21.1 inside dhcprelay timeout 60 threat-detection basic-threat threat-detection statistics port <--- More ---> threat-detection statistics protocol threat-detection statistics access-list no threat-detection statistics tcp-intercept ntp server 10.5.19.253 prefer webvpn enable outside anyconnect image disk0:/anyconnect-win-2.5.2014-k9.pkg 1 anyconnect image disk0:/anyconnect-win-3.1.03103-k9.pkg 2 anyconnect enable tunnel-group-list enable group-policy DfltGrpPolicy attributes vpn-tunnel-protocol ikev1 l2tp-ipsec ssl-clientless group-policy GroupPolicy_c2si internal group-policy GroupPolicy_c2si attributes wins-server none dns-server value 10.5.16.1 10.5.21.1 vpn-tunnel-protocol ssl-client default-domain none group-policy GroupPolicy_91.x.3.39 internal group-policy GroupPolicy_91.x.3.39 attributes vpn-tunnel-protocol ikev1 ikev2 group-policy GroupPolicy_83.x.172.68 internal group-policy GroupPolicy_83.x.172.68 attributes vpn-tunnel-protocol ikev1 ikev2 <--- More ---> group-policy GroupPolicy_23.100.x.177 internal group-policy GroupPolicy_23.100.x.177 attributes vpn-tunnel-protocol ikev1 ikev2 group-policy GroupPolicy_user internal group-policy GroupPolicy_user attributes wins-server none dns-server value 10.5.21.1 10.5.16.1 vpn-tunnel-protocol ssl-client split-tunnel-policy tunnelspecified split-tunnel-network-list value BerkeleyAdmin-clientvpn default-domain value myberkeley.local group-policy GroupPolicy_23.101.x.122 internal group-policy GroupPolicy_23.101.x.122 attributes vpn-tunnel-protocol ikev1 ikev2 group-policy GroupPolicy1 internal group-policy GroupPolicy1 attributes vpn-tunnel-protocol ikev1 ikev2 group-policy BerkeleyUser internal group-policy BerkeleyUser attributes dns-server value 10.5.21.1 10.5.16.1 split-tunnel-policy tunnelspecified split-tunnel-network-list value BerkeleyUser-clientvpn default-domain value myberkeley.local group-policy DS365 internal <--- More ---> group-policy DS365 attributes vpn-idle-timeout none vpn-filter none ipv6-vpn-filter none vpn-tunnel-protocol ikev1 ikev2 group-policy BerkeleyAdmin internal group-policy BerkeleyAdmin attributes dns-server value 10.5.21.1 10.5.16.1 split-tunnel-policy tunnelspecified split-tunnel-network-list value BerkeleyAdmin-clientvpn default-domain value myberkeley.local username acsadmin password V6hUzNl366K37eiV encrypted privilege 15 username atlanta password uxelpvEvM3I7tw.Z encrypted privilege 15 username berkeley password Kj.RBvUp5dtyLw5T encrypted tunnel-group BerkeleyUser type remote-access tunnel-group BerkeleyUser general-attributes address-pool clientvpn authentication-server-group group default-group-policy BerkeleyUser tunnel-group BerkeleyUser ipsec-attributes ikev1 pre-shared-key ***** tunnel-group BerkeleyAdmin type remote-access tunnel-group BerkeleyAdmin general-attributes address-pool clientvpn <--- More ---> authentication-server-group group default-group-policy BerkeleyAdmin tunnel-group BerkeleyAdmin ipsec-attributes ikev1 pre-shared-key ***** tunnel-group user type remote-access tunnel-group user general-attributes address-pool VPN_IP_Pool authentication-server-group group default-group-policy GroupPolicy_user tunnel-group user webvpn-attributes group-alias user enable tunnel-group c2si type remote-access tunnel-group c2si general-attributes address-pool VPN_IP_Pool authentication-server-group group default-group-policy GroupPolicy_c2si tunnel-group c2si webvpn-attributes group-alias c2si enable tunnel-group 83.x.172.68 type ipsec-l2l tunnel-group 83.x.172.68 general-attributes default-group-policy GroupPolicy_83.x.172.68 tunnel-group 83.x.172.68 ipsec-attributes ikev1 pre-shared-key ***** ikev2 remote-authentication pre-shared-key ***** <--- More ---> ikev2 local-authentication pre-shared-key ***** tunnel-group 23.101.x.122 type ipsec-l2l tunnel-group 23.101.x.122 general-attributes default-group-policy GroupPolicy_23.101.x.122 tunnel-group 23.101.x.122 ipsec-attributes ikev1 pre-shared-key ***** ikev2 remote-authentication pre-shared-key ***** ikev2 local-authentication pre-shared-key ***** tunnel-group 91.x.3.39 type ipsec-l2l tunnel-group 91.x.3.39 general-attributes default-group-policy GroupPolicy_91.x.3.39 tunnel-group 91.x.3.39 ipsec-attributes ikev1 pre-shared-key ***** ikev2 remote-authentication pre-shared-key ***** ikev2 local-authentication pre-shared-key ***** tunnel-group 23.100.x.177 type ipsec-l2l tunnel-group 23.100.x.177 general-attributes default-group-policy GroupPolicy_23.100.63.177 tunnel-group 23.100.x.177 ipsec-attributes ikev1 pre-shared-key ***** ikev2 remote-authentication pre-shared-key ***** ikev2 local-authentication pre-shared-key ***** class-map state_bypass match access-list state_bypass policy-map state_bypass_policy class state_bypass set connection advanced-options tcp-state-bypass ! service-policy state_bypass_policy interface inside prompt hostname context call-home reporting anonymous Cryptochecksum:bbc6f2ec2db9b09a1b6eb90270ddfeea : end tbp-ch-asa5505#

[Marvin Rhoads]

So I se you've NAT exempted 10.5.25.0/24 (Atlanta) and traffic from it is coming across your site-site VPN and destined for 172.16.10.10.

You have no static route for that destination network but you are running EIGRP on the ASA. Is the ASA forming EIGRP neighbor relationship with your inside router and is it learning a route to that destination network? ("show eigrp neighbors" and "show route")

[andy_4578]

Hi Marvin,

Remote networks 10.5.25.0/24 & 172.16.10.0 are both connected via site to site vpn tunnels.

10.5.25.0/24 - Microsoft Azure

172.16.10.0/24 DS365 hosting center

Eigrp forms a relationship with the core switches and another data center.

Everything is working apart from 172.16.10.0/24 cant contact 10.5.25.0/24

[Marvin Rhoads · Accepted solution]

Oh OK I see that now.

Your cryptomap for the DS365 cloud is:

access-list outside_cryptomap_2 extended permit ip 10.5.0.0 255.255.0.0 object DS365-Cloud

so that covers the interesting traffic.

Your NAT statement however is:

nat (inside,outside) source static NETWORK_OBJ_10.5.25.0_24 NETWORK_OBJ_10.5.25.0_24 destination static DS365-Cloud DS365-Cloud no-proxy-arp route-lookup

Network 10.5.25.0 is remote so it will actually appear to be an "outside" network so I believe you would need that statement to begin "nat (outside,outside)"

[andy_4578]

Hi Marvin,

Your absolutely right it did need to begin nat (outside,outside).

It's all working now.

I did need to move the nat statement to the top as well due to a conflicting nat rule further up.

Thanks for your assistance.