VPN route-based unable to ping remote IP

[John Bautista]

Hi, I am currently encountering issue on route-based ipsec vpn. I cannot ping my remote IP also the remote tunnel. I have verified that there is no decap showing on packets. I already configured static route between each site and still unsucessful of connectivity.

Crypto map tag: __vti-crypto-map-9-0-10, seq num: 65280, local addr: xxx.xxx.xxx.xxx

local ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
current_peer: xxx.xxx.xxx.xxx

#pkts encaps: 1183, #pkts encrypt: 1183, #pkts digest: 1183
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 1183, #pkts comp failed: 0, #pkts decomp failed: 0
#pre-frag successes: 0, #pre-frag failures: 0, #fragments created: 0
#PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0
#TFC rcvd: 0, #TFC sent: 0
#Valid ICMP Errors rcvd: 0, #Invalid ICMP Errors rcvd: 0
#send errors: 0, #recv errors: 0

local crypto endpt.: xxx.xxx.xxx.xxx/0, remote crypto endpt.: xxx.xxx.xxx.xxx/0
path mtu 1492, ipsec overhead 74(44), media mtu 1500
PMTU time remaining (sec): 0, DF policy: copy-df
ICMP error validation: disabled, TFC packets: disabled
current outbound spi: E2D1DD20
current inbound spi : 069A90C5

But when I do packet tracer. It is showing allowed on the lan side.

packet-tracer input lan icmp 192.168.0.131 8 0 192.100.12.1

Phase: 1
Type: ROUTE-LOOKUP
Subtype: Resolve Egress Interface
Result: ALLOW
Config:
Additional Information:
found next-hop 10.10.10.2 using egress ifc Site_B

Phase: 2
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:

Phase: 3
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:

Phase: 4
Type: INSPECT
Subtype: np-inspect
Result: ALLOW
Config:
Additional Information:

Phase: 5
Type: FLOW-EXPORT
Subtype:
Result: ALLOW
Config:
Additional Information:

Phase: 6
Type: DEBUG-ICMP
Subtype:
Result: ALLOW
Config:
Additional Information:

Phase: 7
Type: VPN
Subtype: encrypt
Result: ALLOW
Config:
Additional Information:

Phase: 8
Type: FLOW-CREATION
Subtype:
Result: ALLOW
Config:
Additional Information:
New flow created with id 340117277, packet dispatched to next module

Result:
input-interface: lan
input-status: up
input-line-status: up
output-interface: Site_B
output-status: up
output-line-status: up
Action: allow

[MHM Cisco World]

First this is ASA so keep away checking the tunnel by ping

Now let focus on zero decap'

Can share tunnel config 

Also did you check that other side use correct routing toward yout tunnel or not?

MHM

[John Bautista]

Sure, Please see the tunnel config. The tunnel configuration is the same between two sites.

interface Tunnel100
nameif Site_A
ip address 10.10.10.2 255.255.255.252
tunnel source interface outside
tunnel destination XXX.XXX.XXX.XXX
tunnel mode ipsec ipv4
tunnel protection ipsec profile Site_A

crypto ipsec profile Site_A
set ikev1 transform-set vpn1
crypto ipsec ikev1 transform-set vpn1 esp-aes-256 esp-sha-hmac


tunnel-group xxx.xxx.xxx.xxx type ipsec-l2l
tunnel-group xxx.xxx.xxx.xxx ipsec-attributes
ikev1 pre-shared-key *****



Yes, I configure the route correctly and pointing it to the remote tunnel. 
route Site_A 192.168.0.0 255.255.0.0 10.10.10.1

route Site_B 192.100.0.0 255.255.0.0 10.10.10.2

[MHM Cisco World]

Config same secuirty traffic permit inter and intra interface 

And check again 

MHM

[John Bautista]

I already configured your suggestion. However, still was not able to ping remote network. Will share you my config to both site for you to check. If there anything missing here like ACL? do I need to configure it? 

SITE A:

crypto ikev1 policy 10
authentication pre-share
encryption aes-256
hash sha
group 2
lifetime 86400

crypto ikev1 enable outside
crypto ipsec profile Site_A
set ikev1 transform-set vpn1
crypto ipsec ikev1 transform-set vpn1 esp-aes-256 esp-sha-hmac


tunnel-group xxx.xxx.xxx.xxx type ipsec-l2l
tunnel-group xxx.xxx.xxx.xxx ipsec-attributes
ikev1 pre-shared-key *****

same-security-traffic permit inter-interface
same-security-traffic permit intra-interface

interface Tunnel100
nameif Site_A
ip address 10.10.10.2 255.255.255.252
tunnel source interface outside
tunnel destination XXX.XXX.XXX.XXX
tunnel mode ipsec ipv4
tunnel protection ipsec profile Site_A

route Site_A 192.168.0.0 255.255.0.0 10.10.10.1

SITE:B

crypto ikev1 policy 10
authentication pre-share
encryption aes-256
hash sha
group 2
lifetime 86400

crypto ikev1 enable outside
crypto ipsec profile Site_B
set ikev1 transform-set vpn1
crypto ipsec ikev1 transform-set vpn1 esp-aes-256 esp-sha-hmac


tunnel-group xxx.xxx.xxx.xxx type ipsec-l2l
tunnel-group xxx.xxx.xxx.xxx ipsec-attributes
ikev1 pre-shared-key *****

same-security-traffic permit inter-interface
same-security-traffic permit intra-interface

interface Tunnel10
nameif Site_B
ip address 10.10.10.1 255.255.255.252
tunnel source interface outside
tunnel destination XXX.XXX.XXX.XXX
tunnel mode ipsec ipv4
tunnel protection ipsec profile Site_B

route Site_B 192.100.0.0 255.255.0.0 10.10.10.2

[Rob Ingram]

@John Bautista so one side the encaps counters are increasing, are the decaps counters increasing on the other side? Provide the output of "show crypto ipsec sa" from both sides for comparison.

Is this static route correct? route Site_B 192.100.0.0 255.255.0.0 10.10.10.2

I don't see why configuring same-security-traffic permit inter-interface/intra-interface would help here.

[John Bautista]

Yes, my SIte_A has private ip 192.100.0.0/16 configured while my Site_B has 192.168.0.0/16. I have configured static route on both 
Site_B(config)#route Site_B 192.100.0.0 255.255.0.0 10.10.10.2 
Site_A(config)#route Site_A 192.168.0.0 255.255.0.0 10.10.10.1

Here is the output of show crypto ipsec sa

interface: Site_A
Crypto map tag: __vti-crypto-map-6-0-100, seq num: 65280, local addr: xxx.xxx.xxx.xxx

local ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
current_peer: xxx.xxx.xxx.xxx

#pkts encaps: 43513, #pkts encrypt: 43513, #pkts digest: 43513
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 43513, #pkts comp failed: 0, #pkts decomp failed: 0
#pre-frag successes: 0, #pre-frag failures: 0, #fragments created: 0
#PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0
#TFC rcvd: 0, #TFC sent: 0
#Valid ICMP Errors rcvd: 0, #Invalid ICMP Errors rcvd: 0
#send errors: 0, #recv errors: 0

interface: Site_B
Crypto map tag: __vti-crypto-map-9-0-10, seq num: 65280, local addr: xxx.xxx.xxx.xxx

local ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
current_peer: xxx.xxx.xxx.xxx

#pkts encaps: 1189, #pkts encrypt: 1189, #pkts digest: 1189
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 1189, #pkts comp failed: 0, #pkts decomp failed: 0
#pre-frag successes: 0, #pre-frag failures: 0, #fragments created: 0
#PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0
#TFC rcvd: 0, #TFC sent: 0
#Valid ICMP Errors rcvd: 0, #Invalid ICMP Errors rcvd: 0
#send errors: 0, #recv errors: 0

[Rob Ingram]

@John Bautista is ESP blocked between the peers (a packet capture will confirm this)?

192.100.0.0/16 is not a private network, I assume you meant a public network.

[MHM Cisco World]

There is nothing wrong in your config' 

Icmp inspection under policy map is config?

If Yes 

And still see zero decrypt share both sites

Show crypto isakmp sa

Show crypto ipsec sa 

Show vpn sessiondb ipsec l2l detail 

MHM

[MHM Cisco World]

friend share what I mention above.

again for both
MHM

[MHM Cisco World]

first the tunnel have security level zero that why I mention you need sysopt permit 
I test disable ICMP inspection (even if I was 99% sure it not cause) and still in lab I can ping from side to side, but @tvotna always suggest something he neve totally know.

NOW 
for ESP I see there mention that there is filter in Path, you can Know that by
capture CAP interface OUT match IP host <peer> host <peer>
this can give us hint if the traffic is hit the ASA or not 
also again share what I ask above for both 

MHM 

[MHM Cisco World]

by the way the lab it done 

[tvotna]

@John Bautista, don't follow above advice of configuring same security and troubleshoot the other side instead. Verify that you have ICMP inspection enabled there and check "decaps" to understand whether packets are lost in transit or they are lost on the remote device or in the remote network. If "decaps" increases, use packet-tracer on the remote ASA with the "decrypted" option to treat simulated packet as IPsec decrypted. Look at syslog messages at level 6 on the remote ASA.

[John Bautista]

Hello, I tried to run packet tracer on my Site_A with decrypted. and this is the result

SITE_A# packet-tracer input inside icmp 192.100.10.13 8 0 192.168.0.13 decrypted

*********************************************************************
WARNING: An existing decryption SA was not found. Please confirm the
IPsec Phase 2 SA or Anyconnect Tunnel is established.
*********************************************************************

Phase: 1
Type: ACCESS-LIST
Subtype:
Result: ALLOW
Config:
Implicit Rule
Additional Information:
MAC Access list

Phase: 2
Type: ROUTE-LOOKUP
Subtype: Resolve Egress Interface
Result: ALLOW
Config:
Additional Information:
found next-hop 10.10.10.1 using egress ifc Site_A

Result:
input-interface: inside
input-status: up
input-line-status: up
output-interface: Site_A
output-status: up
output-line-status: up
Action: drop
Drop-reason: (vpn-context-expired) Expired VPN context

[tvotna]

This should have been:

SITE_A# packet-tracer input outside icmp 192.168.0.13 8 0 192.100.10.13 decrypted

Anyway, like @Rob Ingram mentioned, if decaps is zero on both sides, ESP is probably blocked somewhere in the path, so encapsulated packets do not reach peer. You need to collect capture on the outside interface of both ASAs for traffic between their outside IP addresses to verify this fact.