End to end IP cannot pingable via tunnel and decaps is zero

[Grandong]

Hi All,

I have some issue with my ipsec connection, the idea is IP 10.6.1.41 need to connect to IP 178.248.230.16 and 178.248.230.17 (both are local IP) and vice versa.

Here is the tunnel configuration I was made

protected vrf: (none)
local ident (addr/mask/prot/port): (10.6.1.41/255.255.255.255/0/0)
remote ident (addr/mask/prot/port): (178.248.230.16/255.255.255.255/0/0)
current_peer 178.248.230.243 port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 75, #pkts encrypt: 75, #pkts digest: 75
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0

local crypto endpt.: 202.2.112.50, remote crypto endpt.: 178.248.230.243
path mtu 1500, ip mtu 1500, ip mtu idb Port-channel1
current outbound spi: 0xE416DF6A(3826704234)
PFS (Y/N): Y, DH group: group2

inbound esp sas:
spi: 0x93D6B352(2480321362)
transform: esp-256-aes esp-sha-hmac ,
in use settings ={Tunnel, }
conn id: 389, flow_id: 389, sibling_flags 80000040, crypto map: ttc
sa timing: remaining key lifetime (k/sec): (4608000/2812)
IV size: 16 bytes
replay detection support: Y replay window size: 1024
Status: ACTIVE(ACTIVE)

inbound ah sas:

inbound pcp sas:

outbound esp sas:
spi: 0xE416DF6A(3826704234)
transform: esp-256-aes esp-sha-hmac ,
in use settings ={Tunnel, }
conn id: 390, flow_id: 390, sibling_flags 80000040, crypto map: ttc
sa timing: remaining key lifetime (k/sec): (4608000/2812)
IV size: 16 bytes
replay detection support: Y replay window size: 1024
Status: ACTIVE(ACTIVE)

outbound ah sas:

outbound pcp sas:

protected vrf: (none)
local ident (addr/mask/prot/port): (10.6.1.41/255.255.255.255/0/0)
remote ident (addr/mask/prot/port): (178.248.230.17/255.255.255.255/0/0)
current_peer 178.248.230.243 port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0

local crypto endpt.: 202.2.112.50, remote crypto endpt.: 178.248.230.243
path mtu 1500, ip mtu 1500, ip mtu idb Port-channel1
current outbound spi: 0x1C30191B(472914203)
PFS (Y/N): Y, DH group: group2

inbound esp sas:
spi: 0x59AF4DA6(1504660902)
transform: esp-256-aes esp-sha-hmac ,
in use settings ={Tunnel, }
conn id: 391, flow_id: 391, sibling_flags 80000040, crypto map: ttc
sa timing: remaining key lifetime (k/sec): (4608000/3035)
IV size: 16 bytes
replay detection support: Y replay window size: 1024
Status: ACTIVE(ACTIVE)

inbound ah sas:

inbound pcp sas:

outbound esp sas:
spi: 0x1C30191B(472914203)
transform: esp-256-aes esp-sha-hmac ,
in use settings ={Tunnel, }
conn id: 392, flow_id: 392, sibling_flags 80000040, crypto map: ttc
sa timing: remaining key lifetime (k/sec): (4608000/3035)
IV size: 16 bytes
replay detection support: Y replay window size: 1024
Status: ACTIVE(ACTIVE)

outbound ah sas:

outbound pcp sas:

!
ip nat translation tcp-timeout 300
ip nat translation udp-timeout 20
ip nat translation syn-timeout 15
ip nat translation icmp-timeout 15
ip nat inside source list 1 interface Port-channel1 overload
ip forward-protocol nd
!
!
no ip http server
no ip http secure-server
ip route 0.0.0.0 0.0.0.0 202.2.112.33
ip route 10.5.0.0 255.255.0.0 10.6.1.1
ip route 192.168.0.0 255.255.255.0 10.6.1.1
ip route 192.168.1.0 255.255.255.0 10.6.1.253
!
access-list 1 deny 10.6.1.0 0.0.0.255
access-list 1 permit 192.168.0.0 0.0.0.255
access-list 1 permit 10.5.0.0 0.0.127.255
access-list 100 permit ip host 10.6.1.41 host 178.248.230.16
access-list 100 permit ip host 10.6.1.41 host 178.248.230.17
access-list 101 permit ip host 10.6.1.41 host 178.248.228.120
access-list 102 permit ip host 10.6.1.41 host 193.111.74.132
access-list 103 permit ip host 10.6.1.41 host 193.111.74.133

interface Port-channel1
ip address 202.2.112.51 255.255.255.224 secondary
ip address 202.2.112.50 255.255.255.224
ip nat outside
negotiation auto
crypto map XXX
!

TTC_R1#show crypto sess
Crypto session current status

Interface: Port-channel1
Session status: UP-ACTIVE
Peer: 178.248.230.243 port 500
IKEv1 SA: local 202.2.112.50/500 remote 178.248.230.243/500 Active
IPSEC FLOW: permit ip host 10.6.1.41 host 178.248.230.16
Active SAs: 2, origin: crypto map
IPSEC FLOW: permit ip host 10.6.1.41 host 178.248.230.17
Active SAs: 2, origin: crypto map

Interface: Port-channel1
Session status: DOWN
Peer: 85.233.207.251 port 500
IPSEC FLOW: permit ip host 10.6.1.41 host 178.248.228.120
Active SAs: 0, origin: crypto map

TTC_R1#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
+ - replicated route, % - next hop override

Gateway of last resort is 202.2.112.33 to network 0.0.0.0

S* 0.0.0.0/0 [1/0] via 202.2.112.33
10.0.0.0/8 is variably subnetted, 3 subnets, 3 masks
S 10.5.0.0/16 [1/0] via 10.6.1.1
C 10.6.1.0/24 is directly connected, GigabitEthernet0/2
L 10.6.1.242/32 is directly connected, GigabitEthernet0/2
S 192.168.0.0/24 [1/0] via 10.6.1.1
S 192.168.1.0/24 [1/0] via 10.6.1.253
202.2.112.0/24 is variably subnetted, 3 subnets, 2 masks
C 202.2.112.32/27 is directly connected, Port-channel1
L 202.2.112.50/32 is directly connected, Port-channel1
L 202.2.112.51/32 is directly connected, Port-channel1
TTC_R1#

what I noticed is:

1. the #pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0 is zero

2. from 10.6.1.41 cannot ping to IP 178.248.230.16 and 178.248.230.17

kindly help ....

Regards,

Grndg

[Rob Ingram]

@Grandong if you are encrypting traffic but not not decrypting traffic, as indicated by the counters, then it's usually a problem with nat or routing on the other device.

Do you manage the other side of the VPN (not this router above)? On the other device ensure traffic is not unintentially translated and traffic is actually routing correctly. Can you provide the configuration and the output of "show crypto ipsec sa" assuming it's a cisco device.

[Grandong]

Hi @Rob Ingram ,

Unfortunately I'm not managed the other side of the VPN.

from the other side they also mentioned "Packets are encapsulating but not seeing any decapsulation packets" as below

local ident (addr/mask/prot/port): (178.248.230.16/255.255.255.255/0/0)

remote ident (addr/mask/prot/port): (10.6.1.41/255.255.255.255/0/0)

#pkts encaps: 23, #pkts encrypt: 23, #pkts digest: 23

#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0

#PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0

local ident (addr/mask/prot/port): (178.248.230.17/255.255.255.255/0/0)

remote ident (addr/mask/prot/port): (10.6.1.41/255.255.255.255/0/0)

#pkts encaps: 21, #pkts encrypt: 21, #pkts digest: 21

#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0

#PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0

below the output of "show crypto ipsec sa"

protected vrf: (none)
local ident (addr/mask/prot/port): (10.6.1.41/255.255.255.255/0/0)
remote ident (addr/mask/prot/port): (178.248.230.16/255.255.255.255/0/0)
current_peer 178.248.230.243 port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 75, #pkts encrypt: 75, #pkts digest: 75
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0

local crypto endpt.: 202.2.112.50, remote crypto endpt.: 178.248.230.243
path mtu 1500, ip mtu 1500, ip mtu idb Port-channel1
current outbound spi: 0xC9D1D431(3385971761)
PFS (Y/N): Y, DH group: group2

inbound esp sas:
spi: 0x7EA0C66D(2124465773)
transform: esp-256-aes esp-sha-hmac ,
in use settings ={Tunnel, }
conn id: 421, flow_id: 421, sibling_flags 80000040, crypto map: ttc
sa timing: remaining key lifetime (k/sec): (4608000/3394)
IV size: 16 bytes
replay detection support: Y replay window size: 1024
Status: ACTIVE(ACTIVE)

inbound ah sas:

inbound pcp sas:

outbound esp sas:
spi: 0xC9D1D431(3385971761)
transform: esp-256-aes esp-sha-hmac ,
in use settings ={Tunnel, }
conn id: 422, flow_id: 422, sibling_flags 80000040, crypto map: ttc
sa timing: remaining key lifetime (k/sec): (4608000/3394)
IV size: 16 bytes
replay detection support: Y replay window size: 1024
Status: ACTIVE(ACTIVE)

outbound ah sas:

outbound pcp sas:

protected vrf: (none)
local ident (addr/mask/prot/port): (10.6.1.41/255.255.255.255/0/0)
remote ident (addr/mask/prot/port): (178.248.230.17/255.255.255.255/0/0)
current_peer 178.248.230.243 port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0

local crypto endpt.: 202.2.112.50, remote crypto endpt.: 178.248.230.243
path mtu 1500, ip mtu 1500, ip mtu idb Port-channel1
current outbound spi: 0x77D0E9AE(2010180014)
PFS (Y/N): Y, DH group: group2

inbound esp sas:
spi: 0xBF8428C1(3213109441)
transform: esp-256-aes esp-sha-hmac ,
in use settings ={Tunnel, }
conn id: 415, flow_id: 415, sibling_flags 80000040, crypto map: ttc
sa timing: remaining key lifetime (k/sec): (4608000/515)
IV size: 16 bytes
replay detection support: Y replay window size: 1024
Status: ACTIVE(ACTIVE)

inbound ah sas:

inbound pcp sas:

outbound esp sas:
spi: 0x77D0E9AE(2010180014)
transform: esp-256-aes esp-sha-hmac ,
in use settings ={Tunnel, }
conn id: 416, flow_id: 416, sibling_flags 80000040, crypto map: ttc
sa timing: remaining key lifetime (k/sec): (4608000/515)
IV size: 16 bytes
replay detection support: Y replay window size: 1024
Status: ACTIVE(ACTIVE)

outbound ah sas:

outbound pcp sas:

[Rob Ingram]

@Grandong is ESP blocked some where in the path of the 2 tunnels? Check ACL on both devices.

[Grandong]

Hi @Rob Ingram 

Thanks to always reply my message, sorry I'm newbie with this tunnel setup.

regarding ESP what should I check?, because for IT its handled by other team, I Can ask them to check.

below is the ACL from My router

access-list 1 deny 10.6.1.0 0.0.0.255
access-list 1 permit 192.168.0.0 0.0.0.255
access-list 1 permit 10.5.0.0 0.0.127.255
access-list 100 permit ip host 10.6.1.41 host 178.248.230.16
access-list 100 permit ip host 10.6.1.41 host 178.248.230.17
access-list 101 permit ip host 10.6.1.41 host 178.248.228.120
access-list 102 permit ip host 10.6.1.41 host 193.111.74.132
access-list 103 permit ip host 10.6.1.41 host 193.111.74.133

[MHM Cisco World]

Port-channel1 <<- are you apply the IPSec under the PO ?? If Yes check the IOS ver. not all Ver. support config the IPSec under PO

[Grandong]

Hi @MHM Cisco World 

this the the version

TTC_R1#show version
Cisco IOS Software, 7301 Software (C7301-ADVENTERPRISEK9-M), Version 15.2(4)S5, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2014 by Cisco Systems, Inc.
Compiled Thu 20-Feb-14 07:31 by prod_rel_team

ROM: System Bootstrap, Version 12.3(4r)T4, RELEASE SOFTWARE (fc1)
BOOTLDR: Cisco IOS Software, 7301 Software (C7301-BOOT-M), Version 12.4(12c), RELEASE SOFTWARE (fc1)

TTC_R1 uptime is 19 weeks, 1 day, 4 hours, 52 minutes
System returned to ROM by power-on
System restarted at 08:06:02 UTC Tue Apr 12 2022
System image file is "disk0:c7301-adventerprisek9-mz.152-4.S5.bin"
Last reload type: Normal Reload
Last reload reason: power-on

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

Cisco 7301 (NPE) processor (revision F) with 491520K/32768K bytes of memory.
Processor board ID 74851396
SB-1 CPU at 700MHz, Implementation 1025, Rev 0.2, 512KB L2 Cache
1 slot midplane, Version 2.0

Last reset from power-on
3 Gigabit Ethernet interfaces
509K bytes of NVRAM.

62720K bytes of ATA PCMCIA card at slot 0 (Sector size 512 bytes).
32768K bytes of Flash internal SIMM (Sector size 256K).
Configuration register is 0x2102

[Grandong]

Hi @MHM Cisco World 

Actually it was working before, but then after restarted due power outage now it become disconnect.