Hi there!
We've been running into the same issue a couple of times now and I do not seem to find any information on the ASR itself on when this is happening. Long story short:
- we have a ASR1006-X router (version 16.06.05) where we build different IPSec tunnels for different customers (in different VRFs).
- All of the tunnels are crypto-map based with every customer being a different entry.
- We've had a couple of cases now where IKEv2 (have not noticed IKEv1 to do this) phase-1 is built successfully, phase-2 SAs are established and everything seems to be running smoothly. However one of the SAs (not all) behaves weird, where no traffic is dropped, but no traffic is being encapsulated. After extensive troubleshooting in our network, we confirmed that the traffic is indeed being sent to the Cisco from the MPLS, but nothing is encapsulated into the IPSec.
No drops here:
R1#show cry ses ivrf customer_1 detail
Crypto session current status
Interface: GigabitEthernet0/1/1
Profile: customer_1
Uptime: 02:00:07
Session status: UP-ACTIVE
Peer: x.x.x.x port 500 fvrf: (none) ivrf: customer_1
Phase1_id: x.x.x.x
Desc: (none)
Session ID: 25096162
IKEv2 SA: local z.z.z.z/500 remote x.x.x.x/500 Active
Capabilities:(none) connid:97 lifetime:21:59:53
IPSEC FLOW: permit ip 10.133.80.0/255.255.240.0 10.30.208.0/255.255.240.0
Active SAs: 0, origin: crypto map
Inbound: #pkts dec'ed 999 drop 0 life (KB/Sec) 0/0
Outbound: #pkts enc'ed 0 drop 0 life (KB/Sec) 0/0
IPSEC FLOW: permit ip 10.133.80.0/255.255.240.0 10.30.240.0/255.255.240.0
Active SAs: 0, origin: crypto map
Inbound: #pkts dec'ed 123456 drop 0 life (KB/Sec) 0/0
Outbound: #pkts enc'ed 123456 drop 0 life (KB/Sec) 0/0
IPSEC FLOW: permit ip 10.133.80.0/255.255.240.0 10.153.0.0/255.255.0.0
Active SAs: 0, origin: crypto map
Inbound: #pkts dec'ed 7537 drop 0 life (KB/Sec) 0/0
Outbound: #pkts enc'ed 7537 drop 0 life (KB/Sec) 0/0
IPSEC FLOW: permit ip 10.133.80.0/255.255.240.0 10.155.0.0/255.255.0.0
Active SAs: 0, origin: crypto map
Inbound: #pkts dec'ed 4563456345 drop 0 life (KB/Sec) 0/0
Outbound: #pkts enc'ed 4563456345 drop 0 life (KB/Sec) 0/0
Nor here:
R1#sh crypto ipsec sa peer x.x.x.x detail
interface: GigabitEthernet0/1/1
Crypto map tag: my_crypto-map, local addr z.z.z.z
protected vrf: customer_1
local ident (addr/mask/prot/port): (10.133.80.0/255.255.240.0/0/0)
remote ident (addr/mask/prot/port): (10.30.208.0/255.255.240.0/0/0)
current_peer x.x.x.x port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 999, #pkts decrypt: 999, #pkts verify: 999
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#pkts no sa (send) 0, #pkts invalid sa (rcv) 0
#pkts encaps failed (send) 0, #pkts decaps failed (rcv) 0
#pkts invalid prot (recv) 0, #pkts verify failed: 0
#pkts invalid identity (recv) 0, #pkts invalid len (rcv) 0
#pkts replay rollover (send): 0, #pkts replay rollover (rcv) 0
##pkts replay failed (rcv): 0
#pkts tagged (send): 0, #pkts untagged (rcv): 0
#pkts not tagged (send): 0, #pkts not untagged (rcv): 0
#pkts internal err (send): 0, #pkts internal err (recv) 0
local crypto endpt.: z.z.z.z, remote crypto endpt.: x.x.x.x
plaintext mtu 1500, path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet0/1/1
current outbound spi: 0x0(0)
PFS (Y/N): N, DH group: none
inbound esp sas:
spi: 0xD360A3F0(3546326000)
transform: esp-gcm 256 ,
in use settings ={Tunnel, }
conn id: 42773, flow_id: HW:40773, sibling_flags FFFFFFFF80000048, crypto map: my_crypto-map
sa timing: remaining key lifetime (k/sec): (4607998/3247)
IV size: 8 bytes
replay detection support: Y replay window size: 512
Status: ACTIVE(ACTIVE)
inbound ah sas:
inbound pcp sas:
outbound esp sas:
spi: 0x664C02F0(1716257520)
transform: esp-gcm 256 ,
in use settings ={Tunnel, }
conn id: 42772, flow_id: HW:40772, sibling_flags FFFFFFFF80000048, crypto map: my_crypto-map
sa timing: remaining key lifetime (k/sec): (4607998/3247)
IV size: 8 bytes
replay detection support: Y replay window size: 512
Status: ACTIVE(ACTIVE)
outbound ah sas:
outbound pcp sas:
I've also tried to run monitor capture on the interface connected to the MPLS, to see if the traffic is even reaching the Cisco, but that was not successful. I've also ran every debug option related to "debug crypto", that I could find, but nothing in the logs showed packets to be dropped.
Running out of options, I started to look at the configuration from the customer side, for any hints on what's going on, when I noticed the difference in our and theirs ACLs. I then deleted the crypto-map entry, changed the ACLs on our end to match theirs, re-applied the crypto-map entry and everything worked.
So my question is - what did I miss in terms of troubleshooting (is there some place I could see these silently dropped packets) and why is the IPSec SA established in the first place, when the traffic selectors do not match. And the most interesting part was that even though they did not match, the customer was able to successfully forward traffic via the IPSec from their end.
The configuration on our end is very basic:
crypto ikev2 keyring customer_1
peer customer_1
address x.x.x.x
pre-shared-key My_KEY
!
crypto ikev2 profile customer_1
match identity remote address x.x.x.x 255.255.255.255
authentication remote pre-share
authentication local pre-share
keyring local customer_1
ivrf customer_1
!
crypto map my_crypto-map 2200 ipsec-isakmp
set peer x.x.x.x
set transform-set vpn-gcm-256
set pfs group20
set ikev2-profile customer_1
match address customer_1
reverse-route static
ip access-list extended customer_1
permit ip 10.133.80.0 0.0.15.255 10.30.208.0 0.0.15.255
permit ip 10.133.80.0 0.0.15.255 10.30.240.0 0.0.15.255
permit ip 10.133.80.0 0.0.15.255 10.153.0.0 0.0.255.255
permit ip 10.133.80.0 0.0.15.255 10.155.0.0 0.0.255.255
