IKEv2 - IKE SA UP but IPSEC is UP but tunnel Down. Worked in the Past

[Elito Haylett]

Here are the debugs from both routers. Can some please help make sense as to why the tunnel is not up and passing traffic?

Router-A#
Dec 1 21:13:44.399: IKEv2:Received Packet [From 2.2.2.2:500/To 1.1.1.1:500/VRF i0:f0]
Initiator SPI : AA3C74EE26AAC7C5 - Responder SPI : 0000000000000000 Message id: 0
IKEv2 IKE_SA_INIT Exchange REQUEST
Payload contents:

Dec 1 21:13:44.399: IKEv2:parsing SA payload SA
Dec 1 21:13:44.399: IKEv2:parsing KE payload KE
Dec 1 21:13:44.399: IKEv2:parsing N payload N
Dec 1 21:13:44.399: IKEv2:parsing VID payload VID
Dec 1 21:13:44.399: IKEv2:parsing VID payload VID
Dec 1 21:13:44.399: IKEv2:parsing VID payload VID
Dec 1 21:13:44.399: IKEv2:parsing VID payload VID
Dec 1 21:13:44.400: IKEv2:parsing NOTIFY payload NOTIFY(NAT_DETECTION_SOURCE_IP)
Dec 1 21:13:44.400: IKEv2:parsing NOTIFY payload NOTIFY(NAT_DETECTION_DESTINATION_IP)

Dec 1 21:13:44.400: IKEv2:(SESSION ID = 52,SA ID = 1):Verify SA init message
Dec 1 21:13:44.400: IKEv2:(SESSION ID = 52,SA ID = 1):Insert SA
Dec 1 21:13:44.400: IKEv2:Searching Policy with fvrf 0, local address 1.1.1.1
Dec 1 21:13:44.400: IKEv2:Using the Default Policy for Proposal
Dec 1 21:13:44.400: IKEv2:Found Policy 'default'
Dec 1 21:13:44.400: IKEv2:(SESSION ID = 52,SA ID = 1):Processing IKE_SA_INIT message
Dec 1 21:13:44.400: IKEv2:(SA ID = 1):[IKEv2 -> PKI] Retrieve configured trustpoint(s)
Dec 1 21:13:44.400: IKEv2:(SA ID = 1):[PKI -> IKEv2] Retrieved trustpoint(s): 'ECH-IOS-CA' 'SLA-TrustPoint' 'TP-self-signed-3693526534'
Dec 1 21:13:44.400: IKEv2:(SA ID = 1):[IKEv2 -> PKI] Get Public Key Hashes of trustpoints
Dec 1 21:13:44.400: IKEv2:(SA ID = 1):[PKI -> IKEv2] Getting of Public Key Hashes of trustpoints PASSED
Dec 1 21:13:44.401: IKEv2:(SA ID = 1):[IKEv2 -> PKI] Start PKI Session
Dec 1 21:13:44.401: IKEv2:(SA ID = 1):[PKI -> IKEv2] Starting of PKI Session PASSED
Dec 1 21:13:44.401: IKEv2:(SESSION ID = 52,SA ID = 1):[IKEv2 -> Crypto Engine] Computing DH public key, DH Group 19
Dec 1 21:13:44.402: IKEv2:(SESSION ID = 52,SA ID = 1):(SA ID = 1):[Crypto Engine -> IKEv2] DH key Computation PASSED
Dec 1 21:13:44.402: IKEv2:(SESSION ID = 52,SA ID = 1):Request queued for computation of DH key
Dec 1 21:13:44.403: IKEv2:(SESSION ID = 52,SA ID = 1):[IKEv2 -> Crypto Engine] Computing DH secret key, DH Group 19
Dec 1 21:13:44.408: IKEv2:(SESSION ID = 52,SA ID = 1):(SA ID = 1):[Crypto Engine -> IKEv2] DH key Computation PASSED
Dec 1 21:13:44.408: IKEv2:(SESSION ID = 52,SA ID = 1):Request queued for computation of DH secret
Dec 1 21:13:44.409: IKEv2:(SESSION ID = 52,SA ID = 1):(SA ID = 1):[IKEv2 -> Crypto Engine] Calculate SKEYSEED and create rekeyed IKEv2 SA
Dec 1 21:13:44.409: IKEv2:(SESSION ID = 52,SA ID = 1):(SA ID = 1):[Crypto Engine -> IKEv2] SKEYSEED calculation and creation of rekeyed IKEv2 SA PASSED
Dec 1 21:13:44.409: IKEv2:IKEv2 responder - no config data to send in IKE_SA_INIT exch
Dec 1 21:13:44.409: IKEv2:(SESSION ID = 52,SA ID = 1):Generating IKE_SA_INIT message
Dec 1 21:13:44.409: IKEv2:(SESSION ID = 52,SA ID = 1):IKE Proposal: 1, SPI size: 0 (initial negotiation),
Num. transforms: 4
AES-CBC SHA512 SHA512 DH_GROUP_256_ECP/Group 19
Dec 1 21:13:44.409: IKEv2:(SA ID = 1):[IKEv2 -> PKI] Retrieve configured trustpoint(s)
Dec 1 21:13:44.409: IKEv2:(SA ID = 1):[PKI -> IKEv2] Retrieved trustpoint(s): 'ECH-IOS-CA' 'SLA-TrustPoint' 'TP-self-signed-3693526534'
Dec 1 21:13:44.409: IKEv2:(SA ID = 1):[IKEv2 -> PKI] Get Public Key Hashes of trustpoints
Dec 1 21:13:44.409: IKEv2:(SA ID = 1):[PKI -> IKEv2] Getting of Public Key Hashes of trustpoints PASSED

Dec 1 21:13:44.410: IKEv2:(SESSION ID = 52,SA ID = 1):Sending Packet [To 2.2.2.2:500/From 1.1.1.1:500/VRF i0:f0]
Initiator SPI : AA3C74EE26AAC7C5 - Responder SPI : 0DE0329537507746 Message id: 0
IKEv2 IKE_SA_INIT Exchange RESPONSE
Payload contents:
SA KE N VID VID VID VID NOTIFY(NAT_DETECTION_SOURCE_IP) NOTIFY(NAT_DETECTION_DESTINATION_IP) CERTREQ NOTIFY(HTTP_CERT_LOOKUP_SUPPORTED)

Dec 1 21:13:44.410: IKEv2:(SESSION ID = 52,SA ID = 1):Completed SA init exchange
Dec 1 21:13:44.410: IKEv2:(SESSION ID = 52,SA ID = 1):Starting timer (30 sec) to wait for auth message

Dec 1 21:13:44.438: IKEv2:(SESSION ID = 52,SA ID = 1):Received Packet [From 2.2.2.2:500/To 1.1.1.1:500/VRF i0:f0]
Initiator SPI : AA3C74EE26AAC7C5 - Responder SPI : 0DE0329537507746 Message id: 1
IKEv2 IKE_AUTH Exchange REQUEST
Payload contents:

Dec 1 21:13:44.438: IKEv2:(SESSION ID = 52,SA ID = 1):parsing ENCR payload
Dec 1 21:13:44.438: IKEv2:(SESSION ID = 52,SA ID = 1):parsing VID payload VID
Dec 1 21:13:44.438: IKEv2:(SESSION ID = 52,SA ID = 1):parsing IDi payload IDi
Dec 1 21:13:44.438: IKEv2:(SESSION ID = 52,SA ID = 1):parsing AUTH payload AUTH
Dec 1 21:13:44.438: IKEv2:(SESSION ID = 52,SA ID = 1):parsing SA payload SA
Dec 1 21:13:44.438: IKEv2:(SESSION ID = 52,SA ID = 1):parsing TSi payload TSi
Dec 1 21:13:44.438: IKEv2:(SESSION ID = 52,SA ID = 1):parsing TSr payload TSr
Dec 1 21:13:44.438: IKEv2:(SESSION ID = 52,SA ID = 1):parsing NOTIFY payload NOTIFY(INITIAL_CONTACT)
Dec 1 21:13:44.438: IKEv2:(SESSION ID = 52,SA ID = 1):parsing NOTIFY payload NOTIFY(SET_WINDOW_SIZE)
Dec 1 21:13:44.438: IKEv2:(SESSION ID = 52,SA ID = 1):parsing NOTIFY payload NOTIFY(ESP_TFC_NO_SUPPORT)
Dec 1 21:13:44.438: IKEv2:(SESSION ID = 52,SA ID = 1):parsing NOTIFY payload NOTIFY(NON_FIRST_FRAGS)

Dec 1 21:13:44.438: IKEv2:(SESSION ID = 52,SA ID = 1):Stopping timer to wait for auth message
Dec 1 21:13:44.439: IKEv2:(SESSION ID = 52,SA ID = 1):Checking NAT discovery
Dec 1 21:13:44.439: IKEv2:(SESSION ID = 52,SA ID = 1):NAT not found
Dec 1 21:13:44.439: IKEv2:(SESSION ID = 52,SA ID = 1):Searching policy based on peer's identity '2.2.2.2' of type 'IPv4 address'
Dec 1 21:13:44.439: IKEv2-ERROR:% IKEv2 profile not found
Dec 1 21:13:44.439: IKEv2:% Getting preshared key from profile keyring ECH138220
Dec 1 21:13:44.439: IKEv2:% Matched peer block 'Router-B'
Dec 1 21:13:44.439: IKEv2:(SESSION ID = 52,SA ID = 1):Searching Policy with fvrf 0, local address 1.1.1.1
Dec 1 21:13:44.439: IKEv2:(SESSION ID = 52,SA ID = 1):Using the Default Policy for Proposal
Dec 1 21:13:44.439: IKEv2:(SESSION ID = 52,SA ID = 1):Found Policy 'default'
Dec 1 21:13:44.439: IKEv2:not a VPN-SIP session
Dec 1 21:13:44.439: IKEv2:(SESSION ID = 52,SA ID = 1):Verify peer's policy
Dec 1 21:13:44.439: IKEv2:(SESSION ID = 52,SA ID = 1):Peer's policy verified
Dec 1 21:13:44.439: IKEv2:(SESSION ID = 52,SA ID = 1):Get peer's authentication method
Dec 1 21:13:44.439: IKEv2:(SESSION ID = 52,SA ID = 1):Peer's authentication method is 'PSK'
Dec 1 21:13:44.439: IKEv2:(SESSION ID = 52,SA ID = 1):Get peer's preshared key for 2.2.2.2
Dec 1 21:13:44.439: IKEv2:(SESSION ID = 52,SA ID = 1):Verify peer's authentication data
Dec 1 21:13:44.440: IKEv2:(SESSION ID = 52,SA ID = 1):Use preshared key for id 2.2.2.2, key len 17
Dec 1 21:13:44.440: IKEv2:(SESSION ID = 52,SA ID = 1):[IKEv2 -> Crypto Engine] Generate IKEv2 authentication data
Dec 1 21:13:44.440: IKEv2:(SESSION ID = 52,SA ID = 1):[Crypto Engine -> IKEv2] IKEv2 authentication data generation PASSED
Dec 1 21:13:44.441: IKEv2:(SESSION ID = 52,SA ID = 1):Verification of peer's authentication data PASSED
Dec 1 21:13:44.441: IKEv2:(SESSION ID = 52,SA ID = 1):Processing INITIAL_CONTACT
Dec 1 21:13:44.441: IKEv2:(SESSION ID = 52,SA ID = 1):Processing IKE_AUTH message
Dec 1 21:13:44.442: IKEv2:Requesting IPsec policy verification by ikev2 osal engine

Dec 1 21:13:44.443: IKEv2:(SESSION ID = 52,SA ID = 1):IPSec policy validate request sent for profile Router-A_Profile with psh index 1.

Dec 1 21:13:44.443: IKEv2:(SESSION ID = 52,SA ID = 1):
Dec 1 21:13:44.443: IKEv2:(SESSION ID = 52,SA ID = 1):(SA ID = 1):[IPsec -> IKEv2] Callback received for the validate proposal - PASSED.

Dec 1 21:13:44.443: IKEv2:(SESSION ID = 52,SA ID = 1):PSH: 1 validate proposal callback setting vti_idb GigabitEthernet0/0/2 and ivrf in psh route_info
Dec 1 21:13:44.443: IKEv2:(SESSION ID = 52,SA ID = 1):Get my authentication method
Dec 1 21:13:44.443: IKEv2:(SESSION ID = 52,SA ID = 1):My authentication method is 'PSK'
Dec 1 21:13:44.444: IKEv2:(SESSION ID = 52,SA ID = 1):Get peer's preshared key for 2.2.2.2
Dec 1 21:13:44.444: IKEv2:(SESSION ID = 52,SA ID = 1):Generate my authentication data
Dec 1 21:13:44.444: IKEv2:(SESSION ID = 52,SA ID = 1):Use preshared key for id 1.1.1.1, key len 17
Dec 1 21:13:44.444: IKEv2:(SESSION ID = 52,SA ID = 1):[IKEv2 -> Crypto Engine] Generate IKEv2 authentication data
Dec 1 21:13:44.444: IKEv2:(SESSION ID = 52,SA ID = 1):[Crypto Engine -> IKEv2] IKEv2 authentication data generation PASSED
Dec 1 21:13:44.444: IKEv2:(SESSION ID = 52,SA ID = 1):Get my authentication method
Dec 1 21:13:44.444: IKEv2:(SESSION ID = 52,SA ID = 1):My authentication method is 'PSK'
Dec 1 21:13:44.444: IKEv2:(SESSION ID = 52,SA ID = 1):Generating IKE_AUTH message
Dec 1 21:13:44.444: IKEv2:(SESSION ID = 52,SA ID = 1):Constructing IDr payload: '1.1.1.1' of type 'IPv4 address'
Dec 1 21:13:44.444: IKEv2:(SESSION ID = 52,SA ID = 1):ESP Proposal: 1, SPI size: 4 (IPSec negotiation),
Num. transforms: 3
AES-CBC SHA512 Don't use ESN
Dec 1 21:13:44.444: IKEv2:(SESSION ID = 52,SA ID = 1):Building packet for encryption.
Payload contents:
VID IDr AUTH SA TSi TSr NOTIFY(SET_WINDOW_SIZE) NOTIFY(ESP_TFC_NO_SUPPORT) NOTIFY(NON_FIRST_FRAGS)

Dec 1 21:13:44.445: IKEv2:(SESSION ID = 52,SA ID = 1):Sending Packet [To 2.2.2.2:500/From 1.1.1.1:500/VRF i0:f0]
Initiator SPI : AA3C74EE26AAC7C5 - Responder SPI : 0DE0329537507746 Message id: 1
IKEv2 IKE_AUTH Exchange RESPONSE
Payload contents:
ENCR

Dec 1 21:13:44.445: IKEv2:(SA ID = 1):[IKEv2 -> PKI] Close PKI Session
Dec 1 21:13:44.445: IKEv2:(SA ID = 1):[PKI -> IKEv2] Closing of PKI Session PASSED
Dec 1 21:13:44.445: IKEv2:(SESSION ID = 52,SA ID = 1):IKEV2 SA created; inserting SA into database. SA lifetime timer (86400 sec) started
Dec 1 21:13:44.445: IKEv2:(SESSION ID = 52,SA ID = 1):Session with IKE ID PAIR (2.2.2.2, 1.1.1.1) is UP
Dec 1 21:13:44.445: IKEv2:(SESSION ID = 0,SA ID = 0):IKEv2 MIB tunnel started, tunnel index 1
Dec 1 21:13:44.445: IKEv2:(SESSION ID = 52,SA ID = 1):Load IPSEC key material
Dec 1 21:13:44.446: IKEv2:(SESSION ID = 52,SA ID = 1):(SA ID = 1):[IKEv2 -> IPsec] Create IPsec SA into IPsec database
Dec 1 21:13:44.462: IKEv2:(SESSION ID = 52,SA ID = 1):(SA ID = 1):[IPsec -> IKEv2] Creation of IPsec SA into IPsec database PASSED
Dec 1 21:13:44.464: IKEv2:(SESSION ID = 52,SA ID = 1):Checking for duplicate IKEv2 SA
Dec 1 21:13:44.465: IKEv2:(SESSION ID = 52,SA ID = 1):No duplicate IKEv2 SA found
Dec 1 21:13:44.465: IKEv2:(SESSION ID = 52,SA ID = 1):Starting timer (8 sec) to delete negotiation context
Dec 1 16:14:25: %SW_MATM-4-MACFLAP_NOTIF: Host b252.664d.1a87 in vlan 110 is flapping between port Gi0/1/1 and port Gi0/1/3

Router-A#deb crypto ipsec
Crypto IPSEC debugging is on
Router-A#
Router-A#
Router-A#
Dec 1 21:17:49.704: IPSEC(key_engine): got a queue event with 1 KMI message(s)
Dec 1 21:17:49.704: IDB is NULL : in crypto_ipsec_key_engine_delete_sas (), 6341
Dec 1 21:17:49.705: IPSEC:(SESSION ID = 52) (key_engine_delete_sas) rec'd delete notify from ISAKMP
Dec 1 21:17:49.705: IPSEC:(SESSION ID = 52) (key_engine_delete_sas) delete SA with spi 0xEC2A1DF1 proto 50 for 1.1.1.1
Dec 1 21:17:49.705: IPSEC:(SESSION ID = 52) (delete_sa) deleting SA,
(sa) sa_dest= 1.1.1.1, sa_proto= 50,
sa_spi= 0xEC2A1DF1(3962183153),
sa_trans= esp-aes 256 esp-sha512-hmac , sa_conn_id= 2009
sa_lifetime(k/sec)= (4608000/3600),
(identity) local= 1.1.1.1:0, remote= 2.2.2.2:0,
local_proxy= 172.168.100.0/255.255.255.0/256/0,
remote_proxy= 172.168.102.0/255.255.255.0/256/0
Dec 1 21:17:49.705: IPSEC:(SESSION ID = 52) (delete_sa) deleting SA,
(sa) sa_dest= 2.2.2.2, sa_proto= 50,
sa_spi= 0xC79E0DB1(3349024177),
sa_trans= esp-aes 256 esp-sha512-hmac , sa_conn_id= 2010
sa_lifetime(k/sec)= (4608000/3600),
(identity) local= 1.1.1.1:0, remote= 2.2.2.2:0,
local_proxy= 172.168.100.0/255.255.255.0/256/0,
remote_proxy= 172.168.102.0/255.255.255.0/256/0
Dec 1 21:17:49.705: IPSEC(send_delete_notify_kmi): not sending KEY_ENGINE_DELETE_SAS
Dec 1 21:17:49.706: ipsec_out_sa_hash_idx: sa=0x7575B1E1E078, hash_idx=332, port=500/500, addr=0x8D9BABE6/0x4B7F882E
Dec 1 21:17:49.708: IPSEC:(SESSION ID = 52) (ident_delete_notify_kmi) Failed to send KEY_ENG_DELETE_SAS
Dec 1 21:17:49.709: IPSEC:(SESSION ID = 52) (ident_update_final_flow_stats) Collect Final Stats and update MIB
IPSEC get IKMP peer index from peer 0x7575B1E1ADE8 ikmp handle 0x0
[ident_update_final_flow_stats] : Flow delete complete event received for flow id 0x24000009,peer index 0

Dec 1 21:17:56.244: IPSEC(key_engine): got a queue event with 1 KMI message(s)
Dec 1 21:17:56.244: IPSEC(validate_proposal_request): proposal part #1
Dec 1 21:17:56.244: IPSEC(validate_proposal_request): proposal part #1,
(key eng. msg.) INBOUND local= 1.1.1.1:0, remote= 2.2.2.2:0,
local_proxy= 172.168.100.0/255.255.255.0/256/0,
remote_proxy= 172.168.102.0/255.255.255.0/256/0,
protocol= ESP, transform= esp-aes 256 esp-sha512-hmac (Tunnel), esn= FALSE,
lifedur= 0s and 0kb,
spi= 0x0(0), conn_id= 0, keysize= 256, flags= 0x0
Dec 1 21:17:56.245: Crypto mapdb : proxy_match
src addr : 172.168.100.0
dst addr : 172.168.102.0
protocol : 0
src port : 0
dst port : 0
Dec 1 21:17:56.245: Session ID: 53 Proposal Accepted:
Map:Router-A, Dualstack: N
loc: 1.1.1.1, rem: 2.2.2.2
l_proxy: 172.168.100.0/0/0//24, r_proxy: 172.168.102.0/0/0//24
Dec 1 21:17:56.245: (ipsec_process_proposal)Map Accepted: Router-A, 1
Dec 1 21:17:56.248: IPSEC(key_engine): got a queue event with 1 KMI message(s)
Dec 1 21:17:56.248: Crypto mapdb : proxy_match
src addr : 172.168.100.0
dst addr : 172.168.102.0
protocol : 256
src port : 0
dst port : 0
Dec 1 21:17:56.248: IPSEC:(SESSION ID = 53) (crypto_ipsec_create_ipsec_sas) Map found Router-A, 1
Dec 1 21:17:56.249: IPSEC:(SESSION ID = 53) (get_old_outbound_sa_for_peer) No outbound SA found for peer 7575B1E1ADE8
Dec 1 21:17:56.249: IPSEC:(SESSION ID = 53) (create_sa) sa created,
(sa) sa_dest= 1.1.1.1, sa_proto= 50,
sa_spi= 0x9E22AD78(2653072760),
sa_trans= esp-aes 256 esp-sha512-hmac , sa_conn_id= 2011
sa_lifetime(k/sec)= (4608000/3600),
(identity) local= 1.1.1.1:0, remote= 2.2.2.2:0,
local_proxy= 172.168.100.0/255.255.255.0/256/0,
remote_proxy= 172.168.102.0/255.255.255.0/256/0
Dec 1 21:17:56.249: ipsec_out_sa_hash_idx: sa=0x7575B1E1E188, hash_idx=332, port=500/500, addr=0x8D9BABE6/0x4B7F882E
Dec 1 21:17:56.249: crypto_ipsec_hook_out_sa: ipsec_out_sa_hash_array[332]=0x7575B1E1E188
Dec 1 21:17:56.249: IPSEC:(SESSION ID = 53) (create_sa) sa created,
(sa) sa_dest= 2.2.2.2, sa_proto= 50,
sa_spi= 0xE29355B1(3801306545),
sa_trans= esp-aes 256 esp-sha512-hmac , sa_conn_id= 2012
sa_lifetime(k/sec)= (4608000/3600),
(identity) local= 1.1.1.1:0, remote= 2.2.2.2:0,
local_proxy= 172.168.100.0/255.255.255.0/256/0,
remote_proxy= 172.168.102.0/255.255.255.0/256/0

Router-B#
000843: *Dec 1 21:13:44.508: IKEv2:% Getting preshared key from profile keyring ECH138220
000844: *Dec 1 21:13:44.508: IKEv2:% Matched peer block 'Router-A'
000845: *Dec 1 21:13:44.508: IKEv2:(SESSION ID = 0,SA ID = 0):Searching Policy with fvrf 0, local address 2.2.2.2
000846: *Dec 1 21:13:44.508: IKEv2:(SESSION ID = 0,SA ID = 0):Using the Default Policy for Proposal
000847: *Dec 1 21:13:44.508: IKEv2:(SESSION ID = 0,SA ID = 0):Found Policy 'default'
000848: *Dec 1 21:13:44.509: IKEv2:(SESSION ID = 1,SA ID = 1):[IKEv2 -> Crypto Engine] Computing DH public key, DH Group 19
000849: *Dec 1 21:13:44.510: IKEv2:(SESSION ID = 1,SA ID = 1):(SA ID = 1):[Crypto Engine -> IKEv2] DH key Computation PASSED
000850: *Dec 1 21:13:44.510: IKEv2:(SESSION ID = 1,SA ID = 1):Request queued for computation of DH key
000851: *Dec 1 21:13:44.510: IKEv2:(SESSION ID = 1,SA ID = 1):IKEv2 initiator - no config data to send in IKE_SA_INIT exch
000852: *Dec 1 21:13:44.510: IKEv2:(SESSION ID = 1,SA ID = 1):Generating IKE_SA_INIT message
000853: *Dec 1 21:13:44.510: IKEv2:(SESSION ID = 1,SA ID = 1):IKE Proposal: 1, SPI size: 0 (initial negotiation),
Num. transforms: 9
AES-CBC SHA512 SHA384 SHA512 SHA384 DH_GROUP_256_ECP/Group 19 DH_GROUP_2048_MODP/Group 14 DH_GROUP_521_ECP/Group 21 DH_GROUP_1536_MODP/Group 5

000854: *Dec 1 21:13:44.511: IKEv2:(SESSION ID = 1,SA ID = 1):Sending Packet [To 1.1.1.1:500/From 2.2.2.2:500/VRF i0:f0]
Initiator SPI : AA3C74EE26AAC7C5 - Responder SPI : 0000000000000000 Message id: 0
IKEv2 IKE_SA_INIT Exchange REQUEST
Payload contents:
SA KE N VID VID VID VID NOTIFY(NAT_DETECTION_SOURCE_IP) NOTIFY(NAT_DETECTION_DESTINATION_IP)

000855: *Dec 1 21:13:44.511: IKEv2:(SESSION ID = 1,SA ID = 1):Insert SA

000856: *Dec 1 21:13:44.542: IKEv2:(SESSION ID = 1,SA ID = 1):Received Packet [From 1.1.1.1:500/To 2.2.2.2:500/VRF i0:f0]
Initiator SPI : AA3C74EE26AAC7C5 - Responder SPI : 0DE0329537507746 Message id: 0
IKEv2 IKE_SA_INIT Exchange RESPONSE
Payload contents:

000857: *Dec 1 21:13:44.542: IKEv2:(SESSION ID = 1,SA ID = 1):parsing SA payload SA
000858: *Dec 1 21:13:44.542: IKEv2:(SESSION ID = 1,SA ID = 1):parsing KE payload KE
000859: *Dec 1 21:13:44.542: IKEv2:(SESSION ID = 1,SA ID = 1):parsing N payload N
000860: *Dec 1 21:13:44.542: IKEv2:(SESSION ID = 1,SA ID = 1):parsing VID payload VID
000861: *Dec 1 21:13:44.542: IKEv2:(SESSION ID = 1,SA ID = 1):parsing VID payload VID
000862: *Dec 1 21:13:44.543: IKEv2:(SESSION ID = 1,SA ID = 1):parsing VID payload VID
000863: *Dec 1 21:13:44.543: IKEv2:(SESSION ID = 1,SA ID = 1):parsing VID payload VID
000864: *Dec 1 21:13:44.543: IKEv2:(SESSION ID = 1,SA ID = 1):parsing NOTIFY payload NOTIFY(NAT_DETECTION_SOURCE_IP)
000865: *Dec 1 21:13:44.543: IKEv2:(SESSION ID = 1,SA ID = 1):parsing NOTIFY payload NOTIFY(NAT_DETECTION_DESTINATION_IP)
000866: *Dec 1 21:13:44.543: IKEv2:(SESSION ID = 1,SA ID = 1):parsing CERTREQ payload CERTREQ
000867: *Dec 1 21:13:44.543: IKEv2:(SESSION ID = 1,SA ID = 1):parsing NOTIFY payload NOTIFY(HTTP_CERT_LOOKUP_SUPPORTED)

000868: *Dec 1 21:13:44.543: IKEv2:(SESSION ID = 1,SA ID = 1):Processing IKE_SA_INIT message
000869: *Dec 1 21:13:44.543: IKEv2:(SESSION ID = 1,SA ID = 1):Verify SA init message
000870: *Dec 1 21:13:44.543: IKEv2:(SESSION ID = 1,SA ID = 1):Processing IKE_SA_INIT message
000871: *Dec 1 21:13:44.543: IKEv2:(SESSION ID = 1,SA ID = 1):Checking NAT discovery
000872: *Dec 1 21:13:44.543: IKEv2:(SESSION ID = 1,SA ID = 1):NAT not found
000873: *Dec 1 21:13:44.543: IKEv2:(SESSION ID = 1,SA ID = 1):[IKEv2 -> Crypto Engine] Computing DH secret key, DH Group 19
000874: *Dec 1 21:13:44.548: IKEv2:(SESSION ID = 1,SA ID = 1):(SA ID = 1):[Crypto Engine -> IKEv2] DH key Computation PASSED
000875: *Dec 1 21:13:44.548: IKEv2:(SESSION ID = 1,SA ID = 1):Request queued for computation of DH secret
000876: *Dec 1 21:13:44.548: IKEv2:(SESSION ID = 1,SA ID = 1):(SA ID = 1):[IKEv2 -> Crypto Engine] Calculate SKEYSEED and create rekeyed IKEv2 SA
000877: *Dec 1 21:13:44.548: IKEv2:(SESSION ID = 1,SA ID = 1):(SA ID = 1):[Crypto Engine -> IKEv2] SKEYSEED calculation and creation of rekeyed IKEv2 SA PASSED
000878: *Dec 1 21:13:44.548: IKEv2:(SESSION ID = 1,SA ID = 1):Completed SA init exchange
000879: *Dec 1 21:13:44.548: IKEv2:(SESSION ID = 1,SA ID = 1):Check for EAP exchange
000880: *Dec 1 21:13:44.548: IKEv2:(SESSION ID = 1,SA ID = 1):Generate my authentication data
000881: *Dec 1 21:13:44.549: IKEv2:(SESSION ID = 1,SA ID = 1):Use preshared key for id 2.2.2.2, key len 17
000882: *Dec 1 21:13:44.549: IKEv2:(SESSION ID = 1,SA ID = 1):[IKEv2 -> Crypto Engine] Generate IKEv2 authentication data
000883: *Dec 1 21:13:44.549: IKEv2:(SESSION ID = 1,SA ID = 1):[Crypto Engine -> IKEv2] IKEv2 authentication data generation PASSED
000884: *Dec 1 21:13:44.549: IKEv2:(SESSION ID = 1,SA ID = 1):Get my authentication method
000885: *Dec 1 21:13:44.549: IKEv2:(SESSION ID = 1,SA ID = 1):My authentication method is 'PSK'
000886: *Dec 1 21:13:44.549: IKEv2:(SESSION ID = 1,SA ID = 1):Check for EAP exchange
000887: *Dec 1 21:13:44.549: IKEv2:(SESSION ID = 1,SA ID = 1):Generating IKE_AUTH message
000888: *Dec 1 21:13:44.549: IKEv2:(SESSION ID = 1,SA ID = 1):Constructing IDi payload: '2.2.2.2' of type 'IPv4 address'
000889: *Dec 1 21:13:44.549: IKEv2:(SESSION ID = 1,SA ID = 1):ESP Proposal: 1, SPI size: 4 (IPSec negotiation),
Num. transforms: 3
AES-CBC SHA512 Don't use ESN
000890: *Dec 1 21:13:44.549: IKEv2:(SESSION ID = 1,SA ID = 1):Building packet for encryption.
Payload contents:
VID IDi AUTH SA TSi TSr NOTIFY(INITIAL_CONTACT) NOTIFY(SET_WINDOW_SIZE) NOTIFY(ESP_TFC_NO_SUPPORT) NOTIFY(NON_FIRST_FRAGS)

000891: *Dec 1 21:13:44.550: IKEv2:(SESSION ID = 1,SA ID = 1):Sending Packet [To 1.1.1.1:500/From 2.2.2.2:500/VRF i0:f0]
Initiator SPI : AA3C74EE26AAC7C5 - Responder SPI : 0DE0329537507746 Message id: 1
IKEv2 IKE_AUTH Exchange REQUEST
Payload contents:
ENCR

000892: *Dec 1 21:13:44.579: IKEv2:(SESSION ID = 1,SA ID = 1):Received Packet [From 1.1.1.1:500/To 2.2.2.2:500/VRF i0:f0]
Initiator SPI : AA3C74EE26AAC7C5 - Responder SPI : 0DE0329537507746 Message id: 1
IKEv2 IKE_AUTH Exchange RESPONSE
Payload contents:

000893: *Dec 1 21:13:44.579: IKEv2:(SESSION ID = 1,SA ID = 1):parsing ENCR payload
000894: *Dec 1 21:13:44.579: IKEv2:(SESSION ID = 1,SA ID = 1):parsing VID payload VID
000895: *Dec 1 21:13:44.579: IKEv2:(SESSION ID = 1,SA ID = 1):parsing IDr payload IDr
000896: *Dec 1 21:13:44.579: IKEv2:(SESSION ID = 1,SA ID = 1):parsing AUTH payload AUTH
000897: *Dec 1 21:13:44.580: IKEv2:(SESSION ID = 1,SA ID = 1):parsing SA payload SA
000898: *Dec 1 21:13:44.580: IKEv2:(SESSION ID = 1,SA ID = 1):parsing TSi payload TSi
000899: *Dec 1 21:13:44.580: IKEv2:(SESSION ID = 1,SA ID = 1):parsing TSr payload TSr
000900: *Dec 1 21:13:44.580: IKEv2:(SESSION ID = 1,SA ID = 1):parsing NOTIFY payload NOTIFY(SET_WINDOW_SIZE)
000901: *Dec 1 21:13:44.580: IKEv2:(SESSION ID = 1,SA ID = 1):parsing NOTIFY payload NOTIFY(ESP_TFC_NO_SUPPORT)
000902: *Dec 1 21:13:44.580: IKEv2:(SESSION ID = 1,SA ID = 1):parsing NOTIFY payload NOTIFY(NON_FIRST_FRAGS)

000903: *Dec 1 21:13:44.580: IKEv2:(SESSION ID = 1,SA ID = 1):Process auth response notify
000904: *Dec 1 21:13:44.580: IKEv2:(SESSION ID = 1,SA ID = 1):Searching policy based on peer's identity '1.1.1.1' of type 'IPv4 address'
000905: *Dec 1 21:13:44.580: IKEv2:(SESSION ID = 1,SA ID = 1):Searching Policy with fvrf 0, local address 2.2.2.2
000906: *Dec 1 21:13:44.581: IKEv2:(SESSION ID = 1,SA ID = 1):Using the Default Policy for Proposal
000907: *Dec 1 21:13:44.581: IKEv2:(SESSION ID = 1,SA ID = 1):Found Policy 'default'
000908: *Dec 1 21:13:44.581: IKEv2:(SESSION ID = 1,SA ID = 1):Verify peer's policy
000909: *Dec 1 21:13:44.581: IKEv2:(SESSION ID = 1,SA ID = 1):Peer's policy verified
000910: *Dec 1 21:13:44.581: IKEv2:(SESSION ID = 1,SA ID = 1):Get peer's authentication method
000911: *Dec 1 21:13:44.581: IKEv2:(SESSION ID = 1,SA ID = 1):Peer's authentication method is 'PSK'
000912: *Dec 1 21:13:44.581: IKEv2:(SESSION ID = 1,SA ID = 1):Get peer's preshared key for 1.1.1.1
000913: *Dec 1 21:13:44.581: IKEv2:(SESSION ID = 1,SA ID = 1):Verify peer's authentication data
000914: *Dec 1 21:13:44.581: IKEv2:(SESSION ID = 1,SA ID = 1):Use preshared key for id 1.1.1.1, key len 17
000915: *Dec 1 21:13:44.581: IKEv2:(SESSION ID = 1,SA ID = 1):[IKEv2 -> Crypto Engine] Generate IKEv2 authentication data
000916: *Dec 1 21:13:44.581: IKEv2:(SESSION ID = 1,SA ID = 1):[Crypto Engine -> IKEv2] IKEv2 authentication data generation PASSED
000917: *Dec 1 21:13:44.582: IKEv2:(SESSION ID = 1,SA ID = 1):Verification of peer's authentication data PASSED
000918: *Dec 1 21:13:44.582: IKEv2:(SESSION ID = 1,SA ID = 1):Check for EAP exchange
000919: *Dec 1 21:13:44.582: IKEv2:(SESSION ID = 1,SA ID = 1):Processing IKE_AUTH message
000920: *Dec 1 21:13:44.583: IKEv2:Requesting IPsec policy verification by ikev2 osal engine

000921: *Dec 1 21:13:44.583: IKEv2:(SESSION ID = 1,SA ID = 1):IPSec policy validate request sent for profile Router-B_Profile with psh index 1.

000922: *Dec 1 21:13:44.584: IKEv2:(SESSION ID = 1,SA ID = 1):(SA ID = 1):[IPsec -> IKEv2] Callback received for the validate proposal - PASSED.

000923: *Dec 1 21:13:44.584: IKEv2:(SESSION ID = 1,SA ID = 1):PSH: 1 validate proposal callback setting vti_idb GigabitEthernet0/0/0 and ivrf in psh route_info
000924: *Dec 1 21:13:44.584: IKEv2:(SESSION ID = 1,SA ID = 1):IKEV2 SA created; inserting SA into database. SA lifetime timer (86400 sec) started
000925: *Dec 1 21:13:44.585: IKEv2:(SESSION ID = 1,SA ID = 1):Session with IKE ID PAIR (1.1.1.1, 2.2.2.2) is UP
000926: *Dec 1 21:13:44.585: IKEv2:(SESSION ID = 0,SA ID = 0):IKEv2 MIB tunnel started, tunnel index 1
000927: *Dec 1 21:13:44.585: IKEv2:(SESSION ID = 1,SA ID = 1):Load IPSEC key material
000928: *Dec 1 21:13:44.585: IKEv2:(SESSION ID = 1,SA ID = 1):(SA ID = 1):[IKEv2 -> IPsec] Create IPsec SA into IPsec database
000929: *Dec 1 21:13:44.601: IKEv2:(SESSION ID = 1,SA ID = 1):(SA ID = 1):[IPsec -> IKEv2] Creation of IPsec SA into IPsec database PASSED
000930: *Dec 1 21:13:44.601: IKEv2:(SESSION ID = 1,SA ID = 1):Checking for duplicate IKEv2 SA
000931: *Dec 1 21:13:44.601: IKEv2:(SESSION ID = 1,SA ID = 1):No duplicate IKEv2 SA found
000932: *Dec 1 21:14:30: %SEC-6-IPACCESSLOGP: list SDM_1 permitted udp 172.168.102.6(33548) -> 172.168.100.4(162), 1 packet

Router-B#deb crypto ipsec
Crypto IPSEC debugging is on
Router-B#
000933: *Dec 1 21:17:30: %SEC-6-IPACCESSLOGP: list SSH_Access permitted tcp 1.1.1.1(5902) -> 2.2.2.2(22), 1 packet
Router-B#
Router-B#clear crypto ikev2 sa
Router-B#
000934: *Dec 1 21:17:49.813: IPSEC(key_engine): got a queue event with 1 KMI message(s)
000935: *Dec 1 21:17:49.813: IDB is NULL : in crypto_ipsec_key_engine_delete_sas (), 6341
000936: *Dec 1 21:17:49.813: IPSEC:(SESSION ID = 1) (key_engine_delete_sas) rec'd delete notify from ISAKMP
000937: *Dec 1 21:17:49.814: IPSEC:(SESSION ID = 1) (key_engine_delete_sas) delete SA with spi 0xC79E0DB1 proto 50 for 2.2.2.2
000938: *Dec 1 21:17:49.814: IPSEC:(SESSION ID = 1) (delete_sa) deleting SA,
(sa) sa_dest= 2.2.2.2, sa_proto= 50,
sa_spi= 0xC79E0DB1(3349024177),
sa_trans= esp-aes 256 esp-sha512-hmac , sa_conn_id= 2010
sa_lifetime(k/sec)= (4608000/3600),
(identity) local= 2.2.2.2:0, remote= 1.1.1.1:0,
local_proxy= 172.168.102.0/255.255.255.0/256/0,
remote_proxy= 172.168.100.0/255.255.255.0/256/0
000939: *Dec 1 21:17:49.814: IPSEC:(SESSION ID = 1) (delete_sa) deleting SA,
(sa) sa_dest= 1.1.1.1, sa_proto= 50,
sa_spi= 0xEC2A1DF1(3962183153),
sa_trans= esp-aes 256 esp-sha512-hmac , sa_conn_id= 2009
sa_lifetime(k/sec)= (4608000/3600),
(identity) local= 2.2.2.2:0, remote= 1.1.1.1:0,
local_proxy= 172.168.102.0/255.255.255.0/256/0,
remote_proxy= 172.168.100.0/255.255.255.0/256/0
000940: *Dec 1 21:17:49.814: IPSEC(send_delete_notify_kmi): not sending KEY_ENGINE_DELETE_SAS
000941: *Dec 1 21:17:49.814: ipsec_out_sa_hash_idx: sa=0x7B5076D45238, hash_idx=332, port=500/500, addr=0x4B7F882E/0x8D9BABE6
000942: *Dec 1 21:17:49.816: IPSEC:(SESSION ID = 1) (ident_delete_notify_kmi) Failed to send KEY_ENG_DELETE_SAS
000943: *Dec 1 21:17:49.816: IPSEC:(SESSION ID = 1) (ident_update_final_flow_stats) Collect Final Stats and update MIB
IPSEC get IKMP peer index from peer 0x7B5076D41C98 ikmp handle 0x0
[ident_update_final_flow_stats] : Flow delete complete event received for flow id 0x2400000A,peer index 0

000944: *Dec 1 21:17:56.314: IPSEC(sa_request): ,
(key eng. msg.) OUTBOUND local= 2.2.2.2:500, remote= 1.1.1.1:500,
local_proxy= 172.168.102.0/255.255.255.0/256/0,
remote_proxy= 172.168.100.0/255.255.255.0/256/0,
protocol= ESP, transform= esp-aes 256 esp-sha512-hmac (Tunnel), esn= FALSE,
lifedur= 3600s and 4608000kb,
spi= 0x0(0), conn_id= 0, keysize= 256, flags= 0x0
000945: *Dec 1 21:17:56.381: IPSEC(key_engine): got a queue event with 1 KMI message(s)
000946: *Dec 1 21:17:56.381: IPSEC(validate_proposal_request): proposal part #1
000947: *Dec 1 21:17:56.381: IPSEC(validate_proposal_request): proposal part #1,
(key eng. msg.) INBOUND local= 2.2.2.2:0, remote= 1.1.1.1:0,
local_proxy= 172.168.102.0/255.255.255.0/256/0,
remote_proxy= 172.168.100.0/255.255.255.0/256/0,
protocol= ESP, transform= esp-aes 256 esp-sha512-hmac (Tunnel), esn= FALSE,
lifedur= 0s and 0kb,
spi= 0x0(0), conn_id= 0, keysize= 256, flags= 0x0
000948: *Dec 1 21:17:56.382: Crypto mapdb : proxy_match
src addr : 172.168.102.0
dst addr : 172.168.100.0
protocol : 0
src port : 0
dst port : 0
000949: *Dec 1 21:17:56.382: Session ID: 1 Proposal Accepted:
Map:Router-B, Dualstack: N
loc: 2.2.2.2, rem: 1.1.1.1
l_proxy: 172.168.102.0/0/0//24, r_proxy: 172.168.100.0/0/0//24
000950: *Dec 1 21:17:56.382: (ipsec_process_proposal)Map Accepted: Router-B, 1
000951: *Dec 1 21:17:56.384: IPSEC(key_engine): got a queue event with 1 KMI message(s)
000952: *Dec 1 21:17:56.385: Crypto mapdb : proxy_match
src addr : 172.168.102.0
dst addr : 172.168.100.0
protocol : 256
src port : 0
dst port : 0
000953: *Dec 1 21:17:56.385: IPSEC:(SESSION ID = 1) (crypto_ipsec_create_ipsec_sas) Map found Router-B, 1
000954: *Dec 1 21:17:56.385: IPSEC:(SESSION ID = 1) (get_old_outbound_sa_for_peer) No outbound SA found for peer 7B5076D41C98
000955: *Dec 1 21:17:56.386: IPSEC:(SESSION ID = 1) (create_sa) sa created,
(sa) sa_dest= 2.2.2.2, sa_proto= 50,
sa_spi= 0xE29355B1(3801306545),
sa_trans= esp-aes 256 esp-sha512-hmac , sa_conn_id= 2012
sa_lifetime(k/sec)= (4608000/3600),
(identity) local= 2.2.2.2:0, remote= 1.1.1.1:0,
local_proxy= 172.168.102.0/255.255.255.0/256/0,
remote_proxy= 172.168.100.0/255.255.255.0/256/0
000956: *Dec 1 21:17:56.386: ipsec_out_sa_hash_idx: sa=0x7B5076D45348, hash_idx=332, port=500/500, addr=0x4B7F882E/0x8D9BABE6
000957: *Dec 1 21:17:56.386: crypto_ipsec_hook_out_sa: ipsec_out_sa_hash_array[332]=0x7B5076D45348
000958: *Dec 1 21:17:56.386: IPSEC:(SESSION ID = 1) (create_sa) sa created,
(sa) sa_dest= 1.1.1.1, sa_proto= 50,
sa_spi= 0x9E22AD78(2653072760),
sa_trans= esp-aes 256 esp-sha512-hmac , sa_conn_id= 2011
sa_lifetime(k/sec)= (4608000/3600),
(identity) local= 2.2.2.2:0, remote= 1.1.1.1:0,
local_proxy= 172.168.102.0/255.255.255.0/256/0,
remote_proxy= 172.168.100.0/255.255.255.0/256/0

Router-B#
000959: *Dec 1 21:20:30: %SEC-6-IPACCESSLOGP: list SDM_1 permitted udp 172.168.102.6(33548) -> 172.168.100.4(162), 1 packet

Thanks.....

[Rob Ingram · Accepted solution]

@Elito Haylett if the tunnel is up but no traffic is passing this usually indicates a NAT or routing issue.

Please provide the full output of "show crypto ikev2 sa" and "show crypto ipsec sa" when the tunnel is up

Also provide the full configuration of the router.

[Elito Haylett]

Here are the "show IKEv2 sa" and "show crypto ipsec sa" for both sides.. config will follow after I sanatize it.

Router-A#sho crypto ikev2 sa
IPv4 Crypto IKEv2 SA

Tunnel-id Local Remote fvrf/ivrf Status
1 1.1.1.1/500 2.2.2.2/500 none/none READY
Encr: AES-CBC, keysize: 256, PRF: SHA512, Hash: SHA512, DH Grp:19, Auth sign: PSK, Auth verify: PSK
Life/Active Time: 86400/69518 sec
CE id: 1007, Session-id: 7
Local spi: 8BC065F2C90D455C Remote spi: 48C485BEE7370A7F

IPv6 Crypto IKEv2 SA

Router-A#sho crypto ipsec sa

interface: GigabitEthernet0/0/2
Crypto map tag: Router-A, local addr 1.1.1.1

protected vrf: (none)
local ident (addr/mask/prot/port): (172.168.100.0/255.255.255.0/0/0)
remote ident (addr/mask/prot/port): (172.168.101.0/255.255.255.0/0/0)
current_peer 2.2.2.2 port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0

local crypto endpt.: 1.1.1.1, remote crypto endpt.: 2.2.2.2
plaintext mtu 1500, path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet0/0/2
current outbound spi: 0x0(0)
PFS (Y/N): N, DH group: none

inbound esp sas:

inbound ah sas:

inbound pcp sas:

outbound esp sas:

outbound ah sas:

outbound pcp sas:

protected vrf: (none)
local ident (addr/mask/prot/port): (172.168.100.0/255.255.255.0/0/0)
remote ident (addr/mask/prot/port): (172.168.102.0/255.255.255.0/0/0)
current_peer 2.2.2.2 port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 47, #pkts decrypt: 47, #pkts verify: 47
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0

local crypto endpt.: 1.1.1.1, remote crypto endpt.: 2.2.2.2
plaintext mtu 1422, path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet0/0/2
current outbound spi: 0xAAB1D072(2863779954)
PFS (Y/N): N, DH group: none

inbound esp sas:
spi: 0xF5E2F580(4125291904)
transform: esp-256-aes esp-sha512-hmac ,
in use settings ={Tunnel, }
conn id: 2061, flow_id: ESG:61, sibling_flags FFFFFFFF80000048, crypto map: Router-A, initiator : False
sa timing: remaining key lifetime (k/sec): (4607984/2396)
IV size: 16 bytes
replay detection support: Y
Status: ACTIVE(ACTIVE)

inbound ah sas:

inbound pcp sas:

outbound esp sas:
spi: 0xAAB1D072(2863779954)
transform: esp-256-aes esp-sha512-hmac ,
in use settings ={Tunnel, }
conn id: 2062, flow_id: ESG:62, sibling_flags FFFFFFFF80000048, crypto map: Router-A, initiator : False
sa timing: remaining key lifetime (k/sec): (4608000/2396)
IV size: 16 bytes
replay detection support: Y
Status: ACTIVE(ACTIVE)

outbound ah sas:

outbound pcp sas:

protected vrf: (none)
local ident (addr/mask/prot/port): (172.168.100.0/255.255.255.0/0/0)
remote ident (addr/mask/prot/port): (172.168.103.0/255.255.255.0/0/0)
current_peer 2.2.2.2 port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0

local crypto endpt.: 1.1.1.1, remote crypto endpt.: 2.2.2.2
plaintext mtu 1500, path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet0/0/2
current outbound spi: 0x0(0)
PFS (Y/N): N, DH group: none

inbound esp sas:

inbound ah sas:

inbound pcp sas:

outbound esp sas:

outbound ah sas:

outbound pcp sas:

Router-B#sho crypto ikev2 sa
IPv4 Crypto IKEv2 SA

Tunnel-id Local Remote fvrf/ivrf Status
1 2.2.2.2/500 1.1.1.1/500 none/none READY
Encr: AES-CBC, keysize: 256, PRF: SHA512, Hash: SHA512, DH Grp:19, Auth sign: PSK, Auth verify: PSK
Life/Active Time: 86400/68626 sec
CE id: 1007, Session-id: 7
Local spi: 48C485BEE7370A7F Remote spi: 8BC065F2C90D455C

IPv6 Crypto IKEv2 SA

Router-B#sho crypto ipsec sa

interface: GigabitEthernet0/0/0
Crypto map tag: Router-B, local addr 2.2.2.2

protected vrf: (none)
local ident (addr/mask/prot/port): (172.168.102.0/255.255.255.0/0/0)
remote ident (addr/mask/prot/port): (172.168.100.0/255.255.255.0/0/0)
current_peer 1.1.1.1 port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 2525, #pkts encrypt: 2525, #pkts digest: 2525
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0

local crypto endpt.: 2.2.2.2, remote crypto endpt.: 1.1.1.1
plaintext mtu 1422, path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet0/0/0
current outbound spi: 0xF5E2F580(4125291904)
PFS (Y/N): N, DH group: none

inbound esp sas:
spi: 0xAAB1D072(2863779954)
transform: esp-256-aes esp-sha512-hmac ,
in use settings ={Tunnel, }
conn id: 2062, flow_id: ESG:62, sibling_flags FFFFFFFF80004048, crypto map: Router-B, initiator : True
sa timing: remaining key lifetime (k/sec): (4608000/1571)
IV size: 16 bytes
replay detection support: Y
Status: ACTIVE(ACTIVE)

inbound ah sas:

inbound pcp sas:

outbound esp sas:
spi: 0xF5E2F580(4125291904)
transform: esp-256-aes esp-sha512-hmac ,
in use settings ={Tunnel, }
conn id: 2061, flow_id: ESG:61, sibling_flags FFFFFFFF80004048, crypto map: Router-B, initiator : True
sa timing: remaining key lifetime (k/sec): (4607974/1571)
IV size: 16 bytes
replay detection support: Y
Status: ACTIVE(ACTIVE)

outbound ah sas:

outbound pcp sas:

protected vrf: (none)
local ident (addr/mask/prot/port): (172.168.102.0/255.255.255.0/0/0)
remote ident (addr/mask/prot/port): (172.168.101.0/255.255.255.0/0/0)
current_peer 1.1.1.1 port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0

local crypto endpt.: 2.2.2.2, remote crypto endpt.: 1.1.1.1
plaintext mtu 1500, path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet0/0/0
current outbound spi: 0x0(0)
PFS (Y/N): N, DH group: none

inbound esp sas:

inbound ah sas:

inbound pcp sas:

outbound esp sas:

outbound ah sas:

outbound pcp sas:

[Rob Ingram]

@Elito Haylett Router-B is encrypting traffic and Router-A is decrypting (some) traffic. Router-A is not encrypting any traffic either nothing is being sent or traffic is unintentially translated and therefore does not match the crypto ACL.

Check routing and make traffic is routed to the router and check NAT to ensure VPN traffic is not unintentially translated (modify the NAT ACL and deny traffic between the local and remote VPN networks).

Also check ESP is not blocked in the path between the 2 peers, take a packet capture to confirm.

[Elito Haylett]

see my response to @NHM. Thank you very much for you help...

[MHM Cisco World]

Share config let me check 

MHM

[Elito Haylett]

will do after I sanatize it.

[MHM Cisco World · Accepted solution]

No need 

Router B

#pkts encaps: 2525, #pkts encrypt: 2525, #pkts digest: 2525
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0

There is no decrypt that meaning there is issue in other side.

Router A

#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 47, #pkts decrypt: 47, #pkts verify: 47

Must likely Router-A run NAT and dont use no-NAT for traffic must pass through ipsec vpn.

MHM

[Elito Haylett · Accepted solution]

It was the NAT statement that changed. I compared it to one of my older configs with the statement and it was modified. I made the change back to when it was working and my tunnel came up. Thanks you guys very much... This is why I love this site..... Your expertise is the best.....

Current NAT Statement:

ip access-list extended 100
40 deny ip any host 172.168.100.162
50 deny ip any host 172.168.100.163
60 deny ip any host 172.168.100.164
70 deny ip any host 172.168.100.165
80 deny ip any host 172.168.100.166
90 deny ip any host 172.168.100.167
100 deny ip any host 172.168.100.168
110 deny ip any host 172.168.100.169
120 deny ip any host 172.168.100.170
140 permit ip 172.168.100.0 0.0.0.255 any
150 permit ip 172.168.110.0 0.0.0.255 any
160 permit ip 172.168.120.0 0.0.0.255 any
170 permit ip 172.168.138.0 0.0.0.255 any
180 permit ip 172.168.140.0 0.0.0.255 any
190 permit ip 172.168.150.0 0.0.0.255 any

This was the NAT statement when it was working:

ip access-list extended 100
10 remark NAT_ACL
10 remark IPSec_Rule
10 deny ip 172.168.100.0 0.0.0.255 172.168.101.0 0.0.0.255
20 deny ip 172.168.100.0 0.0.0.255 172.168.102.0 0.0.0.255
30 deny ip any host 172.168.100.161
40 deny ip any host 172.168.100.162
50 deny ip any host 172.168.100.163
60 deny ip any host 172.168.100.164
70 deny ip any host 172.168.100.165
80 deny ip any host 172.168.100.166
90 deny ip any host 172.168.100.167
100 deny ip any host 172.168.100.168
110 deny ip any host 172.168.100.169
120 deny ip any host 172.168.100.170
130 permit ip 10.1.10.0 0.0.0.3 any
140 permit ip 172.168.100.0 0.0.0.255 any
150 permit ip 172.168.110.0 0.0.0.255 any
160 permit ip 172.168.120.0 0.0.0.255 any
170 permit ip 172.168.138.0 0.0.0.255 any
180 permit ip 172.168.140.0 0.0.0.255 any
190 permit ip 172.168.150.0 0.0.0.255 any

Router-A#sho crypto ipsec sa

interface: GigabitEthernet0/0/2
Crypto map tag: Router-A, local addr 1.1.1.1

protected vrf: (none)
local ident (addr/mask/prot/port): (172.168.100.0/255.255.255.0/0/0)
remote ident (addr/mask/prot/port): (172.168.101.0/255.255.255.0/0/0)
current_peer 2.2.2.2 port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0

local crypto endpt.: 1.1.1.1, remote crypto endpt.: 2.2.2.2
plaintext mtu 1500, path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet0/0/2
current outbound spi: 0x0(0)
PFS (Y/N): N, DH group: none

inbound esp sas:

inbound ah sas:

inbound pcp sas:

outbound esp sas:

outbound ah sas:

outbound pcp sas:

protected vrf: (none)
local ident (addr/mask/prot/port): (172.168.100.0/255.255.255.0/0/0)
remote ident (addr/mask/prot/port): (172.168.102.0/255.255.255.0/0/0)
current_peer 2.2.2.2 port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 18815, #pkts encrypt: 18815, #pkts digest: 18815
#pkts decaps: 56794, #pkts decrypt: 56794, #pkts verify: 56794
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0

local crypto endpt.: 1.1.1.1, remote crypto endpt.: 2.2.2.2
plaintext mtu 1422, path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet0/0/2
current outbound spi: 0x26C71001(650579969)
PFS (Y/N): N, DH group: none

inbound esp sas:
spi: 0x63BA23EA(1673143274)
transform: esp-256-aes esp-sha512-hmac ,
in use settings ={Tunnel, }
conn id: 2065, flow_id: ESG:65, sibling_flags FFFFFFFF80000048, crypto map: Router-A, initiator : False
sa timing: remaining key lifetime (k/sec): (4571660/3504)
IV size: 16 bytes
replay detection support: Y
Status: ACTIVE(ACTIVE)

inbound ah sas:

inbound pcp sas:

outbound esp sas:
spi: 0x26C71001(650579969)
transform: esp-256-aes esp-sha512-hmac ,
in use settings ={Tunnel, }
conn id: 2066, flow_id: ESG:66, sibling_flags FFFFFFFF80000048, crypto map: Router-A, initiator : False
sa timing: remaining key lifetime (k/sec): (4606550/3504)
IV size: 16 bytes
replay detection support: Y
Status: ACTIVE(ACTIVE)

outbound ah sas:

outbound pcp sas:

protected vrf: (none)
local ident (addr/mask/prot/port): (172.168.100.0/255.255.255.0/0/0)
remote ident (addr/mask/prot/port): (172.168.103.0/255.255.255.0/0/0)
current_peer 2.2.2.2 port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0

local crypto endpt.: 1.1.1.1, remote crypto endpt.: 2.2.2.2
plaintext mtu 1500, path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet0/0/2
current outbound spi: 0x0(0)
PFS (Y/N): N, DH group: none

inbound esp sas:

inbound ah sas:

inbound pcp sas:

outbound esp sas:

outbound ah sas:

outbound pcp sas:

[MHM Cisco World]

You are so so welcome 

Have  a nice  weekend 

MHM

[Elito Haylett]

you too.......