IPSEC VPN Username:Unknown IKEv2 Received a IKE_INIT_SA request

[firewallguyadmin]

I have an HA cluster of ASA5525 Firewalls that have a bi-directional tunnel built to a Cisco ASA5505.  I also have a bidirectional tunnel to Auzre and a bidirectional tunnel build to a DR/BR site.  These configurations have been working for 2 years with no issues.  They re-establish when needed no issues up until 3 weeks ago.

I suffered a power out with my HA Cluster and when the power came back on by tunnel to the DR/BR and Azure sites all came back up , but my IPSEC tunnel for the 5505 keeps giving my the error:  Username:Unknown IKEv2 Received a IKE_INIT_SA request.

In ASDM when you go into Monitor you see this.  The top one is a session that works and the bottom one is the session in question.  It increments the transmit and then fails.

I have backups of my configurations and went through both the HA cluster and the 5505 configurations with no changes in either one. I for the life of me can't figure this out.  I even rebuilt the connection Profile on the HA cluster for the 5505 and updated the keys on both sides but no luck.  I'm now to the step of just going to the remote site.  Write erasing the config on the 5505 and starting over from scratch.  I don't want to do that.

I would appreciate a direction to go here to start trying to understand this problem or if anyone else has ran into this before?

[DanielP211]

Hello,

I ran into the same issue, but in my case the issue was misconfiguration of IKEv1 and IKEv2. Which I doubth here is the case... Could you do a debug crypto of the peer?

[MHM Cisco World]

I have idea about what happened here 

there is some feature centralized and other distribution in FW cluster, 
the feature like VPN I think it centralized what meaning that the one of FW cluster will form IPsec to other Peer, and other FW in cluster dot have role in this process.
so here the VPN router  before cluster down is different than the VPN router after cluster up and hence the VPN failed 
you need only to re-initiate the VPN in far end device and check the result. 

[firewallguyadmin]

I thought the same early in this process.  My thinking was that both the primary and the fail over where powered back on at the exact same time.  I failed over to my standby and powered off the primary for about 4 hours.  Cleared the ARP on the DMZ switch to make sure nothing was still hanging out there.  I'm going to go ahead and post a debug for the peer but it will take me a bit since I need to drive to the site. 

 I will fail the cluster back over tonight to the primary and reload the standby just to make sure everything is clean and truly the way it was.

[firewallguyadmin]

Failed the cluster over to the original configuration of primary and secondary but no luck.  I will get a debug tomorrow.

[MHM Cisco World]

Failed but did you clear crypto isakmp and ipsec sa ?

This need to clear to notify the far peer about change in your vpn

[firewallguyadmin]

So if I'm reading this correctly.  It looks like the problem is not with the HA cluster but with the 5509.  So it looks like HA(.92) is going out to 5509(.241).  So I'm going to the site on Monday to work on this and get the crypto debug.

[firewallguyadmin]

I ran an debug crypto ipsec on the HA cluster and this is what I see

Rule Lookup for local 10.200.0.0 to remote 10.210.0.0
Crypto map: peer xxx.xxx.xxx.241 doesn't match map entry
Crypto map: peer xxx.xxx.xxx.241 doesn't match map entry
Crypto map: peer xxx.xxx.xxx.241 doesn't match map entry
Crypto map: peer xxx.xxx.xxx.241 doesn't match map entry
Crypto map outside_att_1g_map seq 5: no proposals
Crypto map outside_att_1g_map seq 6: no proposals
Crypto map outside_att_1g_map seq 7: no proposals
PROXY MATCH on crypto map outside_att_1g_map seq 8

I then tried to catch a sh crypto ipsec sa peer xxx.xxx.xxx.241 on the HA cluster and here is what I see:

navstlasa1# sh crypto ipsec sa peer xxx.xxx.xxx.241
peer address: xxx.xxx.xxx.241
Crypto map tag: outside_att_1g_map, seq num: 8, local addr: xxx.xxx.xxx.92

access-list remote-colo_att_1g_cryptomap extended permit ip 10.200.0.0 255.255.0.0 10.210.0.0 255.255.0.0 log debugging
local ident (addr/mask/prot/port): (10.200.0.0/255.255.0.0/0/0)
remote ident (addr/mask/prot/port): (10.210.0.0/255.255.0.0/0/0)
current_peer: xxx.xxx.xxx.241

#pkts encaps: 1, #pkts encrypt: 1, #pkts digest: 1
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 1, #pkts comp failed: 0, #pkts decomp failed: 0
#pre-frag successes: 0, #pre-frag failures: 0, #fragments created: 0
#PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0
#TFC rcvd: 0, #TFC sent: 0
#Valid ICMP Errors rcvd: 0, #Invalid ICMP Errors rcvd: 0
#send errors: 0, #recv errors: 0

local crypto endpt.: xxx.xxx.xxx.92/500, remote crypto endpt.: xxx.xxx.xxx.241/500
path mtu 1500, ipsec overhead 74(44), media mtu 1500
PMTU time remaining (sec): 0, DF policy: copy-df
ICMP error validation: disabled, TFC packets: disabled
current outbound spi: EFB41AC9
current inbound spi : BA1FA279

*** Tunnel rekeyed or deleted ***

I then tried sh vpn-sessiondb detail l2l filter ipaddress xxx.xxx.xxx.241

asaha# sh vpn-sessiondb detail l2l filter ipaddress xxx.xxx.xxx.241

Session Type: LAN-to-LAN Detailed

INFO: Session duration is invalid as time has been reset
Connection : xxx.xxx.xxx.241
Index : 50997 IP Addr : xxx.xxx.xxx.241
Protocol : IKEv2
Encryption : IKEv2: (1)AES256 Hashing : IKEv2: (1)SHA1
Bytes Tx : 0 Bytes Rx : 0
Login Time : 14:44:22 CDT Mon May 1 2023
Duration : 0h:00m:00s

IKEv2 Tunnels: 1

IKEv2:
Tunnel ID : 50997.1
UDP Src Port : 500 UDP Dst Port : 500
Rem Auth Mode: preSharedKeys
Loc Auth Mode: preSharedKeys
Encryption : AES256 Hashing : SHA1
Rekey Int (T): 86400 Seconds Rekey Left(T): 86400 Seconds
PRF : SHA1 D/H Group : 5
Filter Name :

I have not gone to the remote site yet but I do have backups of the configs and here is how they match up:

HA has a crypto map that states:
crypto map outside_att_1g_map 8 match address remote-colo_carrier_1g_cryptomap (this object is: 10.210.0.0_16)
crypto map outside_att_1g_map 8 set peer xxxx.xxxx.xxxx.241
crypto map outside_att_1g_map 8 set ikev2 ipsec-proposal AES256
crypto map outside_att_1g_map 8 set security-association lifetime seconds 604800
crypto map outside_att_1g_map 8 set security-association lifetime kilobytes unlimited

Remote site has a crypto map:

crypto map outside_map 1 match address outside_cryptomap (this object is: 10.210.0.0_16)
crypto map outside_map 1 set peer 45.19.226.92
crypto map outside_map 1 set ikev2 ipsec-proposal AES256
crypto map outside_map 1 set security-association lifetime seconds 604800
crypto map outside_map 1 set security-association lifetime kilobytes unlimited

If the remote sides network is 10.210.0.0/16 and the HA network is 10.200.0.0/16 thenI'm thinking the remote site crypto map config is wrong and that's why the HA cluster can't establish a connection?

[MHM Cisco World]

Yes I see the ACL for VPN is same in both peer 10.210.0.0/16 check this point 
but 
VPN-dbsession Rx and Tx is 0 and I see encrypt =1 ? can you make double check this point 

[firewallguyadmin · Accepted solution]

This has been resolved and the problem is something that I only figured out as I sat on the floor of an IDF with cold air blowing down my back for 2 and 1/2 hours.  At this point I just wanted to get this thing setup so I can remotely configure it and I will go back to the office and do it from there.  I kept getting the error when connecting via any connect "Failed to get AAA handle".  

I found this post: https://community.cisco.com/t5/vpn/failed-to-get-aaa-hendle/td-p/2907780 

This lead me to a syslog entry that used TCP and I realized it didn't have the "logging permit-hostdown"

I remove the TCP syslog line and the site came up with no issues.  

[MHM Cisco World]

Huge thanks for update me, 

So syslog to tcp server make Asa refused add new connection. 

I read about this point and you are correct, if fw loss connection to tcp syslog server the fw stop add new connection. 

Thanks again 

MHM