06-05-2019 02:15 AM - edited 02-21-2020 09:40 PM
Hi All,
First post, but avid viewer be kind..!
Got a strange problem which I've never encountered before with a site to site VPN between a head-end 2921 and the satellite 1941.
Site lost power and it appears the time was out when restored - having reset the time and re-enrolled the satellite site and renewing the expired certificate I'm seeing %CRYPTO-6-IKMP_NO_ID_CERT_ADDR_MATCH messages every 90 seconds or so on the satellite.
So sh cryp session brief on the headend shows the satelite as UA - the uptime lasts for about 1:30 before dropping. Satelite side shows no pkts.
#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0
Head end :
PERMIT, flags={origin_is_acl,}
#pkts encaps: 10, #pkts encrypt: 10, #pkts digest: 10
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0
Logging shows a similar picture:
n 5 09:03:49.039: IPSEC(create_sa): sa created,
(sa) sa_dest= [HEADEND], sa_proto= 50,
sa_spi= 0xC20BC1B6(3255550390),
sa_trans= esp-aes esp-sha-hmac , sa_conn_id= 3695
sa_lifetime(k/sec)= (4608000/3600)
Jun 5 09:03:49.039: IPSEC(create_sa): sa created,
(sa) sa_dest= [BRANCH], sa_proto= 50,
sa_spi= 0x829B8D42(2191232322),
sa_trans= esp-aes esp-sha-hmac , sa_conn_id= 3696
sa_lifetime(k/sec)= (4608000/3600)
Jun 5 09:03:49.039: IPSEC: Expand action denied, notify RP
Jun 5 09:03:49.043: IPSEC(rte_mgr): VPN Route Event Install new outbound sa: Create IPV4 route from ACL for [BRANCH]
Jun 5 09:03:49.043: %LINEPROTO-5-UPDOWN: Line protocol on Interface Virtual-Access3, changed state to up
Jun 5 09:05:18.807: ISAKMP:(0):: peer matches headend-profile profile
Jun 5 09:05:18.815: %CRYPTO-6-IKMP_NO_ID_CERT_ADDR_MATCH: ID of [BRANCH] (type 1) and certificate addr with
Jun 5 09:05:18.815: %CRYPTO-6-IKMP_NO_ID_CERT_ADDR_MATCH: ID of [BRANCH] (type 1) and certificate addr with
Jun 5 09:05:18.823: %CRYPTO-5-IKEV2_SESSION_STATUS: Crypto tunnel v2 is DOWN. Peer [BRANCH]:500 Id: [BRANCH]
Jun 5 09:05:18.823: IPSEC(key_engine): got a queue event with 1 KMI message(s)
Jun 5 09:05:18.823: IPSEC(key_engine_delete_sas): rec'd delete notify from ISAKMP
Jun 5 09:05:18.823: IPSEC(key_engine_delete_sas): delete SA with spi 0xC20BC1B6 proto 50 for [HEADEND]
Jun 5 09:05:18.823: IPSEC(update_current_outbound_sa): updated peer [BRANCH] current outbound sa to SPI 829B8D42
Jun 5 09:05:18.823: IPSEC(delete_sa): deleting SA,
(sa) sa_dest= [HEADEND], sa_proto= 50,
sa_spi= 0xC20BC1B6(3255550390),
sa_trans= esp-aes esp-sha-hmac , sa_conn_id= 3695
sa_lifetime(k/sec)= (4608000/3600),
(identity) local= [HEADEND]:0, remote= [BRANCH]:0,
local_proxy= 0.0.0.0/0.0.0.0/256/0,
remote_proxy= 0.0.0.0/0.0.0.0/256/0
Jun 5 09:05:18.823: IPSEC(update_current_outbound_sa): updated peer [BRANCH] current outbound sa to SPI 829B8D42
Jun 5 09:05:18.823: IPSEC(delete_sa): deleting SA,
(sa) sa_dest= [BRANCH], sa_proto= 50,
sa_spi= 0x829B8D42(2191232322),
sa_trans= esp-aes esp-sha-hmac , sa_conn_id= 3696
sa_lifetime(k/sec)= (4608000/3600),
(identity) local= [HEADEND]:0, remote= [BRANCH]:0,
local_proxy= 0.0.0.0/0.0.0.0/256/0,
remote_proxy= 0.0.0.0/0.0.0.0/256/0
Jun 5 09:05:18.823: IPSEC(key_engine): got a queue event with 1 KMI message(s)
Jun 5 09:05:18.823: IPSEC(key_engine_delete_sas): rec'd delete notify from ISAKMP
Jun 5 09:05:18.823: IPSEC(key_engine): got a queue event with 1 KMI message(s)
Jun 5 09:05:18.823: %LINEPROTO-5-UPDOWN: Line protocol on Interface Virtual-Access3, changed state to down
Jun 5 09:05:18.827: IPSEC(key_engine): got a queue event with 1 KMI message(s)
Jun 5 09:05:18.835: %LINEPROTO-5-UPDOWN: Line protocol on Interface Virtual-Access1, changed state to down
Any ideas/insights would be appreciated.
06-05-2019 03:28 AM
06-05-2019 05:06 AM
Hi RJI,
Thanks for the quick reply.
I didn't change the FQDN - same hostname and yes, same CA (its other spokes are fine) and the problem router does have the same root certificate. I've just tried creating a new trustpoint and re-enrolling to no avail. The only thing I can see in the debugging is there is no inbound activity on the spoke whereas a working one does (obviously)..
Quick note on the configs - for my own sanity I've done a line by line comparison of two spoke configs (one working and this one) and they are identical less hostnames/ local ips. So it does look to be an issue relating to the pki.
cfgs/output attached
06-05-2019 05:59 AM
Hi,
On your Hub you have 2 x IKEv2 Profiles, 1 with Certificate authentication (headend-profile) and the other with PSK (azure-profile).
I assume you should must be using the headend-profile if you are using certificates? This matches a remote identity of 80.x.x.206 - is that IP address of the spoke that isn't working?
You mentioned other spokes, with this configuration only 1 IP address 80.x.x.206 could match that profile. You'd be better using a Certificate Map and matching the remote identity on that.
HTH
06-05-2019 06:30 AM
Hi RJI,
Correct I'm using the headend-profile on the spoke ignore the other profile, 80.x.x.206 is the spoke that isn't working correct.
Apologies I did truncate the other spokes from the headend-profile for readability - these would also be in there in the live scenario. I'll take a look at setting up a cert map.
However this cfg has worked before though, unsure as to why it would stop with a loss of power and forced re-enrol - I did note that after the restart of the spoke the stored nvram cert was very old - the CA cert had expired so hence I did a fresh re-enroll/authenticate. (No cfg changes on the hub since either).
The looping 90 seconds of uptime on the hub is all I can seem to get atm!
sh cryp session brief
Status: A- Active, U - Up, D - Down, I - Idle, S - Standby, N - Negotiating
K - No IKE
ivrf = (none)
Peer I/F Username Group/Phase1_id Uptime Status
80.x.x.206 Vi3 80.x.x.212 00:00:01 UA
06-05-2019 06:35 AM
06-05-2019 08:06 AM
06-13-2019 01:35 AM
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide