cisco router 2911 ios upgrade issue

rajmohan30 · ‎06-24-2017

Hello,

The current ios version of our router is c2900-universalk9-mz.SPA.152-2.T3.bin and would like to upgrade it to latest version c2900-universalk9-mz.SPA.154-3.M7.bin. After the upgradation we lost almost all spoke location which was connected via vpn tunnel (IPSec over Dynamic GRE).

Please let me know if there is any bug or something. Or less please let me know next compactible latest ios where we will not face such an issue. Once i downgrade the verison back to older one the connection was established with all spoke locations.

Before IOS upgrade:- (there was 11 EIGRP nei)

H Address Interface Hold Uptime SRTT RTO Q Seq
(sec) (ms) Cnt Num
9 170.205.200.186 Tu186 10 00:41:48 52 1398 0 18
6 170.205.200.99 Tu99 12 01:40:55 269 1614 0 37969
1 170.205.200.185 Tu24 14 06:54:42 40 1398 0 549708
8 170.205.200.19 Tu19 11 1d11h 79 1398 0 51398
7 170.205.200.98 Tu98 12 1d11h 101 1398 0 37697
4 170.205.200.63 Tu63 12 1d13h 350 2100 0 12857
10 10.111.109.248 Vl980 13 1w3d 1 100 0 2163348
0 10.111.109.254 Vl980 13 1w3d 1 100 0 91571
3 170.205.200.21 Tu21 14 1w6d 287 1722 0 11600467
2 170.205.200.22 Tu22 10 1w6d 290 1740 0 12269474
5 170.205.200.196 Tu196 14 5w1d 26 1398 0 382437

After IOS upgrade:- (there was only 4 EIGRP nei)

EIGRP-IPv4 Neighbors for AS(3300)
H Address Interface Hold Uptime SRTT RTO Q Seq
(sec) (ms) Cnt Num
3 10.111.109.254 Vl980 11 00:29:20 1 100 0 91583
2 10.111.109.248 Vl980 13 00:29:20 1 100 0 2163625
1 170.205.200.22 Tu22 11 00:29:37 297 1782 0 12272758
0 170.205.200.21 Tu21 12 00:29:38 298 1788 0 11603739

All spoke location which was connected via IPSEC over GRE was down after the ios upgrade. Please do let me know if you require further info for analysis.

Mark Malone · ‎06-25-2017

Hi

Did you see anything in logs related to the EIGRP/GRE when you upgraded

or try and tshoot the eigrp neighbours to see what was wrong and why they weren't coming up ànd why the GRE was down , did you diff the config incase something was removed in the upgrade between the software versions

Please let me know if there is any bug or something. Or less please let me know next compactible latest is where we will not face such an issue

This is really a question for TAC , you could check your release notes and see if there is something related to eigrp there maybe an open bug , release notes are found exactly where you downloaded the software on Cisco website

I would try the other only available recommended image too see if the issue is there too

c2900-universalk9-mz.SPA.155-3.M5.bin

rajmohan30 · ‎06-26-2017

Hello,

Thanks for the response. I guess the issue could be more related to IPSec. After the upgrade on Hub router, i have ran debug isakmp/ipsec commands and found that ISAKMP has estabhlished but no IPSec peers. Spoke will initiate the connection as the Hub was config'ed as dynamic.

rajmohan30 · ‎06-26-2017

Hello,

Spoke:-
crypto isakmp policy 1
encr 3des
hash md5
authentication pre-share
group 2
crypto isakmp key XYZ123 address 0.0.0.0
!
crypto ipsec transform-set 3des esp-3des esp-md5-hmac
mode tunnel
!
IPSEC profile default
Security association lifetime: 4608000 kilobytes/3600 seconds
Security association idletime: 60 seconds
Responder-Only (Y/N): N
PFS (Y/N): N
Mixed-mode : Disabled
Transform sets={
default: { esp-aes esp-sha-hmac } ,
}

!
crypto map MYVPN 1 ipsec-isakmp
set peer X.X.X.X
set transform-set 3des
match address HUB to Spoke ()
!

HUB:-
Crypto Map Template"ios" 1
ISAKMP Profile: lan
No matching address list set.
Security association lifetime: 4608000 kilobytes/3600 seconds
Responder-Only (Y/N): N
PFS (Y/N): N
Transform sets={
3DES: { esp-3des esp-md5-hmac } ,
}

IKEv1 PROFILE lan
Ref Count = 6
Identities matched are:
ip-address 0.0.0.0
Certificate maps matched are:
Identity presented is: ip-address
keyring(s): MYKEY
trustpoint(s): <all>

crypto keyring MYKEY vrf MYVRF
pre-shared-key address 0.0.0.0 0.0.0.0 key XYZ123

sh cryp ipsec profile
IPSEC profile default
Security association lifetime: 4608000 kilobytes/3600 seconds
Responder-Only (Y/N): N
PFS (Y/N): N
Transform sets={
default: { esp-aes esp-sha-hmac } ,
}

Could you please confirm if the following things can create the issue.

1) Mixed mode is missing at HUB.
2) Should i use vrfname at the end in match identity in IKEV1 profile (match identity address 0.0.0.0 vrfname).

I have seen the similar output in the debugs on HUB, no issue at spoke.

Jun 30 21:22:56.616 EDT: map_db_check_isakmp_profile profile did not match
Jun 30 21:22:56.616 EDT: map_db_check_isakmp_profile profile did not match
Jun 30 21:22:56.616 EDT: map_db_find_best did not find matching map
Jun 30 21:22:56.616 EDT: IPSEC(ipsec_process_proposal): proxy identities not supported
Jun 30 21:22:56.616 EDT: ISAKMP:(2001): IPSec policy invalidated proposal with error 32
Jun 30 21:22:56.616 EDT: ISAKMP:(2001):Checking IPSec proposal 2
... snip ...
Jun 30 21:22:56.616 EDT: ISAKMP:(2001): phase 2 SA policy not acceptable! (local <hub-ip-physical> remote <spoke-ip-physical>)

rajmohan30 · ‎06-27-2017

Hello ,

I found the issue and fixed it. Now it is working fine. But i have not understand how it works in the background, because it is working perfectly with older IOS version c2900-universalk9-mz.SPA.152-2.T3.bin.

The problem was that the vrf name was missing at the end of match identity statement in IKEv1 profile at Hub.

IKEv1 PROFILE lan
Ref Count = 6
Identities matched are:
ip-address 0.0.0.0
Certificate maps matched are:
Identity presented is: ip-address
keyring(s): MYKEY
trustpoint(s): <all>

After adding VRF name at the end, it starts working with newer IOS as well.

Identities matched are:
ip-address 0.0.0.0 MYVRF

Honestly, i did not know the reason why it started working after adding VRF. Could you pls explain me he following.

how it was working without VRF name in the older version ?

how it was not working without VRF in the newer IOS version ?

rajmohan30 · ‎06-30-2017

Hello,

Could someone please help me answer and make me to unserstand the reason ?