cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Announcements
Webcast SD-WAN
497
Views
0
Helpful
13
Replies
Highlighted
Beginner

Phase 1 ISAKMP failure_DMVPN

Site-to-site vpn tunnel drops off suddenly (connection is flapping).
I issued the 'debug crypto isakmp' command on this router and realized my connection was flapping.
The problematic router (C897VA) is a new and configuration on it is the same with other already working routers (spoke routers) on the network. This router is going to be a new spoke on the network. There is only one vpn tunnel established on it. Configuring DMVPN using GRE over IPSec. 

The next hop device Is ISP. The problematic router performs the NAT.

Public IP  is pingable during both situation:  when the VPN is down, and when the VPN is up

Software version is quite new.

 

crypto isakmp policy 1
encr aes
authentication pre-share
group 2
crypto isakmp key <removed> address 0.0.0.0 no-xauth
crypto isakmp keepalive 20 3

 

Here is a debug output:
ISAKMP-ERROR: (0):deleting SA reason "Death by retransmission P1" state (I) MM_NO_STATE
ISAKMP: (0):Deleting peer node by peer_reap for X.X.X.X: 2866AA4
ISAKMP: (0):deleting node 1802653311 error FALSE reason "IKE deleted"
ISAKMP: (0):deleting node -506512794 error FALSE reason "IKE deleted"
ISAKMP: (0):Input = IKE_MESG_INTERNAL, IKE_PHASE1_DEL
ISAKMP: (0):Old State = IKE_I_MM1 New State = IKE_DEST_SA

 

Typically, ISAKMP uses UDP as its transport protocol. ISAKMP traffic normally goes over UDP port 500, unless NAT-T is used in which case UDP port 4500 is used. When Phase 1 ISAKMP fails I noticed that the debug output shows ISAKMP traffic going over UDP port 500, not over UDP port 4500:

ISAKMP: (0):beginning Main Mode exchange
ISAKMP-PAK: (0):sending packet to X.X.X.X my_port 500 peer_port 500 (I) MM_NO_STATE

------------------------------------

When everything is ok with connection output of  the debug shows this line below:

 ISAKMP-PAK: (2001):sending packet to X.X.X.X my_port 4500 peer_port 4500 (I) MM_KEY_EXCH  (… and so on)

 

When it’s not ok then it starts from this line:

ISAKMP-PAK: (0):sending packet to X.X.X.X  my_port 500 peer_port 500 (I) MM_NO_STATE   (… and so on)

------------------------------------------

For more info open the attached files! All public IP-addresses given in the files are replaced with: X.X.X.X,  Y.Y.Y.Y,  Z.Z.Z.Z

13 REPLIES 13
VIP Advocate RJI VIP Advocate
VIP Advocate

Re: Phase 1 ISAKMP failure_DMVPN

Hi,

What interface is this spoke router using? The debugs indicate the Cellular0 modem several times,

 

11:33:49: %CISCO800-2-MODEM_UP: Cellular0 modem is now UP
11:33:53: Call failed....

11:33:57: %LINEPROTO-5-UPDOWN: Line protocol on Interface Cellular0, changed state to up

 

is that the WAN interface and is it flapping?....therefore causing the tunnel to drop

 

You have IP SLA configured:-

 

11:32:42: %TRACK-6-STATE: 2 ip sla 2 reachability Down -> Up

 

What is this tracking? Is it related to the Tunnel/WAN interface?

 

You will still see ISAKMP UDP/500 traffic until ISAKMP Phase 1 is complete, at which point the encrypted traffic will encapsulated inside UDP/4500 packets.

Beginner

Re: Phase 1 ISAKMP failure_DMVPN

Thanks for the reply!
Yes WAN interface is Cellular0:
interface Cellular0
ip address negotiated
ip mtu 1300
ip virtual-reassembly in
encapsulation slip
dialer in-band
dialer string lte
dialer-group 1

Yes it is related to the Tunnel/WAN interface:
ip sla 2
icmp-echo X.X.X.X source-interface Cellular0 (X.X.X.X- public IP-address)
threshold 500
timeout 10000
frequency 15
ip sla schedule 2 life forever start-time now

track 2 ip sla 2 reachability
delay down 180
!
---------------------------
Related to this: "You will still see ISAKMP UDP/500 traffic until ISAKMP Phase 1 is complete, at which point the encrypted traffic will encapsulated inside UDP/4500 packets." - YES I know, building of the ISAKMP Phase1 takes too long time and it's repeating these output:
ISAKMP-PAK: (0):sending packet to X.X.X.X my_port 500 peer_port 500 (I) MM_NO_STATE (… and so on)
When everything is ok with connection and the ISAKMP Phase1 is completed output of the debug shows this line below:
ISAKMP-PAK: (2001):sending packet to X.X.X.X my_port 4500 peer_port 4500 (I) MM_KEY_EXCH (… and so on)

Beginner

Re: Phase 1 ISAKMP failure_DMVPN

Hello
Is there any solution?
Hall of Fame Expert

Re: Phase 1 ISAKMP failure_DMVPN

Hello incognito,

according to the following link

https://learningnetwork.cisco.com/thread/36740

 

The UDP port 4500 is used with NAT-T also to encapsulate ESP packets not only ISAKMP packets.

 

From the log files:

Your WAN interface is cell0, and it has getting a private IP address, that will be NATTED later in mobile service provider backbone.

To be noted the Cell0 gets the SAME private IP address in both cases when the VPN is setup and when ISAKMP phase I on UDP port 500 is stucked.

From the log file about VPN up some lines may be missing at the begining as the first lines include

>> 12:58:12: ISAKMP-PAK: (2001):sending packet to X.X.X.X my_port 4500 peer_port 4500 (I) MM_KEY_EXCH

 

What happened before? It is important to understand the initial packet exchange when VPN setup is successful.

 

Hope to help

Giuseppe

 

Beginner

Re: Phase 1 ISAKMP failure_DMVPN

Thanks for the answer!

 

There are all debugs (when VPN setup is successful and not successful) are attached "when VPN is down_debug.txt" and  "when VPN is up_debug.txt"

 

When everything is ok (when VPN is up) with the connection output of the debug starts showing this line below:

ISAKMP-PAK: (2001):sending packet to X.X.X.X my_port 4500 peer_port 4500 (I) MM_KEY_EXCH (… and so on)

 

When it’s not ok (when VPN is down) then it starts from this line:

ISAKMP-PAK: (0):sending packet to X.X.X.X my_port 500 peer_port 500 (I) MM_NO_STATE (… and so on)
--------------------------------------------------------------
For more info open a new attached file: "when VPN is down for a few seconds.txt" this file contains a debug when VPN was down for a few seconds.
---------------------------------------------------------
All public IP-addresses given in the files are replaced with: X.X.X.X, Y.Y.Y.Y, Z.Z.Z.Z

Hall of Fame Master

Re: Phase 1 ISAKMP failure_DMVPN

In the normal process of negotiating for ISAKMP it begins by using port 500. If the negotiation proceeds successfully it detects that the peer is associated with NAT and begins to use port 4500. Your statements are consistent with that. In the file where the VPN is up it uses 4500 and in the files where VPN is down it uses only 500. What the file shows is that in the file where VPN is down it is sending attempts to negotiate to the peer but is not receiving any messages. We need to figure out why the peer is not responding in that case.

 

HTH

 

Rick

Beginner

Re: Phase 1 ISAKMP failure_DMVPN

Thank you Richard for the answer

 

I attached new word files and images with an explanations. Please take a look

The problem is related to IPSec (the problem is related to encryption).

In the network there is configured DMVPN Phase 1 (spoke-hub-spoke)

The problem is related with IPSec IKE Phase 1 (ISAKMP)  or in other words - ISAKMP negotiation fail. 

The "show crypto isakmp sa" command shows the ISAKMP SA to be in MM_NO_STATE, meaning the main-mode (IKE Phase 1) failed.

In the beginning the problem (IPSec IKE Phase 1 (ISAKMP)) was with "spoke-to-hub" connection. Now it's functioning, but this problem hasn't gone totatlly and 

it is still working unstable (VPN tunnel flapping).

In addition to this we ran into "spoke-to-spoke" connection problem and the result of "Debug crypto isakmp" output is the same with "spoke-to-hub"s 

debug output. 

To be precise, some "spoke-to-spoke" connections are working fine while some of them aren't working.

More information is in attached files.  Please pay attention to the attached files!

Hall of Fame Master

Re: Phase 1 ISAKMP failure_DMVPN

Your message says there are new word files. But I do not see any files.

 

HTH

 

Rick

Beginner
Hall of Fame Master

Re: Phase 1 ISAKMP failure_DMVPN

Thank you for the new files. The output of the new file for when not working is similar to the earlier output. It shows very clearly that your router is attempting to initiate isakmp negotiation. It is transmitting, retransmitting, incrementing the error count, over and over and then deletes the peer, and starts over again. It is clear in this output that you are not receiving any isakmp packet from the peer. So the question now is why is the remote peer not sending any isakmp packets?

 

HTH

 

Rick

Beginner

Re: Phase 1 ISAKMP failure_DMVPN

hello

"So the question now is why is the remote peer not sending any isakmp packets?" - Yes

Why some "spoke-to-spoke" connections are working fine while some of them aren't working. We have one hub and 17 spokes. Everything fine with 7 of them, but with 10 of them we have a problem...

Hall of Fame Master

Re: Phase 1 ISAKMP failure_DMVPN

Please help me understand the symptoms a bit better. There is 1 hub and 17 spokes. Am I understanding correctly that every spoke is able to successfully establish the encrypted tunnel to the hub? Am I understanding correctly that for the 7 spokes that work, they can establish encrypted tunnels among the 7? Can these 7 that work establish encrypted tunnels to any of the 10? For the 10 that do not work are they able to establish encrypted tunnels to any other hub?

 

Other than having unique IP addressing, are there things in the config of the 10 problem spokes that is different from the 7 successful spokes? Perhaps debug from one of the problem spokes might provide some insight.

 

HTH

 

Rick

Beginner

Re: Phase 1 ISAKMP failure_DMVPN

Hello, Giuseppe

Could you please take a look to newly attached files?

 

I attached new word files and images with an explanations.

The problem is related to IPSec (the problem is related to encryption).

In the network there is configured DMVPN Phase 1 (spoke-hub-spoke)

The problem is related with IPSec IKE Phase 1 (ISAKMP)  or in other words - ISAKMP negotiation fail. 

The "show crypto isakmp sa" command shows the ISAKMP SA to be in MM_NO_STATE, meaning the main-mode

(IKE Phase 1) failed.

In the beginning the problem (IPSec IKE Phase 1 (ISAKMP)) was with "spoke-to-hub" connection. Now it's functioning, but this problem hasn't gone totatlly and it is still working unstable (VPN tunnel flapping).

In addition to this we ran into "spoke-to-spoke" connection problem and the result of "Debug crypto isakmp" output is the same with "spoke-to-hub"s debug output. 

To be precise, some "spoke-to-spoke" connections are working fine while some of them aren't working.

More information is in attached files.  Please pay attention to the attached files!