Re: NHRP DMVPN query

SZ1 · ‎01-10-2009

Hi,

I have a DMVPN setup with 6509 as HUBS and 3845 as spoke. The 6509 speaks with 3845 using NAT-T

I have enabled DPD as well in the environment.

All works fine except that every random interval I get folloing observations which brings some of the DMVPN tunnels DOWN.

In 6509 when I use the sh ip nhrp detail command I get following output for the tunnel which is down

sh ip nhrp detail

10.253.78.250/32, Tunnel1 created 00:02:55, expire 00:00:09

Type: incomplete, Flags: negative

Cache hits: 7

In the 3845 I get following

Tunnel1, Type:Spoke, NHRP Peers:4,

# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb

----- --------------- --------------- ----- -------- -----

1 155.231.76.13 10.253.78.129 NHRP 00:50:38 S

1 155.231.76.14 10.253.78.130 UP 11w1d S

1 155.231.76.29 10.253.78.131 UP 00:53:17 S

1 155.231.76.30 10.253.78.132 UP 00:53:17 S

HOW DO I SOLVE THE PROBLEM....I am SURE it is something to do with NHRP config or IOS issue on 6509....

Giuseppe Larosa · ‎01-10-2009

Hello Sandeep,

NHRP shouldn't be the root cause of your problem you should investigate investigate on SA lifetimes.

NHRP shows incomplete because the IPSec communication is down and the peer failed to send periodic NHRP register messages to the NHRP server (the HUB 6509).

Look in the log of the C6509 to see possible related messages

Hope to help

Giuseppe

SZ1 · ‎01-12-2009

Hello Giuseppe

Thanks for your response. What you said makes perfect sense. Can you pls throw some light on how exactly should I investigate on SA lifetimes. I am sure my IPSec configuration parameters are fine and that is why the tunnels come up....so that is why I am asking how exactly should I investgate on SA lifetimes.

I have also observed that when the show dmvpn shows one of the tunnel in NHRP state and if I use the command "sh crypto session detail"....the 3845 thinks the tunnel is UP and shows the output in "sh crypto session detail" but the 6509 (HUB ) thinks the tunnel is not there and there is no ouput in "sh crypto session detail"..

The following output shows that Tunnel 0 is UP in 3845 but down in 6509

3845 output sh crypto session detail ::::::::::::::::

Interface: Tunnel0

Session status: UP-ACTIVE

Peer: 155.231.76.11 port 4500 fvrf: (none) ivrf: (none)

Phase1_id: SW01

Desc: (none)

IKE SA: local 172.20.7.3/4500 remote 155.231.76.11/4500 Active

Capabilities:N connid:9143 lifetime:06:15:41

IPSEC FLOW: permit 47 host 172.20.7.3 host 155.231.76.11

Active SAs: 2, origin: crypto map

Inbound: #pkts dec'ed 74661599 drop 4 life (KB/Sec) 4517249/80142

Outbound: #pkts enc'ed 600887 drop 1527 life (KB/Sec) 4517204/80142

Interface: Tunnel1

Session status: UP-ACTIVE

Peer: 155.231.76.13 port 4500 fvrf: (none) ivrf: (none)

Phase1_id: SW01

Desc: (none)

IKE SA: local 172.20.7.4/4500 remote 155.231.76.13/4500 Active

Capabilities:N connid:9139 lifetime:07:11:42

IPSEC FLOW: permit 47 host 172.20.7.4 host 155.231.76.13

Active SAs: 2, origin: crypto map

Inbound: #pkts dec'ed 1496742 drop 1 life (KB/Sec) 4548725/83503

Outbound: #pkts enc'ed 1906746 drop 6159 life (KB/Sec) 4548536/83503

6509 sh crypto session detail ::::::::::::::::::::::::

Interface: Tunnel1

Session status: UP-ACTIVE

Peer: 195.105.228.153/4500 fvrf: (none) ivrf: (none)

Phase1_id: RT02

Desc: (none)

IKE SA: local 155.231.76.13/4500 remote 195.105.228.153/4500 Active

Capabilities:DN connid:5 lifetime:07:15:18

IPSEC FLOW: permit 47 host 155.231.76.13 host 195.105.228.153

Active SAs: 2, origin: crypto map

Inbound: #pkts dec'ed 18369 drop 0 life (KB/Sec) 4592296/83720

Outbound: #pkts enc'ed 14529 drop 170 life (KB/Sec) 4592452/83720

chris.grammer · ‎01-12-2009

When I have had problems with DMVPN it has always been a reachability problem to the remote peer generally because I have to go through an Internet service provider NAT to a private IP or a firewall in between hub and spoke that you dont know about.

If there is a scenerio where there is a private IP, the remote router must initiate the ISAKMP session and then IPSEC session is initiated and data starts flowing. After IPSEC session is established, it is my understanding that the ISAKMP session is no longer needed unless the IPSEC tunnel rekeys. The first side to expire the re-key interval tries to send the re-key down the isakmp tunnel which doesnt exist and drops its side of the IPSEC session, but the opposite side has no idea the other side dropped IPSEC.

Since the ISAKMP session is UDP and very little traffic is sent after IPSEC is establish, firewalls will drop the ISAKMP session, but both ends dont know this.

If you look at:

show crypto ipsec sa

You will see the re-key interval. I feel like this re-key interval may need to be less than what a comman firewall will drop an inactive ISAKMP UDP session, which will force traffic on the ISAKMP session and reset the firewall timer.

I believe also there is an ISAKMP keepalive which can be turned on both ends, which I will research to see if I can provide any further information.

chris.grammer · ‎01-12-2009

Here is the keepalive information I was looking for. Read carefully that DPD only works if the router needs to send traffic. If it is idle it will not send a DPD. Therefore, to keep the session up, the keepalive will need to be forced at peroidic intervals.

How DPD and Cisco IOS Keepalive Features Work

DPD and Cisco IOS keepalives function on the basis of the timer. If the timer is set for 10 seconds, the router will send a "hello" message every 10 seconds (unless, of course, the router receives a "hello" message from the peer). The benefit of IOS keepalives and periodic DPD is earlier detection of dead peers. However, IOS keepalives and periodic DPD rely on periodic messages that have to be sent with considerable frequency. The result of sending frequent messages is that the communicating peers must encrypt and decrypt more packets.

DPD also has an on-demand approach. The contrasting on-demand approach is the default. With on-demand DPD, messages are sent on the basis of traffic patterns. For example, if a router has to send outbound traffic and the liveliness of the peer is questionable, the router sends a DPD message to query the status of the peer. If a router has no traffic to send, it never sends a DPD message. If a peer is dead, and the router never has any traffic to send to the peer, the router will not find out until the IKE or IPSec security association (SA) has to be rekeyed (the liveliness of the peer is unimportant if the router is not trying to communicate with the peer). On the other hand, if the router has traffic to send to the peer, and the peer does not respond, the router will initiate a DPD message to determine the state of the peer.

Using the IPSec Dead Peer Detection Periodic Message Option

With the IPSec Dead Peer Detection Periodic Message Option feature, you can configure your router so that DPD messages are "forced" at regular intervals. This forced approach results in earlier detection of dead peers. For example, if a router has no traffic to send, a DPD message is still sent at regular intervals, and if a peer is dead, the router does not have to wait until the IKE SA times out to find out.

If you want to configure the DPD periodic message option, you should use the crypto isakmp keepalive command with the periodic keyword. If you do not configure the periodic keyword, the router defaults to the on-demand approach.

http://www.cisco.com/en/US/docs/ios/12_3t/12_3t7/feature/guide/gtdpmo.html

SZ1 · ‎01-15-2009

Hi Chris...thanks for all your inputs.

I am using DPD periodic of 15 seconds. You have mentioned that the re-key interval should be observed to make sure the firewall is not timing out the UDP sessons...With DPD periodic of 15 seconds do you think it still needs to be observed?

The observations is that the spoke ( 3845 ) thinks that the tunnel is UP while 6509 thinks it is DOWN...So basically when i use Sh crypto ipsec sa there is some output in the 3845 but the relavent output towards the spoke 3845 is not there in 6509.