DMVPN Tunnel went to NHRP state After Spoke Router Reboot

sathish.062 · ‎09-12-2018

DMVPN Tunnel went to NHRP state After Spoke Router Reboot, Once tunnel interface configuration removed and deployed again issue got resolve. this issue happens when spoke router reboot. Kindly suggest on this. Please find below tunnel configuration of Hub and Spoke end.

Spoke End:

interface Tunnel1
ip address 172.16.254.20 255.255.255.0
no ip redirects
ip mtu 1400
ip pim nbma-mode
ip pim sparse-dense-mode
no ip next-hop-self eigrp 50
ip nhrp map 172.16.254.1 X.X.X.X
ip nhrp map multicast X.X.X.X
ip nhrp network-id 101
ip nhrp holdtime 300
ip nhrp nhs 172.16.254.1
ip tcp adjust-mss 1360
delay 12
tunnel source GigabitEthernet0/0/0
tunnel mode gre multipoint
tunnel protection ipsec profile ODMVPN2

Hub End:

interface Tunnel1
bandwidth 200000
ip address 172.16.254.1 255.255.255.0
no ip redirects
ip mtu 1400
ip wccp redirect exclude in
no ip next-hop-self eigrp 50
no ip split-horizon eigrp 50
ip pim nbma-mode
ip pim sparse-dense-mode
ip nhrp map multicast dynamic
ip nhrp network-id 101
ip nhrp holdtime 300
ip tcp adjust-mss 1360
delay 120
tunnel source GigabitEthernet0/3.305
tunnel mode gre multipoint
tunnel protection ipsec profile ODMVPN2
end

Debug Logs Before and After Tunnel Interface 1.

Debug Before resetting tunnel 1 interface:

Sep 11 10:59:57.043: NHRP: No SNMP node found to add requestID
Sep 11 10:59:57.043: NHRP: Attempting to send packet through interface Tunnel1 via DEST dst 172.16.254.1
Sep 11 10:59:57.043: NHRP: Send Registration Request via Tunnel1 vrf global(0x0), packet size: 92
Sep 11 10:59:57.044: NHRP-DETAIL: Unable to get dst from pak sb
Sep 11 10:59:57.044: NHRP: Encapsulation succeeded. Sending NHRP Control Packet NBMA Address: X.X.X.X
Sep 11 10:59:57.044: NHRP: 116 bytes out Tunnel1
Sep 11 10:59:57.044: NHRP: Resetting retransmit due to hold-timer for 172.16.254.1

Debug After resetting tunnel 1 interface:
.Sep 11 12:43:56.610: NHRP: Cache already has a subblock node attached for Tunnel Endpoints (VPN: 172.16.254.1, NBMA: X.X.X.X)
.Sep 11 12:43:56.610: NHRP: No SNMP node found to add requestID
.Sep 11 12:43:56.610: NHRP: Attempting to send packet through interface Tunnel1 via DEST dst 172.16.254.1
.Sep 11 12:43:56.610: NHRP: Send Registration Request via Tunnel1 vrf global(0x0), packet size: 92
.Sep 11 12:43:56.611: NHRP-DETAIL: Unable to get dst from pak sb
.Sep 11 12:43:56.611: NHRP-CACHE: Setting 'used' flag on cache entry with nhop: 172.16.254.1
.Sep 11 12:43:56.611: NHRP: Encapsulation succeeded. Sending NHRP Control Packet NBMA Address: X.X.X.X
.Sep 11 12:43:56.611: NHRP: 116 bytes out Tunnel1
.Sep 11 12:43:56.611: NHRP: Resetting retransmit due to hold-timer for 172.16.254.1
.Sep 11 12:43:57.489: NHRP: Setting retrans delay to 2 for nhs dst 172.16.254.1
.Sep 11 12:43:57.489: NHRP: Attempting to send packet through interface Tunnel1 via DEST dst 172.16.254.1
.Sep 11 12:43:57.489: NHRP: Send Registration Request via Tunnel1 vrf global(0x0), packet size: 92
.Sep 11 12:43:57.489: src: 172.16.254.20, dst: 172.16.254.1
.Sep 11 12:43:57.489: NHRP-DETAIL: Unable to get dst from pak sb
.Sep 11 12:43:57.489: NHRP: Encapsulation succeeded. Sending NHRP Control Packet NBMA Address: X.X.X.X
.Sep 11 12:43:57.489: NHRP: 116 bytes out Tunnel1
.Sep 11 12:43:57.490: NHRP-RATE: Sending initial Registration Request for 172.16.254.1, reqid 211
.Sep 11 12:43:58.602: NHRP: Setting retrans delay to 2 for nhs dst 172.16.254.1
.Sep 11 12:43:58.602: IPSEC-IFC MGRE/Tu1(75.99.252.194/X.X.X.X): connection lookup returned 7F36C58B4818
.Sep 11 12:43:58.602: NHRP: Attempting to send packet through interface Tunnel1 via DEST dst 172.16.254.1
.Sep 11 12:43:58.602: NHRP: Send Registration Request via Tunnel1 vrf global(0x0), packet size: 92
.Sep 11 12:43:58.602: src: 172.16.254.20, dst: 172.16.254.1
.Sep 11 12:43:58.603: NHRP-DETAIL: Unable to get dst from pak sb
.Sep 11 12:43:58.603: NHRP: Encapsulation succeeded. Sending NHRP Control Packet NBMA Address: X.X.X.X
.Sep 11 12:43:58.603: NHRP: 116 bytes out Tunnel1
.Sep 11 12:43:58.603: NHRP-RATE: Retransmitting Registration Request for 172.16.254.1, reqid 211, (retrans ivl 2 sec)
.Sep 11 12:43:58.615: NHRP: Receive Registration Reply via Tunnel1 vrf global(0x0), packet size: 112
.Sep 11 12:43:58.615: NHRP-DETAIL: netid_in = 0, t
o_us = 1
.Sep 11 12:43:58.615: NHRP: NHS 172.16.254.1 Tunnel1 vrf 0 Cluster 0 Priority 0 Transitioned to 'RE' from 'E'
.Sep 11 12:43:58.615: NHRP: NHS-UP: 172.16.254.1

Georg Pauwen · ‎09-12-2018

Hello,

post the full configs...

sathish.062 · ‎09-13-2018

Hi Georg,

Sorry for responding late. Please find attached spoke full config.

Georg Pauwen · ‎09-14-2018

Hello,

the config looks good actually. You might want to try the changes below marked in bold:

crypto ipsec transform-set DMVPN2-TS esp-3des
mode tunnel

!

crypto ipsec profile ODMVPN2
set transform-set DMVPN2-TS
set pfs group2

sathish.062 · ‎09-14-2018

Hi Georg,

Thanks for your response. I tired what you have suggest but unfortunately its not working. I rebooted spoke router and checked DMVPN status is in NHRP state, after removed and config again tunnel 1 issue got fix temporarily. If i reboot spoke router again i will go to NHRP state.

Is this related any BUG? any idea.

Before Tunnel 1 config reset:

Interface: Tunnel1, IPv4 NHRP Details
Type:Spoke, NHRP Peers:1,

# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb
----- --------------- --------------- ----- -------- -----
1 69.46.229.182 172.16.254.1 NHRP 00:00:10 S

After tunnel config reset:

Interface: Tunnel1, IPv4 NHRP Details
Type:Spoke, NHRP Peers:1,

# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb
----- --------------- --------------- ----- -------- -----
1 69.46.229.182 172.16.254.1 UP 00:00:04 S

Georg Pauwen · ‎09-14-2018

Hello,

you could set the nhrp registration timeout to a low value as below. NHRP state means the spoke does't register with the hub, so forcing the spoke to register might help:

ip nhrp registration timeout 5

sathish.062 · ‎09-14-2018

Hi Georg,

Thank for your response. I tried NHRP registration cmd (ip nhrp registration timeout 5) under tunnel interface but unfortunately DMVPN went to NHRP state after spoke reboot. any other suggestion Georg.

Georg Pauwen · ‎09-14-2018

Hello,

try and configure:

ip nhrp registration no-unique

on your tunnel interface.

sathish.062 · ‎09-14-2018

Hi Georg,

Thanks for your support. I tried this command on tunnel interface and reboot the spoke device but unfortunately DMVPN went to NHRP state again, after resetting tunnel interface DMVPN tunnel back to normal.

Interface: Tunnel1, IPv4 NHRP Details
Type:Spoke, NHRP Peers:1,

# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb
----- --------------- --------------- ----- -------- -----
1 69.46.229.182 172.16.254.1 NHRP 00:00:13 S

Georg Pauwen · ‎09-14-2018

Hello,

which of the crypto policies is the one matching your hub ? Can you post the config of the hub as well ?

What happens if you remove 'delay 120' from the tunnel interface ?

sathish.062 · ‎09-17-2018

Hi Georg,

Sorry for responding late. Please find crypto policy configs of Hub & Spoke. no affect on removing delay from tunnel interface.

HUB Tunnel Config:

Current configuration : 510 bytes
!
interface Tunnel1
description "DMVPN HUB 01 - VERIZON DMVPN-1"
bandwidth 200000
ip address 172.16.254.1 255.255.255.0
no ip redirects
ip mtu 1400
ip wccp redirect exclude in
no ip next-hop-self eigrp 50
no ip split-horizon eigrp 50
ip pim nbma-mode
ip pim sparse-dense-mode
ip nhrp map multicast dynamic
ip nhrp network-id 101
ip nhrp holdtime 300
ip tcp adjust-mss 1360
delay 120
tunnel source GigabitEthernet0/3.305
tunnel mode gre multipoint
tunnel protection ipsec profile ODMVPN2
end

Current configuration : 290 bytes
!
interface GigabitEthernet0/3.305
description LightTower Circuit
bandwidth 200000
encapsulation dot1Q 305
ip address 199.102.115.X 255.255.255.240 secondary
ip address 69.46.229.X 255.255.255.252
ip nat outside
ip virtual-reassembly in
service-policy output PM_SHAPE_OUT_LT
end

Spoke Config:

interface Tunnel0
description "DMVPN SPOKE 20 - DMVPN-0"
ip address 10.254.254.20 255.255.255.0
no ip redirects
ip mtu 1400
no ip next-hop-self eigrp 50
ip nhrp map multicast 108.58.212.26
ip nhrp map 10.254.254.1 108.58.212.26
ip nhrp network-id 100
ip nhrp holdtime 300
ip nhrp nhs 10.254.254.1
ip tcp adjust-mss 1360
tunnel source GigabitEthernet0/0/0
tunnel mode gre multipoint
tunnel key 0
tunnel protection ipsec profile ODMVPN
!

interface GigabitEthernet0/0/0
description external to internet (CableVision)
ip address 75.99.252.X 255.255.255.248
ip nat outside
negotiation auto

Crypto Policy on Spoke End:

Global IKE policy
Protection suite of priority 1
encryption algorithm: Three key triple DES
hash algorithm: Message Digest 5
authentication method: Pre-Shared Key
Diffie-Hellman group: #2 (1024 bit)
lifetime: 86400 seconds, no volume limit
Protection suite of priority 2
encryption algorithm: Three key triple DES
hash algorithm: Secure Hash Standard
authentication method: Pre-Shared Key
Diffie-Hellman group: #2 (1024 bit)
lifetime: 86400 seconds, no volume limit
Protection suite of priority 5
encryption algorithm: Three key triple DES
hash algorithm: Message Digest 5
authentication method: Pre-Shared Key
Diffie-Hellman group: #2 (1024 bit)
lifetime: 86400 seconds, no volume limit
Protection suite of priority 10
encryption algorithm: AES - Advanced Encryption Standard (256 bit keys).
hash algorithm: Secure Hash Standard
authentication method: Pre-Shared Key
Diffie-Hellman group: #2 (1024 bit)
lifetime: 28800 seconds, no volume limit

Crypto Policy on Hub end.

Global IKE policy
Protection suite of priority 1
encryption algorithm: Three key triple DES
hash algorithm: Message Digest 5
authentication method: Pre-Shared Key
Diffie-Hellman group: #2 (1024 bit)
lifetime: 86400 seconds, no volume limit
Protection suite of priority 2
encryption algorithm: Three key triple DES
hash algorithm: Secure Hash Standard
authentication method: Pre-Shared Key
Diffie-Hellman group: #2 (1024 bit)
lifetime: 86400 seconds, no volume limit
Protection suite of priority 5
encryption algorithm: DES - Data Encryption Standard (56 bit keys).
hash algorithm: Message Digest 5
authentication method: Pre-Shared Key
Diffie-Hellman group: #1 (768 bit)
lifetime: 86400 seconds, no volume limit
Protection suite of priority 10
encryption algorithm: AES - Advanced Encryption Standard (256 bit keys).
hash algorithm: Secure Hash Standard
authentication method: Pre-Shared Key
Diffie-Hellman group: #2 (1024 bit)
lifetime: 86400 seconds, no volume limit

Georg Pauwen · ‎09-17-2018

When the tunnel is in NHRP state, can you ping the NHS address(es) ?

sathish.062 · ‎09-17-2018

Hi Georg,

Thank for the response. I'm unable to ping the NHS address while tunnel is in NHRP state.

Georg Pauwen · ‎09-17-2018

Hello,

if you cannot ping the NHS address, that is the problem. Try and add static routes for your NHS addresses:

ip route 172.16.254.1 255.255.255.255 79.99.252.y
ip route 10.254.254.1 255.255.255.255 79.99.252.y
ip route 108.58.212.26 255.255.255.255 79.99.252.y

The IP address needs to be the next hop of GigabitEthernet0/0/0. If you don't know the IP address of the next hop, use:

ip route 172.16.254.1 255.255.255.255 GigabitEthernet0/0/0
ip route 10.254.254.1 255.255.255.255 GigabitEthernet0/0/0
ip route 108.58.212.26 255.255.255.255 GigabitEthernet0/0/0

sathish.062 · ‎09-17-2018

Hi Georg,

Thanks for the suggestion. let me try and update you.