11-27-2022 12:17 PM - edited 11-27-2022 12:22 PM
I am trying out a solution using an ISR 829 with a cellular connection. This device will be a spoke in a DMVPN setup. It successfully connects with the hub and I am able to ping the Hubs tunnel address as well as ping between the sites behind the tunnel addresses.
The ISR829 is using a Verizon connection and sits behind a NAT address of 100.108.7.202/32
I have tried both transport mode and tunnel mode in my IPSEC configuration and both have worked with one showing the N attribute (Transport) and no N attribute with Tunneled.
(Transport)
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb
----- --------------- --------------- ----- -------- -----
1 xxx.xxx.xxx.xxx 172.16.124.5 UP 00:20:07 DN
DMVPN_POC_HUB#ping 172.16.124.5
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.124.5, timeout is 2 seconds:
!!!!!
(Tunneled)
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb
----- --------------- --------------- ----- -------- -----
1 100.108.7.202 172.16.124.5 UP 00:00:46 D
DMVPN_POC_HUB#ping 172.16.124.5
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.124.5, timeout is 2 seconds:
!!!!!
After a certain amount of time(as early as a couple of minutes) traffic stops flowing even though the tunnel still shows up and connected.
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb
----- --------------- --------------- ----- -------- -----
1 xxx.xxx.xxx.xxx. 172.16.124.5 UP 00:03:04 DN
DMVPN_POC_HUB#ping 172.16.124.5
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.124.5, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
#HUB
interface Tunnel0
description -> DMVPN HUB Tunnel
ip address 172.16.124.1 255.255.255.0
no ip redirects
ip nhrp authentication dmvpnpoc
ip nhrp network-id 10
tunnel source GigabitEthernet0/0/0
tunnel mode gre multipoint
tunnel key 1
tunnel protection ipsec profile IPSEC_PROFILE
end
#SPOKE
interface Tunnel1
description -> Spoke Tunnel
ip address 172.16.124.5 255.255.255.0
no ip redirects
ip mtu 1440
ip nhrp authentication dmvpnpoc
ip nhrp map multicast xxx.xxx.xxx.xxx
ip nhrp map 172.16.124.1 xxx.xxx.xxx
ip nhrp network-id 10
ip nhrp nhs 172.16.124.1
tunnel source Cellular0/0
tunnel mode gre multipoint
tunnel key 1
tunnel protection ipsec profile IPSEC_PROFILE
I have tried issue the Debug Crypto IPSEC and ISAKMP but nothing useful has come from those logs.
Any help on next steps in troubleshooting would be appreciated.
11-27-2022 02:20 PM
in Spoke do you check the cellular is it UP or not ?
in Spoke can you share show ip route ?
11-27-2022 02:24 PM
Cellular0/0 100.108.7.202 YES IPCP up up
Cellular1/0 unassigned YES NVRAM up up
Cellular0/1 unassigned YES NVRAM up up
Tunnel1 172.16.124.5 YES manual up up
11-27-2022 02:25 PM
I am able to ping the Hubs external IP address from the spoke and I am able to get to the internet as well
11-27-2022 02:21 PM
Here are some additional debugs before the tunnel stops passing traffic:
DMVPN_POC_HUB#
*Nov 27 23:56:08.368: NHRP: Receive Registration Request via Tunnel0 vrf global(0x0), packet size: 108
*Nov 27 23:56:08.369: NHRP: Tunnels gave us pak src addr: 100.108.7.202
*Nov 27 23:56:08.369: NHRP: Adding Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 27 23:56:08.369: NHRP: NHRP subblock already exists for Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 27 23:56:08.369: NHRP: Peer capability:0
*Nov 27 23:56:08.369: NHRP: Cache already has a subblock node attached for Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 27 23:56:08.369: NHRP: Adding Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 27 23:56:08.370: NHRP: NHRP subblock already exists for Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 27 23:56:08.370: NHRP: Peer capability:0
*Nov 27 23:56:08.370: NHRP: Cache already has a subblock node attached for Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 27 23:56:08.370: NHRP: nhrp_subblock_check_for_map() - Map Already Exists
*Nov 27 23:56:08.370: NHRP: Sending Registration Reply if_in Tunnel0 vrf global(0x0) dst 172.16.124.5 nbma 100.108.7.202 code: no error(0)
*Nov 27 23:56:08.370: NHRP: Attempting to send packet through interface Tunnel0 via DEST dst 172.16.124.5
*Nov 27 23:56:08.370: NHRP: Send Registration Reply via Tunnel0 vrf global(0x0), packet size: 128
*Nov 27 23:56:08.371: src: 172.16.124.1, dst: 172.16.124.5
*Nov 27 23:56:08.371: NHRP: Encapsulation succeeded. Sending NHRP Control Packet NBMA Address: 100.108.7.202
*Nov 27 23:56:08.371: NHRP: 156 bytes out Tunnel0
*Nov 27 23:56:08.828: NHRP: Receive Registration Request via Tunnel0 vrf global(0x0), packet size: 108
*Nov 27 23:56:08.829: NHRP: Tunnels gave us pak src addr: 100.108.7.202
*Nov 27 23:56:08.829: NHRP: Adding Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 27 23:56:08.829: NHRP: NHRP subblock already exists for Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 27 23:56:08.829: NHRP: Peer capability:0
*Nov 27 23:56:08.829: NHRP: Cache already has a subblock node attached for Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 27 23:56:08.829: NHRP: Adding Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 27 23:56:08.829: NHRP: NHRP subblock already exists for Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 27 23:56:08.830: NHRP: Peer capability:0
*Nov 27 23:56:08.830: NHRP: Cache already has a subblock node attached for Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 27 23:56:08.830: NHRP: nhrp_subblock_check_for_map() - Map Already Exists
*Nov 27 23:56:08.830: NHRP: Sending Registration Reply if_in Tunnel0 vrf global(0x0) dst 172.16.124.5 nbma 100.108.7.202 code: no error(0)
*Nov 27 23:56:08.830: NHRP: Attempting to send packet through interface Tunnel0 via DEST dst 172.16.124.5
*Nov 27 23:56:08.830: NHRP: Send Registration Reply via Tunnel0 vrf global(0x0), packet size: 128
*Nov 27 23:56:08.831: src: 172.16.124.1, dst: 172.16.124.5
*Nov 27 23:56:08.831: NHRP: Encapsulation succeeded. Sending NHRP Control Packet NBMA Address: 100.108.7.202
*Nov 27 23:56:08.831: NHRP: 156 bytes out Tunnel0
*Nov 27 23:56:10.807: NHRP: Receive Registration Request via Tunnel0 vrf global(0x0), packet size: 108
*Nov 27 23:56:10.807: NHRP: Tunnels gave us pak src addr: 100.108.7.202
*Nov 27 23:56:10.807: NHRP: Adding Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 27 23:56:10.807: NHRP: NHRP subblock already exists for Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 27 23:56:10.807: NHRP: Peer capability:0
*Nov 27 23:56:10.808: NHRP: Cache already has a subblock node attached for Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 27 23:56:10.808: NHRP: Adding Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 27 23:56:10.808: NHRP: NHRP subblock already exists for Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 27 23:56:10.808: NHRP: Peer capability:0
*Nov 27 23:56:10.808: NHRP: Cache already has a subblock node attached for Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 27 23:56:10.808: NHRP: nhrp_subblock_check_for_map() - Map Already Exists
*Nov 27 23:56:10.808: NHRP: Sending Registration Reply if_in Tunnel0 vrf global(0x0) dst 172.16.124.5 nbma 100.108.7.202 code: no error(0)
*Nov 27 23:56:10.809: NHRP: Attempting to send packet through interface Tunnel0 via DEST dst 172.16.124.5
*Nov 27 23:56:10.809: NHRP: Send Registration Reply via Tunnel0 vrf global(0x0), packet size: 128
*Nov 27 23:56:10.809: src: 172.16.124.1, dst: 172.16.124.5
*Nov 27 23:56:10.809: NHRP: Encapsulation succeeded. Sending NHRP Control Packet NBMA Address: 100.108.7.202
*Nov 27 23:56:10.809: NHRP: 156 bytes out Tunnel0
*Nov 27 23:56:14.589: NHRP: Receive Registration Request via Tunnel0 vrf global(0x0), packet size: 108
*Nov 27 23:56:14.589: NHRP: Tunnels gave us pak src addr: 100.108.7.202
*Nov 27 23:56:14.590: NHRP: Adding Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 27 23:56:14.590: NHRP: NHRP subblock already exists for Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 27 23:56:14.590: NHRP: Peer capability:0
*Nov 27 23:56:14.590: NHRP: Cache already has a subblock node attached for Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 27 23:56:14.590: NHRP: Adding Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 27 23:56:14.590: NHRP: NHRP subblock already exists for Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 27 23:56:14.590: NHRP: Peer capability:0
*Nov 27 23:56:14.591: NHRP: Cache already has a subblock node attached for Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
11-27-2022 02:28 PM - edited 11-27-2022 02:31 PM
https://blog.ipspace.net/2010/09/dmvpn-non-unique-nhrp-registrations.html
ip nhrp registration non-unique <<- this what you need only I think
after run command clear nhrp in hub to take effect
11-27-2022 02:48 PM
I tried that on my spoke but with no change. My external NAT address from the carrier and my cellular ip address do not appear to be changing.
*Nov 28 00:22:10.254: NHRP: Receive Registration Request via Tunnel0 vrf global(0x0), packet size: 108
*Nov 28 00:22:10.254: (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1
*Nov 28 00:22:10.254: shtl: 4(NSAP), sstl: 0(NSAP)
*Nov 28 00:22:10.254: pktsz: 108 extoff: 52
*Nov 28 00:22:10.254: (M) flags: "nat ", reqid: 546
*Nov 28 00:22:10.254: src NBMA: 100.108.7.202
*Nov 28 00:22:10.254: src protocol: 172.16.124.5, dst protocol: 172.16.124.1
*Nov 28 00:22:10.255: (C-1) code: no error(0)
*Nov 28 00:22:10.255: prefix: 32, mtu: 17912, hd_time: 600
*Nov 28 00:22:10.255: addr_len: 0(NSAP), subaddr_len: 0(NSAP), proto_len: 0, pref: 255
*Nov 28 00:22:10.255: NHRP: Tunnels gave us pak src addr: 100.108.7.202
*Nov 28 00:22:10.255: NHRP: Adding Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 28 00:22:10.255: NHRP: NHRP subblock already exists for Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 28 00:22:10.255: NHRP: Peer capability:0
*Nov 28 00:22:10.255: NHRP: Cache already has a subblock node attached for Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 28 00:22:10.256: NHRP: Adding Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 28 00:22:10.256: NHRP: NHRP subblock already exists for Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 28 00:22:10.256: NHRP: Peer capability:0
*Nov 28 00:22:10.256: NHRP: Cache already has a subblock node attached for Tunnel Endpoints (VPN: 172.16.124.5, NBMA: 100.108.7.202)
*Nov 28 00:22:10.256: NHRP: nhrp_subblock_check_for_map() - Map Already Exists
*Nov 28 00:22:10.256: NHRP: Sending Registration Reply if_in Tunnel0 vrf global(0x0) dst 172.16.124.5 nbma 100.108.7.202 code: no error(0)
*Nov 28 00:22:10.256: NHRP: Attempting to send packet through interface Tunnel0 via DEST dst 172.16.124.5
*Nov 28 00:22:10.257: NHRP: Send Registration Reply via Tunnel0 vrf global(0x0), packet size: 128
*Nov 28 00:22:10.257: src: 172.16.124.1, dst: 172.16.124.5
*Nov 28 00:22:10.257: (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1
*Nov 28 00:22:10.257: shtl: 4(NSAP), sstl: 0(NSAP)
*Nov 28 00:22:10.257: pktsz: 128 extoff: 52
*Nov 28 00:22:10.257: (M) flags: "nat ", reqid: 546
*Nov 28 00:22:10.257: src NBMA: 100.108.7.202
*Nov 28 00:22:10.257: src protocol: 172.16.124.5, dst protocol: 172.16.124.1
*Nov 28 00:22:10.257: (C-1) code: no error(0)
*Nov 28 00:22:10.257: prefix: 32, mtu: 17912, hd_time: 600
*Nov 28 00:22:10.258: addr_len: 0(NSAP), subaddr_len: 0(NSAP), proto_len: 0, pref: 255
*Nov 28 00:22:10.258: NHRP: Encapsulation succeeded. Sending NHRP Control Packet NBMA Address: 100.108.7.202
It tries to send it but nothing goes through. They only way I can get to send traffic again is to shut/ no shut the tunnel interface on the spoke
11-27-2022 03:08 PM
with non-unique and transport IPSec Mode, can you share
show crypto iskamp sa ?
11-27-2022 03:39 PM
I ran the clear ip NHRP on the hub, bounced both tunnel interfaces and it came but stopped sending traffic shortly after. I changed IPSEC mode to transport. Also noted that the "ip nhrp registration no-unique " did not show up in my tunnel interface config even though I added it.
DMVPN_POC_HUB#show crypto isakmp sa
IPv4 Crypto ISAKMP SA
dst src state conn-id status
xxx.xx.xxx.xxx 70.184.116.40 QM_IDLE 15191 ACTIVE
Spoke:
IR800#show crypto isakmp sa
IPv4 Crypto ISAKMP SA
dst src state conn-id status
xxx.xxx.xxx.xxx 100.108.7.202 QM_IDLE 1128 ACTIVE dmvpn-tun0
11-28-2022 01:28 AM
I will run lab, need some info.
are you use IKEv1 or IKEv2 ?
are Spoke behind NAT or Hub behind NAT or both ?
11-28-2022 06:43 AM
I run lab and same issue as you,
but I detect issue from where,
do
show dmvpn detail
there is only outbound in Spoke and in hub there is no Inbound and Outbound.
the issue is NAT router not NATing udp 4500 port
it can be bug if you use ISR as NAT router.
11-28-2022 01:58 PM
So my hub router has a public IP and does not NAT. It is an ISR 4331
My Spoke Router is an ISR829 with a cellular connection as the outbound connection. There is no other connection on this router providing outbound services. The ip that is given to the cellular interface comes from Verizon and is100.108.7.202. This is an IP address that is given and is is not a public reachable address. Verizon uses this to NST to a public address.
11-28-2022 02:17 PM
can I see
show dmvpn detail
show ip nhrp traffic
in spoke ?
11-29-2022 07:05 AM
I solve the issue I face in my lab, waiting your share of output
show dmvpn detail
show ip nhrp traffic
11-29-2022 03:37 PM - edited 11-29-2022 03:40 PM
IR800#ping 172.16.124.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.124.1, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
IR800#show dmvpn detail
Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete
N - NATed, L - Local, X - No Socket
T1 - Route Installed, T2 - Nexthop-override
C - CTS Capable, I2 - Temporary
# Ent --> Number of NHRP entries with same NBMA peer
NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting
UpDn Time --> Up or Down Time for a Tunnel
==========================================================================
Interface Tunnel1 is up/up, Addr. is 172.16.124.5, VRF ""
Tunnel Src./Dest. addr: 100.93.34.92/MGRE, Tunnel VRF ""
Protocol/Transport: "multi-GRE/IP", Protect "shiva"
Interface State Control: Disabled
nhrp event-publisher : Disabled
IPv4 Registration Timer: 65535 seconds
IPv4 NHS:
172.16.124.1 RE priority = 0 cluster = 0
Type:Spoke, Total NBMA Peers (v4/v6): 1
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb Target Network
----- --------------- --------------- ----- -------- ----- -----------------
1 xxx.xxx.xxx.xxx 172.16.124.1 UP 00:06:34 S 172.16.124.1/32
Crypto Session Details:
--------------------------------------------------------------------------------
Interface: Tunnel1
Session: [0x141A9B28]
Session ID: 0
IKEv1 SA: local 100.93.34.92/4500 remote 216.185.188.10/4500 Active
Capabilities:N connid:1207 lifetime:00:23:24
Crypto Session Status: UP-ACTIVE
fvrf: (none), Phase1_id: xxx.xxx.xxx.xxx
IPSEC FLOW: permit 47 host 100.93.34.92 host. xxx.xxx.xxx.xxx
Active SAs: 2, origin: crypto map
Inbound: #pkts dec'ed 18 drop 0 life (KB/Sec) 4169132/3204
Outbound: #pkts enc'ed 23 drop 0 life (KB/Sec) 4169132/3204
Outbound SPI : 0xCA3CE5F1, transform : esp-aes esp-sha-hmac
Socket State: Open
Pending DMVPN Sessions:
IR800#show ip nhrp traffic
Tunnel1: Max-send limit:10000Pkts/10Sec, Usage:0%
Sent: Total 197
1 Resolution Request 0 Resolution Reply 196 Registration Request
0 Registration Reply 0 Purge Request 0 Purge Reply
0 Error Indication 0 Traffic Indication 0 Redirect Suppress
Rcvd: Total 55
0 Resolution Request 1 Resolution Reply 0 Registration Request
54 Registration Reply 0 Purge Request 0 Purge Reply
0 Error Indication 0 Traffic Indication 0 Redirect Suppress
IR800#
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide