Re: DMVPN Config Issue or Bug? - Page 2

corycandia · ‎11-08-2020

Experts,

We either have a config issue, or there's a bug in IOS 15.4, maybe you can help.

Issue: DMVPN isn't passing traffic, EIGRP can't form neighbor relationship.

Possible cause: Does IPSEC SA look weird? Is spoke misconfigured or is there a bug when DMVPN goes over a dialer/PPPoE interface?

Note: The branch/spoke config was built and tested using regular ethernet internet (on cellular hotspot). It worked perfect. The branch internet is PPPoE over fiber, so the config was adjusted for the PPPoE once it arrived there. The DMVPN is not working correctly after adjusting for dialer interface (tunnel source dialer, changed NAT to dialer overload, etc.).

Local spoke subnets = 172.20.32.0/255.255.248.0

Remote (hub) subnets = 172.20.16.0/255.255.248.0

Here's the spoke's relevant pieces:

crypto ikev2 proposal IKEV2-PROPOSAL_CISCO
encryption aes-cbc-256
integrity sha512
group 24
!
crypto ikev2 profile IKEV2-PROFILE_DMVPN
match identity remote any
authentication remote pre-share key ####
authentication local pre-share key ####
!
crypto ipsec transform-set TRANSFORMSET-IPSEC_DMVPN esp-aes 256 esp-sha256-hmac
mode transport
!
crypto ipsec profile IPSEC-PROFILE_DMVPN
set transform-set TRANSFORMSET-IPSEC_DMVPN
set ikev2-profile IKEV2-PROFILE_DMVPN
!
interface Tunnel0
description DMVPN
bandwidth 100000
bandwidth qos-reference 100000
ip address 172.20.254.2 255.255.255.0
no ip redirects
ip mtu 1400
no ip next-hop-self eigrp 1
no ip split-horizon eigrp 1
ip pim sparse-mode
ip nat inside
ip nhrp authentication CNDMNTCS
ip nhrp map multicast dynamic !Added
ip nhrp network-id 1
ip nhrp holdtime 300
ip virtual-reassembly in
zone-member security LAN
ip summary-address eigrp 1 172.20.16.0 255.255.248.0
ip tcp adjust-mss 1360
delay 100
nhrp group DMVPN-QOS_100MBPS
nhrp map group DMVPN-QOS_100MBPS service-policy output POLICYMAP_DMVPN-100MBPS-LINE-RATE
nhrp map group DMVPN-QOS_60MBPS service-policy output POLICYMAP_DMVPN-60MBPS-LINE-RATE
qos pre-classify
tunnel source GigabitEthernet0/1
tunnel mode gre multipoint
tunnel key 1
tunnel protection ipsec profile IPSEC-PROFILE_DMVPN
end

!
interface GigabitEthernet0
ip address dhcp
no ip proxy-arp
ip virtual-reassembly in
duplex auto
speed auto
no cdp enable
!
interface GigabitEthernet0.7
description TELEKOM SUB-INTERFACE
encapsulation dot1Q 7
ip virtual-reassembly in
pppoe enable group global
pppoe-client dial-pool-number 1
no cdp enable
!
interface Vlan320
description Default gateway for LAN
ip address 172.20.32.254 255.255.255.0
no ip proxy-arp
ip nat inside
ip virtual-reassembly in
zone-member security LAN
!
interface Dialer1
description PPPoE configuration
mtu 1492
ip address negotiated
ip nat outside
ip virtual-reassembly in
zone-member security INTERNET
encapsulation ppp
ip tcp adjust-mss 1452
dialer pool 1
dialer-group 1
ppp authentication chap pap callin
ppp chap hostname #####@t-online.de
ppp chap password #####
ppp pap sent-username #####@t-online.de password #####
ppp ipcp route default
!
router eigrp 1
network 172.20.32.0 0.0.0.255
network 172.20.39.254 0.0.0.0
network 172.20.42.0 0.0.0.255
network 172.20.254.0 0.0.0.255
passive-interface default
no passive-interface Tunnel0
!
ip nat inside source list ACL_NAT interface Dialer1 overload
ip route 0.0.0.0 0.0.0.0 62.155.246.30 5
ip route 0.0.0.0 0.0.0.0 Dialer1 10
!
ip access-list extended ACL_NAT
deny ip 172.20.0.0 0.0.255.255 172.20.0.0 0.0.255.255
deny ip 172.20.42.0 0.0.0.255 any
permit ip 172.20.32.0 0.0.0.255 any
permit ip host 172.20.39.254 any

Here's the Hub Config:

rypto ikev2 proposal IKEV2-PROPOSAL_CISCO 
encryption aes-cbc-256
integrity sha512
group 24
!
crypto ikev2 profile IKEV2-PROFILE_DYNAMIC
description ** Allows dynamic tunnels **
match identity remote any
identity local address 199.27.251.53
authentication remote pre-share key ###
authentication local pre-share key ###
!
crypto ipsec transform-set TRANSFORMSET-IPSEC_DMVPN esp-aes 256 esp-sha256-hmac 
mode transport
!
crypto ipsec profile IPSEC-PROFILE_DMVPN
set transform-set TRANSFORMSET-IPSEC_DMVPN 
set ikev2-profile IKEV2-PROFILE_DYNAMIC
!
interface Tunnel0
description DMVPN
bandwidth 100000
bandwidth qos-reference 100000
ip address 172.20.254.2 255.255.255.0
no ip redirects
ip mtu 1400
no ip next-hop-self eigrp 1
no ip split-horizon eigrp 1
ip pim sparse-mode
ip nhrp authentication CNDMNTCS
ip nhrp network-id 1
ip nhrp holdtime 300
zone-member security LAN
ip summary-address eigrp 1 172.20.16.0 255.255.248.0
ip tcp adjust-mss 1360
delay 100
nhrp group DMVPN-QOS_100MBPS
nhrp map group DMVPN-QOS_100MBPS service-policy output POLICYMAP_DMVPN-100MBPS-LINE-RATE
nhrp map group DMVPN-QOS_60MBPS service-policy output POLICYMAP_DMVPN-60MBPS-LINE-RATE
qos pre-classify
tunnel source GigabitEthernet0/1
tunnel mode gre multipoint
tunnel key 1
tunnel protection ipsec profile IPSEC-PROFILE_DMVPN
!
interface GigabitEthernet0/1
description ** METRONET ISP @ 100MBPS **
mac-address XXX.9b1d.9954
bandwidth 100000
ip address XXX.27.251.53 255.255.255.192
ip nat outside
ip virtual-reassembly in
zone-member security INTERNET
ip traffic-export apply TRAFFIC_EXPORT
duplex auto
speed auto
service-policy input POLICYMAP_MARK-WAN-INGRESS
service-policy output POLICYMAP_ISP-SUB-LINE-RATE
!
router eigrp 1
network 172.20.17.64 0.0.0.15
network 172.20.17.254 0.0.0.0
network 172.20.19.246 0.0.0.0
network 172.20.19.248 0.0.0.7
network 172.20.254.0 0.0.0.255
redistribute static
redistribute ospf 1 metric 100 1 255 255 1500
passive-interface default
no passive-interface Tunnel0
no passive-interface GigabitEthernet0/0
!
ip nat inside source list ACL_NAT interface GigabitEthernet0/1 overload
!
ip access-list extended ACL_NAT
deny ip 172.20.0.0 0.0.255.255 172.20.0.0 0.0.255.255
permit ip 172.20.30.0 0.0.0.255 any
permit ip host 172.20.16.3 any
permit ip host 172.20.16.2 any
permit ip host 172.20.16.11 any
permit ip host 172.20.19.253 any
permit ip host 172.20.19.32 any
permit ip host 172.20.19.38 any
permit ip host 172.20.19.39 any
permit ip host 172.20.19.249 any
permit ip 172.20.42.0 0.0.0.255 any
permit ip 172.20.16.128 0.0.0.127 any
permit ip host 172.20.19.254 any
permit ip 172.20.19.240 0.0.0.7 any
permit ip host 172.20.19.246 any
permit ip 172.20.31.0 0.0.0.255 any
permit ip host 172.20.17.254 any

Here's the output of the show crypto IPSEC SA's that shows incorrect protection:

interface: Tunnel0
Crypto map tag: Tunnel0-head-0, local addr XX.199.198.71

protected vrf: (none)
local ident (addr/mask/prot/port): (79.199.198.71/255.255.255.255/47/0)  ! should be (0.0.0.0/0.0.0.0/0/0) right?
remote ident (addr/mask/prot/port): (XXX.27.251.53/255.255.255.255/47/0) ! should be (0.0.0.0/0.0.0.0/0/0) right?
current_peer XXX.27.251.53 port 500
PERMIT, flags={origin_is_acl,ipsec_sa_request_sent}
#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 1646, #recv errors 0

local crypto endpt.: XX.199.198.71, remote crypto endpt.: XX.27.251.53
plaintext mtu 1500, path mtu 1500, ip mtu 1500, ip mtu idb (none)
current outbound spi: 0x0(0)
PFS (Y/N): N, DH group: none

inbound esp sas:

inbound ah sas:

inbound pcp sas:

outbound esp sas:

outbound ah sas:

outbound pcp sas:

The other VTI VPNs show all 0's and let routing statements direct traffic as normal, I think DMVPN is supposed to be the same?

Lastly, show DMVPN detail:

gateway#show dmvpn detail
Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete
N - NATed, L - Local, X - No Socket
T1 - Route Installed, T2 - Nexthop-override
C - CTS Capable
# Ent --> Number of NHRP entries with same NBMA peer
NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting
UpDn Time --> Up or Down Time for a Tunnel
==========================================================================

Interface Tunnel0 is up/up, Addr. is 172.20.254.3, VRF ""
Tunnel Src./Dest. addr: XX.199.198.71/MGRE, Tunnel VRF ""
Protocol/Transport: "multi-GRE/IP", Protect "IPSEC-PROFILE_DMVPN"
Interface State Control: Disabled
nhrp event-publisher : Disabled

IPv4 NHS:
172.20.254.2 E priority = 0 cluster = 0
Type:Spoke, Total NBMA Peers (v4/v6): 2

# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb Target Network
----- --------------- --------------- ----- -------- ----- -----------------
5 UNKNOWN 172.20.16.1 NHRP never IX 172.20.16.1/32
UNKNOWN 172.20.16.7 NHRP never IX 172.20.16.7/32
UNKNOWN 172.20.16.8 NHRP never IX 172.20.16.8/32
UNKNOWN 172.20.16.13 NHRP never IX 172.20.16.13/32
UNKNOWN 172.20.16.201 NHRP never IX 172.20.16.201/32
1 XXX.27.251.53 172.20.254.2 NHRP 01:26:48 S 172.20.254.2/32


Crypto Session Details:
--------------------------------------------------------------------------------

Interface: Tunnel0
Session: [0x119CF024]
Crypto Session Status: DOWN
fvrf: (none), IPSEC FLOW: permit 47 host XX.199.198.71 host XXX.27.251.53
Active SAs: 0, origin: crypto map
Inbound: #pkts dec'ed 0 drop 0 life (KB/Sec) 0/0
Outbound: #pkts enc'ed 0 drop 1653 life (KB/Sec) 0/0
Outbound SPI : 0x 0, transform :
Socket State: Closed

Notes: I can ping each end of the tunnel from the routers (172.20.254.2 and 172.20.254.3), but that's it, no other subnets. EIGRP is not sharing routes. The hub sees the branch as a neighbor, but the spoke has no neighbors.

corycandia · ‎11-12-2020

I did, it did not fully provide full functionality. The EIGRP adjacency still didn't come up.

I believe my configuration issue was related to missing these two commands:

ip pim dr-priority 10
ip pim nbma-mode

I am using PIM, so I think the second one was the key. The spokes have higher IP addresses, so I had to prevent them from taking DR from hub also.

This was my final config, but still revising as I learn better ways:

interface Tunnel0
description DMVPN
bandwidth 100000
bandwidth qos-reference 100000
ip address 172.20.254.2 255.255.255.0
no ip redirects
ip mtu 1400
no ip next-hop-self eigrp 1
no ip split-horizon eigrp 1
ip pim dr-priority 10
ip pim nbma-mode
ip pim sparse-mode
ip nat inside
ip nhrp authentication CNDMNTCS
ip nhrp network-id 1
ip nhrp holdtime 300
ip nhrp redirect
ip virtual-reassembly in
zone-member security LAN
ip summary-address eigrp 1 172.20.16.0 255.255.248.0
ip tcp adjust-mss 1360
delay 100
nhrp group DMVPN-QOS_100MBPS
nhrp map group DMVPN-QOS_100MBPS service-policy output POLICYMAP_DMVPN-100MBPS-LINE-RATE
nhrp map group DMVPN-QOS_60MBPS service-policy output POLICYMAP_DMVPN-60MBPS-LINE-RATE
qos pre-classify
tunnel source GigabitEthernet0/1
tunnel mode gre multipoint
tunnel key 1
tunnel path-mtu-discovery
tunnel protection ipsec profile IPSEC-PROFILE_DMVPN
end

Rob Ingram · ‎11-12-2020

@corycandia

Your configuration above doesn't appear to have the command ip nhrp map multicast dynamic as @MHM Cisco World previously mentioned. Did you try this?

Reference

https://www.cisco.com/c/en/us/support/docs/security-vpn/ipsec-negotiation-ike-protocols/41940-dmvpn.html

Supporting Dynamic Routing Protocols

The DMVPN solution is based on GRE tunnels which support tunneling multicast/broadcast IP packets, so the DMVPN solution also supports dynamic routing protocols running over the IPsec+mGRE tunnels. Previously, NHRP required you to explicitly configure the broadcast/multicast mapping for the tunnel destination IP addresses to support GRE tunneling of Multicast and Broadcast IP packets. For example, at the hub you would need the ip nhrp map multicast <spoke-n-addr> configuration line for each spoke. With the DMVPN solution, the spoke addresses are not known in advance, so this configuration is not possible. Instead, NHRP can be configured to automatically add each spoke to the multicast destination list on the hub with the ip nhrp map multicast dynamic command. With this command, when the spoke routers register their unicast NHRP mapping with the NHRP server (hub), NHRP will also create a broadcast/multicast mapping for this spoke. This eliminates the need for the spoke addresses to be known in advance.

HTH

corycandia · ‎11-14-2020

Here's the weird thing: I have already put that in. I think there's a condition where 'ip nhrp map multicast dynamic' is maybe replaced with something else in my configuration. Maybe 'ip pim nbma-mode'? When I add it to the interface, it doesn't show up.

Oddly, it's working now. I'd need to test to confirm, but I am thinking using PIM spare-mode on the tunnel interface cause EIGRP not to do normal multicast, so in that case, I needed 'ip pim nbma-mode', but if not doing PIM, maybe 'IP nhrp multicast dynamic' is the requirement?

Either way, It's functional now with the addition of 'ip pim nbma-mode'

corycandia · ‎11-15-2020

Last follow for anyone interested:

On my hub, ip nhrp map multicast dynamic seems to be default because I keep putting it in, but it doesn't show up?

Also, after switching the tunnel interfaces from 'IP PIM SPARSE-MODE' to 'IP PIM SPARSE-DENSE-MODE', 'ip pim nbma-mode' seems to not be needed.

Rob Ingram · ‎11-08-2020

@corycandia

Check the ZBFW is not stopping EIGRP form an adjacency.