11-08-2020 11:30 AM - edited 11-09-2020 06:05 PM
Experts,
We either have a config issue, or there's a bug in IOS 15.4, maybe you can help.
Issue: DMVPN isn't passing traffic, EIGRP can't form neighbor relationship.
Possible cause: Does IPSEC SA look weird? Is spoke misconfigured or is there a bug when DMVPN goes over a dialer/PPPoE interface?
Note: The branch/spoke config was built and tested using regular ethernet internet (on cellular hotspot). It worked perfect. The branch internet is PPPoE over fiber, so the config was adjusted for the PPPoE once it arrived there. The DMVPN is not working correctly after adjusting for dialer interface (tunnel source dialer, changed NAT to dialer overload, etc.).
Local spoke subnets = 172.20.32.0/255.255.248.0
Remote (hub) subnets = 172.20.16.0/255.255.248.0
Here's the spoke's relevant pieces:
crypto ikev2 proposal IKEV2-PROPOSAL_CISCO
encryption aes-cbc-256
integrity sha512
group 24
!
crypto ikev2 profile IKEV2-PROFILE_DMVPN
match identity remote any
authentication remote pre-share key ####
authentication local pre-share key ####
!
crypto ipsec transform-set TRANSFORMSET-IPSEC_DMVPN esp-aes 256 esp-sha256-hmac
mode transport
!
crypto ipsec profile IPSEC-PROFILE_DMVPN
set transform-set TRANSFORMSET-IPSEC_DMVPN
set ikev2-profile IKEV2-PROFILE_DMVPN
!
interface Tunnel0
description DMVPN
bandwidth 100000
bandwidth qos-reference 100000
ip address 172.20.254.2 255.255.255.0
no ip redirects
ip mtu 1400
no ip next-hop-self eigrp 1
no ip split-horizon eigrp 1
ip pim sparse-mode
ip nat inside
ip nhrp authentication CNDMNTCS
ip nhrp map multicast dynamic !Added
ip nhrp network-id 1
ip nhrp holdtime 300
ip virtual-reassembly in
zone-member security LAN
ip summary-address eigrp 1 172.20.16.0 255.255.248.0
ip tcp adjust-mss 1360
delay 100
nhrp group DMVPN-QOS_100MBPS
nhrp map group DMVPN-QOS_100MBPS service-policy output POLICYMAP_DMVPN-100MBPS-LINE-RATE
nhrp map group DMVPN-QOS_60MBPS service-policy output POLICYMAP_DMVPN-60MBPS-LINE-RATE
qos pre-classify
tunnel source GigabitEthernet0/1
tunnel mode gre multipoint
tunnel key 1
tunnel protection ipsec profile IPSEC-PROFILE_DMVPN
end
!
interface GigabitEthernet0
ip address dhcp
no ip proxy-arp
ip virtual-reassembly in
duplex auto
speed auto
no cdp enable
!
interface GigabitEthernet0.7
description TELEKOM SUB-INTERFACE
encapsulation dot1Q 7
ip virtual-reassembly in
pppoe enable group global
pppoe-client dial-pool-number 1
no cdp enable
!
interface Vlan320
description Default gateway for LAN
ip address 172.20.32.254 255.255.255.0
no ip proxy-arp
ip nat inside
ip virtual-reassembly in
zone-member security LAN
!
interface Dialer1
description PPPoE configuration
mtu 1492
ip address negotiated
ip nat outside
ip virtual-reassembly in
zone-member security INTERNET
encapsulation ppp
ip tcp adjust-mss 1452
dialer pool 1
dialer-group 1
ppp authentication chap pap callin
ppp chap hostname #####@t-online.de
ppp chap password #####
ppp pap sent-username #####@t-online.de password #####
ppp ipcp route default
!
router eigrp 1
network 172.20.32.0 0.0.0.255
network 172.20.39.254 0.0.0.0
network 172.20.42.0 0.0.0.255
network 172.20.254.0 0.0.0.255
passive-interface default
no passive-interface Tunnel0
!
ip nat inside source list ACL_NAT interface Dialer1 overload
ip route 0.0.0.0 0.0.0.0 62.155.246.30 5
ip route 0.0.0.0 0.0.0.0 Dialer1 10
!
ip access-list extended ACL_NAT
deny ip 172.20.0.0 0.0.255.255 172.20.0.0 0.0.255.255
deny ip 172.20.42.0 0.0.0.255 any
permit ip 172.20.32.0 0.0.0.255 any
permit ip host 172.20.39.254 any
Here's the Hub Config:
rypto ikev2 proposal IKEV2-PROPOSAL_CISCO
encryption aes-cbc-256
integrity sha512
group 24
!
crypto ikev2 profile IKEV2-PROFILE_DYNAMIC
description ** Allows dynamic tunnels **
match identity remote any
identity local address 199.27.251.53
authentication remote pre-share key ###
authentication local pre-share key ###
!
crypto ipsec transform-set TRANSFORMSET-IPSEC_DMVPN esp-aes 256 esp-sha256-hmac
mode transport
!
crypto ipsec profile IPSEC-PROFILE_DMVPN
set transform-set TRANSFORMSET-IPSEC_DMVPN
set ikev2-profile IKEV2-PROFILE_DYNAMIC
!
interface Tunnel0
description DMVPN
bandwidth 100000
bandwidth qos-reference 100000
ip address 172.20.254.2 255.255.255.0
no ip redirects
ip mtu 1400
no ip next-hop-self eigrp 1
no ip split-horizon eigrp 1
ip pim sparse-mode
ip nhrp authentication CNDMNTCS
ip nhrp network-id 1
ip nhrp holdtime 300
zone-member security LAN
ip summary-address eigrp 1 172.20.16.0 255.255.248.0
ip tcp adjust-mss 1360
delay 100
nhrp group DMVPN-QOS_100MBPS
nhrp map group DMVPN-QOS_100MBPS service-policy output POLICYMAP_DMVPN-100MBPS-LINE-RATE
nhrp map group DMVPN-QOS_60MBPS service-policy output POLICYMAP_DMVPN-60MBPS-LINE-RATE
qos pre-classify
tunnel source GigabitEthernet0/1
tunnel mode gre multipoint
tunnel key 1
tunnel protection ipsec profile IPSEC-PROFILE_DMVPN
!
interface GigabitEthernet0/1
description ** METRONET ISP @ 100MBPS **
mac-address XXX.9b1d.9954
bandwidth 100000
ip address XXX.27.251.53 255.255.255.192
ip nat outside
ip virtual-reassembly in
zone-member security INTERNET
ip traffic-export apply TRAFFIC_EXPORT
duplex auto
speed auto
service-policy input POLICYMAP_MARK-WAN-INGRESS
service-policy output POLICYMAP_ISP-SUB-LINE-RATE
!
router eigrp 1
network 172.20.17.64 0.0.0.15
network 172.20.17.254 0.0.0.0
network 172.20.19.246 0.0.0.0
network 172.20.19.248 0.0.0.7
network 172.20.254.0 0.0.0.255
redistribute static
redistribute ospf 1 metric 100 1 255 255 1500
passive-interface default
no passive-interface Tunnel0
no passive-interface GigabitEthernet0/0
!
ip nat inside source list ACL_NAT interface GigabitEthernet0/1 overload
!
ip access-list extended ACL_NAT
deny ip 172.20.0.0 0.0.255.255 172.20.0.0 0.0.255.255
permit ip 172.20.30.0 0.0.0.255 any
permit ip host 172.20.16.3 any
permit ip host 172.20.16.2 any
permit ip host 172.20.16.11 any
permit ip host 172.20.19.253 any
permit ip host 172.20.19.32 any
permit ip host 172.20.19.38 any
permit ip host 172.20.19.39 any
permit ip host 172.20.19.249 any
permit ip 172.20.42.0 0.0.0.255 any
permit ip 172.20.16.128 0.0.0.127 any
permit ip host 172.20.19.254 any
permit ip 172.20.19.240 0.0.0.7 any
permit ip host 172.20.19.246 any
permit ip 172.20.31.0 0.0.0.255 any
permit ip host 172.20.17.254 any
Here's the output of the show crypto IPSEC SA's that shows incorrect protection:
interface: Tunnel0
Crypto map tag: Tunnel0-head-0, local addr XX.199.198.71
protected vrf: (none)
local ident (addr/mask/prot/port): (79.199.198.71/255.255.255.255/47/0) ! should be (0.0.0.0/0.0.0.0/0/0) right?
remote ident (addr/mask/prot/port): (XXX.27.251.53/255.255.255.255/47/0) ! should be (0.0.0.0/0.0.0.0/0/0) right?
current_peer XXX.27.251.53 port 500
PERMIT, flags={origin_is_acl,ipsec_sa_request_sent}
#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 1646, #recv errors 0
local crypto endpt.: XX.199.198.71, remote crypto endpt.: XX.27.251.53
plaintext mtu 1500, path mtu 1500, ip mtu 1500, ip mtu idb (none)
current outbound spi: 0x0(0)
PFS (Y/N): N, DH group: none
inbound esp sas:
inbound ah sas:
inbound pcp sas:
outbound esp sas:
outbound ah sas:
outbound pcp sas:
The other VTI VPNs show all 0's and let routing statements direct traffic as normal, I think DMVPN is supposed to be the same?
Lastly, show DMVPN detail:
gateway#show dmvpn detail
Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete
N - NATed, L - Local, X - No Socket
T1 - Route Installed, T2 - Nexthop-override
C - CTS Capable
# Ent --> Number of NHRP entries with same NBMA peer
NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting
UpDn Time --> Up or Down Time for a Tunnel
==========================================================================
Interface Tunnel0 is up/up, Addr. is 172.20.254.3, VRF ""
Tunnel Src./Dest. addr: XX.199.198.71/MGRE, Tunnel VRF ""
Protocol/Transport: "multi-GRE/IP", Protect "IPSEC-PROFILE_DMVPN"
Interface State Control: Disabled
nhrp event-publisher : Disabled
IPv4 NHS:
172.20.254.2 E priority = 0 cluster = 0
Type:Spoke, Total NBMA Peers (v4/v6): 2
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb Target Network
----- --------------- --------------- ----- -------- ----- -----------------
5 UNKNOWN 172.20.16.1 NHRP never IX 172.20.16.1/32
UNKNOWN 172.20.16.7 NHRP never IX 172.20.16.7/32
UNKNOWN 172.20.16.8 NHRP never IX 172.20.16.8/32
UNKNOWN 172.20.16.13 NHRP never IX 172.20.16.13/32
UNKNOWN 172.20.16.201 NHRP never IX 172.20.16.201/32
1 XXX.27.251.53 172.20.254.2 NHRP 01:26:48 S 172.20.254.2/32
Crypto Session Details:
--------------------------------------------------------------------------------
Interface: Tunnel0
Session: [0x119CF024]
Crypto Session Status: DOWN
fvrf: (none), IPSEC FLOW: permit 47 host XX.199.198.71 host XXX.27.251.53
Active SAs: 0, origin: crypto map
Inbound: #pkts dec'ed 0 drop 0 life (KB/Sec) 0/0
Outbound: #pkts enc'ed 0 drop 1653 life (KB/Sec) 0/0
Outbound SPI : 0x 0, transform :
Socket State: Closed
Notes: I can ping each end of the tunnel from the routers (172.20.254.2 and 172.20.254.3), but that's it, no other subnets. EIGRP is not sharing routes. The hub sees the branch as a neighbor, but the spoke has no neighbors.
11-08-2020 11:52 AM
Hi @corycandia
protected vrf: (none)
local ident (addr/mask/prot/port): (79.199.198.71/255.255.255.255/47/0) ! should be (0.0.0.0/0.0.0.0/0/0) right?
remote ident (addr/mask/prot/port): (XXX.27.251.53/255.255.255.255/47/0) ! should be (0.0.0.0/0.0.0.0/0/0) right?
These local/remote ident are correct, DMVPN uses GRE (protocol 47). I'd expect the ident to be 0.0.0.0/0.0.0.0 if you were using a VTI, which you aren't.
You have no inbound/outbound ESP SAs established, so obviously an issue there. Can you run some ikev2/ipsec debugs and upload the output please?
I assume you've established IKEv2 SA correctly, can you provide the output of "show crypto ikev2 sa detail"
As you are using ZBFW, can you provide the configuration for sefl and internet zones for review please?
If you took of the ZBFW temporarily for testing, does the VPN establish correctly?
11-08-2020 12:17 PM
11-08-2020 12:22 PM
11-08-2020 12:49 PM
The tunnel source on the Hub's tunnel interface is loopback 0, but the IP address of the hub's Gig0/1 is xxx.27.251.53 - which is the IP address defined as the NBMA on the spoke. Double check the tunnel source on the hub is correct.
11-08-2020 04:14 PM
I adjusted it back to gi 0/1 (outside).
I'll edit the previous posted config, and send new debugs.
11-08-2020 01:46 PM
Tunnel ip in hub and spoke must be in same subnet
ip summary not required for phase 2 dmvpn
11-08-2020 03:14 PM
11-08-2020 04:53 PM
Yes
11-08-2020 04:56 PM - edited 11-08-2020 04:56 PM
Here's updated show command output:
Looks like crypto pieces are working, but not EIGRP:
*Nov 9 00:50:57.440: EIGRP: Sending HELLO on Tu0 - paklen 20
*Nov 9 00:50:57.440: AS 1, Flags 0x0:(NULL), Seq 0/0 interfaceQ 0/0 iidbQ un/rely 0/0
*Nov 9 00:50:57.568: EIGRP: Received UPDATE on Tu0 - paklen 0 nbr 172.20.254.2
*Nov 9 00:50:57.568: AS 1, Flags 0x1:(INIT), Seq 26/0 interfaceQ 0/0
*Nov 9 00:50:57.568: EIGRP: Neighbor(172.20.254.2) not yet found
gateway#
*Nov 9 00:51:02.068: EIGRP: Received UPDATE on Tu0 - paklen 0 nbr 172.20.254.2
*Nov 9 00:51:02.068: AS 1, Flags 0x1:(INIT), Seq 26/0 interfaceQ 0/0
*Nov 9 00:51:02.068: EIGRP: Neighbor(172.20.254.2) not yet found
*Nov 9 00:51:02.256: EIGRP: Sending HELLO on Tu0 - paklen 20
*Nov 9 00:51:02.256: AS 1, Flags 0x0:(NULL), Seq 0/0 interfaceQ 0/0 iidbQ un/rely 0/0
gateway#
*Nov 9 00:51:06.711: EIGRP: Sending HELLO on Tu0 - paklen 20
*Nov 9 00:51:06.711: AS 1, Flags 0x0:(NULL), Seq 0/0 interfaceQ 0/0 iidbQ un/rely 0/0
*Nov 9 00:51:07.063: EIGRP: Received UPDATE on Tu0 - paklen 0 nbr 172.20.254.2
*Nov 9 00:51:07.063: AS 1, Flags 0x1:(INIT), Seq 26/0 interfaceQ 0/0
*Nov 9 00:51:07.067: EIGRP: Neighbor(172.20.254.2) not yet found
gateway#
I double checked the eigrp network statements, they look right. tunnel interface IPs are in the same network. They can actually ping each other (172.20.254.2 <--> 172.20.254.3), but no neighbor. I've been reseaching the 'not yet found', but haven't gotten anywhere yet.
11-08-2020 05:07 PM - edited 11-09-2020 08:47 AM
.....
11-08-2020 05:42 PM
Latest Update:
I had to add neighbor statements on both routers, then the adjacency was good.
I think this means there is a multicast problem, since EIGRP should have done this on it's own without manual/static programming. I removed the pim dense statements, and that didn't make any difference.
Reading the other posts about this issue didn't really shed light on why it's screwed up, just a way to cheat and force it to work.
Any ideas?
11-09-2020 02:28 AM
Do you add
ip nhrp map multicast dynamic
to hub?
11-09-2020 06:08 PM
I was missing this for some reason, went back through the config guide.
Frustrating part, I tested removing the forced neighbor statement on the hub and the tunnel dropped. I need to check what ROb said and see if firewall is blocking EIGRP, but I didn't see any dropped packets using term mon. I saw an article talking about 15.x ZBF breaking DMVPN EIGRP, I'll find that again and test.
11-10-2020 09:38 AM
Hi
do you check after add this command ?
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: