What is probably causing below EIGRP neighbor adjacency flapping?

[MicJameson1]

Hello.

Below are logs from CSR_1000v.

What is probably causing below EIGRP neighbor adjacency flapping?

May you please suggest the best commands to troubleshoot below?

Thank you!

(obfuscated)...

*Jul 10 16:48:46.961: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor 192.168.30.44 (Tunnel77) is down: retry limit exceeded
*Jul 10 16:48:49.554: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor 192.168.30.44 (Tunnel77) is up: new adjacency
*Jul 10 16:50:09.064: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor 192.168.30.44 (Tunnel77) is down: retry limit exceeded
*Jul 10 16:50:12.699: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor 192.168.30.44 (Tunnel77) is up: new adjacency
*Jul 10 16:51:32.205: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor 192.168.30.44 (Tunnel77) is down: retry limit exceeded
*Jul 10 16:51:35.640: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor 192.168.30.44 (Tunnel77) is up: new adjacency
*Jul 10 16:52:55.148: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor 192.168.30.44 (Tunnel77) is down: retry limit exceeded

[MHM Cisco World]

if tunnel77 is DMVPN tunnel 
if router have two or more DMVPN tunnel with same source interface 
AND IF tunnelS use same tunnel key then this can happened 

[MicJameson1]

Hi MHM.

if tunnel77 is DMVPN tunnel = TRUE
if router have two or more DMVPN tunnel with same source interface =TRUE
tunnels use same tunnel key = FALSE.

additional data...
-tunnel protection ipsec profiles are different= TRUE
-tunnels themselves are not flapping. = TRUE

Please advise.

[MHM Cisco World]

you answer is excellent, 
will check via LAB.

thank
MHM

[MicJameson1]

fyi.. 

(obfuscated & output ommitted)
2921#sh run int tu20
interface Tunnel66
bandwidth 100000
ip address 192.168.102.66 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication ALPHA1
ip nhrp map multicast 216.1.1.1
ip nhrp map 192.168.1.1 216.1.1.1
ip nhrp network-id 66
ip nhrp nhs 192.168.1.1
ip tcp adjust-mss 1360
tunnel source GigabitEthernet0/1
tunnel mode gre multipoint
tunnel key 6666
tunnel vrf VRF66
tunnel protection ipsec profile PROFILE66
end

2921#sh run int tu77
Building configuration...

Current configuration : 457 bytes
!
interface Tunnel30
bandwidth 20000
ip address 192.168.2.2 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication BETA1
ip nhrp map 192.168.77.1 52.1.1.1
ip nhrp map multicast 52.1.1.1
ip nhrp network-id 77
ip nhrp holdtime 450
ip nhrp nhs 192.168.77.1
ip nhrp redirect
ip tcp adjust-mss 1360
tunnel source GigabitEthernet0/1
tunnel mode gre multipoint
tunnel key 7777
tunnel protection ipsec profile PROFILE77
end

==========================================

The initial data was from the hub device. The above data is from the spoke device.

The spoke shows that currently the hub is NOT an EIGRP neighbor.

==========================================

Hub data below...

CSR_1000v_HUB#sh run int tu77

interface Tunnel77

bandwidth 700000
ip address 192.168.77.1 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication BETA1
ip nhrp network-id 77
ip nhrp holdtime 450
ip nhrp redirect
ip tcp adjust-mss 1360
delay 1000
tunnel source GigabitEthernet1
tunnel mode gre multipoint
tunnel key 7777
tunnel protection ipsec profile PROFILE77
end

[MHM Cisco World]

tunnel vrf VRF66 <<- why tunnel vrf, are you use front-VRF in your router ?

[MicJameson1]

There are three total tunnels on spoke-- tu55,tu66,tu77.

66 and 77 use same interface.

55, 66 use different interfaces to same (other) 66 hub. tu77 hub is different device than tunnel 66 hub.

hub66, hub 77 have NO connection between them at all.

Below is VRF info from the 77 hub 1000v router...

(I don't understand info below. I do not believe we are using VRF on the hub. Spoke has strange situation, it is unique config using VRF. I believe this was done because we use two ISPs for two tunnels to same tu66 device.)

(Hub77 1000v)#sh vrf ipv4 br
Name        Default RD         Protocols          Interfaces
FVRF          <not set>             ipv4
GS             100:100              ipv4

[MHM Cisco World]

I run lab test two tunnel two profile two key and it work.

What I notice here is holdtime is high 450 I think register timeout is more less than hold time and this make Hub remove this tunnel and make eigrp flapping.

Check register timeout in hub 

Thanks 

MHM

IOU3# show runb
IOU3# show runb
^
% Invalid input detected at '^' marker.

IOU3#show run
IOU3#show running-config
Building configuration...

Current configuration : 2963 bytes
!
! Last configuration change at 08:20:47 UTC Wed Jul 12 2023
!
version 15.5
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname IOU3
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
!
!
!
bsd-client server url https://cloudsso.cisco.com/as/token.oauth2
mmi polling-interval 60
no mmi auto-configure
no mmi pvc
mmi snmp-timeout 180
!
!
!
!
!
no ip icmp rate-limit unreachable
!
!
!
!
!
!
!
!

!
!
!
!
no ip domain lookup
ip cef
no ipv6 cef
!
multilink bundle-name authenticated
!
!
!
!
!
!
!
cts logging verbose
!
!
!
redundancy
!
!
ip tcp synwait-time 5
!
!
!
!
!
crypto isakmp policy 10
hash md5
authentication pre-share
crypto isakmp key mhm address 0.0.0.0
!
!
crypto ipsec transform-set mhm esp-des
mode tunnel
!
!
crypto ipsec profile mhm15
set transform-set mhm
!
crypto ipsec profile mhm5
set transform-set mhm
!
!
!
!
!
!
interface Loopback0
ip address 3.3.3.3 255.255.255.255
!
interface Tunnel0
ip address 5.0.0.3 255.255.255.0
no ip redirects
ip nhrp map 5.0.0.1 100.0.0.1
ip nhrp map multicast 100.0.0.1
ip nhrp network-id 5
ip nhrp nhs 5.0.0.1
tunnel source Ethernet0/0
tunnel mode gre multipoint
tunnel key 5
tunnel protection ipsec profile mhm5
!
interface Tunnel1
ip address 15.0.0.3 255.255.255.0
no ip redirects
ip nhrp map 15.0.0.2 110.0.0.2
ip nhrp map multicast 110.0.0.2
ip nhrp network-id 15
ip nhrp nhs 15.0.0.2
tunnel source Ethernet0/0
tunnel mode gre multipoint
tunnel key 15
tunnel protection ipsec profile mhm15
!
interface Ethernet0/0
ip address 120.0.0.3 255.255.255.0
!
interface Ethernet0/1
no ip address
shutdown
!
interface Ethernet0/2
no ip address
shutdown
!
interface Ethernet0/3
no ip address
shutdown
!
interface Ethernet1/0
no ip address
shutdown
!
interface Ethernet1/1
no ip address
shutdown
!
interface Ethernet1/2
no ip address
shutdown
!
interface Ethernet1/3
no ip address
shutdown
!
interface Serial2/0
no ip address
shutdown
serial restart-delay 0
!
interface Serial2/1
no ip address
shutdown
serial restart-delay 0
!
interface Serial2/2
no ip address
shutdown
serial restart-delay 0
!
interface Serial2/3
no ip address
shutdown
serial restart-delay 0
!
interface Serial3/0
no ip address
shutdown
serial restart-delay 0
!
interface Serial3/1
no ip address
shutdown
serial restart-delay 0
!
interface Serial3/2
no ip address
shutdown
serial restart-delay 0
!
interface Serial3/3
no ip address
shutdown
serial restart-delay 0
!
!
router eigrp 5
network 3.3.3.3 0.0.0.0
network 5.0.0.0 0.0.0.255
network 15.0.0.0 0.0.0.255
!
ip forward-protocol nd
!
!
no ip http server
no ip http secure-server
ip route 0.0.0.0 0.0.0.0 120.0.0.4
!
!
!
!
control-plane
!
!
!
!
!
!
!
!
line con 0
exec-timeout 0 0
privilege level 15
logging synchronous
line aux 0
exec-timeout 0 0
privilege level 15
logging synchronous
line vty 0 4
login
transport input none
!
!

[MicJameson1]

THnak you MHM.

"What I notice here is holdtime is high 450 I think register timeout is more less than hold time and this make Hub remove this tunnel and make eigrp flapping.

Check register timeout in hub"

How do I check this? What are commands?

[MHM Cisco World]

[MicJameson1]

Hello Team.

Because of the strict nature of this enterprise I cannot troubleshoot this tunnel until a service window is opened. That will not be for two weeks. 

I will update this thread at that time. 

Thank you for your inspired effort.

[David Ruess]

Hello,

Can you provide the configuration between the 2 devices giving this error along with a diagram?

Additionally if you take the IPSEC profiles off does the neighborship form without issue? If that's the case your issues is likely the IPSEC configuration.

[paul driver]

Hello

---

@MicJameson1 wrote:

(obfuscated & output ommitted)
2921#sh run int tu20
interface Tunnel66
bandwidth 100000
ip address 192.168.102.66 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication ALPHA1
ip nhrp map multicast 216.1.1.1
ip nhrp map 192.168.1.1 216.1.1.1
ip nhrp network-id 66
ip nhrp nhs 192.168.1.1
ip tcp adjust-mss 1360
tunnel source GigabitEthernet0/1
tunnel mode gre multipoint
tunnel key 6666
tunnel vrf VRF66
tunnel protection ipsec profile PROFILE66
end

2921#sh run int tu77
Building configuration...

Current configuration : 457 bytes
!
interface Tunnel30
bandwidth 20000
ip address 192.168.2.2 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication BETA1
ip nhrp map 192.168.77.1 52.1.1.1
ip nhrp map multicast 52.1.1.1
ip nhrp network-id 77
ip nhrp holdtime 450
ip nhrp nhs 192.168.77.1
ip nhrp redirect
ip tcp adjust-mss 1360
tunnel source GigabitEthernet0/1
tunnel mode gre multipoint
tunnel key 7777
tunnel protection ipsec profile PROFILE77
end

==========================================

The initial data was from the hub device. The above data is from the spoke device.

The spoke shows that currently the hub is NOT an EIGRP neighbor.

==========================================

Hub data below...

CSR_1000v_HUB#sh run int tu77

interface Tunnel77

bandwidth 700000
ip address 192.168.77.1 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication BETA1
ip nhrp network-id 77
ip nhrp holdtime 450
ip nhrp redirect
ip tcp adjust-mss 1360
delay 1000
tunnel source GigabitEthernet1
tunnel mode gre multipoint
tunnel key 7777
tunnel protection ipsec profile PROFILE77
end

---

Append the following and test again:

NHS - Hub
Int tun xx
 ip nhrp map multicast dynamic
no ip next-hop-self eigrp xx
no ip split-horizon eigrp xx


NHC -client
Int tun xx
ip nhrp map multicast dynamic