EIGRP-5-nbrchange Dual Vlan 1 down retry limit received

[TRACY HARTMANN]

We have two 9300 Nexus and the issue we are having is some Cisco routers we have connected have the following error.

2020 Aug 21 20:55:32 INHQSW01 %EIGRP-5-NBRCHANGE_DUAL: eigrp-XXX [29795] (defau
lt-base) IP-EIGRP(0) 800: Neighbor XX.XX.XX.XX (Vlan1) is up: new adjacency
2020 Aug 21 20:55:32 INHQSW01 %EIGRP-5-NBRCHANGE_DUAL: eigrp-XXX [29795] (defau
lt-base) IP-EIGRP(0) 800: Neighbor xx.xx.xx.xx (Vlan1) is up: new adjacency
2020 Aug 21 20:56:51 INHQSW01 %EIGRP-5-NBRCHANGE_DUAL: eigrp-XXX [29795] (defau
lt-base) IP-EIGRP(0) 800: Neighbor xx.xx.xx.xx (Vlan1) is down: Peer goodbye rec
eived
2020 Aug 21 20:56:51 INHQSW01 %EIGRP-5-NBRCHANGE_DUAL: eigrp-XXX [29795] (defau
lt-base) IP-EIGRP(0) 800: Neighbor xx.xx.xx.xx (Vlan1) is down: retry limit exce
eded
2020 Aug 21 20:56:51 INHQSW01 %EIGRP-5-NBRCHANGE_DUAL: eigrp-XXX[29795] (defau
lt-base) IP-EIGRP(0) 800: Neighbor xx.xx.xx.xx (Vlan1) is down: Peer goodby

Nothing looks wrong and we have checked ports to layer 2 is fine, they are on the same EIGRP AS #.

[Richard Burts]

We do not have much information to work with. There is mention of 2 9300 and some routers but not much about them. And it is not clear to me whether the log messages posted are from the 9300 or from one of the other routers. And the replacement of all IP address information by xxx.xxx makes it impossible to know if there is one neighbor doing this or if there are multiple neighbors.

The most significant part of the log messages posted are the messages like this

lt-base) IP-EIGRP(0) 800: Neighbor xx.xx.xx.xx (Vlan1) is down: Peer goodbye rec
eived

which indicate that a peer goodbye was received. This is an enhancement introduced into EIGRP that when a L3 device running EIGRP is going to terminate the neighbor relationship for some reason that it sends a goodbye message. So the problem is not so much on this device as it is on the neighbor that sent the goodbye message. 

[TRACY HARTMANN]

Sorry the logs are from the 9300 that does not have the routers directly connected to it. The other 9300 has no issues.

2020 Aug 22 00:07:47 INHQSW01 %EIGRP-5-NBRCHANGE_DUAL: eigrp-800 [29795] (defau
lt-base) IP-EIGRP(0) 800: Neighbor 10.12.64.87 (Vlan1) is up: new adjacency
2020 Aug 22 00:08:16 INHQSW01 %EIGRP-5-NBRCHANGE_DUAL: eigrp-800 [29795] (defau
lt-base) IP-EIGRP(0) 800: Neighbor 10.12.64.41 (Vlan1) is down: retry limit exce
eded
2020 Aug 22 00:08:16 INHQSW01 %EIGRP-5-NBRCHANGE_DUAL: eigrp-800 [29795] (defau
lt-base) IP-EIGRP(0) 800: Neighbor 10.12.64.41 (Vlan1) is up: new adjacency

[Richard Burts]

Thanks for the additional information. And thanks for letting the IP addresses be seen. Since they are all in the private network 10.0.0.0 there are no issues about privacy or security for those addresses. 

I am interested that in the original post there were several instances of the Peer goodbye received message, but in this post there are none. Is it common to have them in the logs or is it unusual? As I mentioned when you get this message the router with an issue is the router that sent the Peer Goodbye message and not the router that received the message.

Perhaps it is just terminology but I am a bit puzzled when you describe the 9300 as "the 9300 that does not have the routers directly connected to it." If the 9300 is not directly connected then how are they EIGRP neighbors? Perhaps you can give us some information about the topology of this network?

The messages in this post show a neighbor at 10.12.64.41 going down because of retry limit exceeded. It might be helpful to look at the logs from that router and see what it shows in that time period. I am surprised to see the next log message with the same time stamp shows that neighbor is back up with a new adjacency.

[Christopher Hart]

Hi Tracy!

I am going to assume that both of your Nexus 9300s are in a vPC domain, and they are connected to one or more EIGRP-enabled routers through a vPC VLAN. One Nexus 9300 appears to be directly connected to the routers, while the other Nexus 9300 would need to communicate with the routers over the vPC Peer-Link.

If we refer to the "Supported Topologies for Routing in a vPC Environment" section of the Supported Topologies for Routing over Virtual Port Channel on Nexus Platforms document, we see a topology. In your scenario, the EIGRP adjacency that is flapping is from Nexus-B to L3-A.

If the above is correct, then you need to verify that both the peer-gateway and layer3 peer-router commands are enabled under the vPC domain of both Nexus 9300s. The layer3 peer-router command was introduced in NX-OS software release 7.0(3)I5(1), so if you do not have the ability to run this command, you likely need to upgrade to a newer NX-OS software release (I would recommend NX-OS software release 7.0(3)I7(8) - it's the current general recommended release).

I hope this helps - thank you!

-Christopher

[TRACY HARTMANN]

The one thing I am seeing on the two 9300 is they are not the same config:

INHQSW01#

vpc domain 4

  role priority 1

  peer-keepalive destination 10.12.64.14

  peer-gateway

  layer3 peer-router

  ip arp synchronize

INHQSW02#

vpc domain 4

  role priority 1

  peer-keepalive destination 10.12.64.13

  peer-gateway

  ip arp synchronize

From what I read it looks like we should have both 9300 with the  layer3 peer-router....

[Christopher Hart]

Hi Tracy!

You are correct - both vPC peers need to have the layer3 peer-router command applied under the vPC domain in order for it to take effect in a supported manner.

Thank you!

-Christopher