Nexus 5596UP repeating CDP "device discovered of type Nexus 1010

sokarlsson · ‎02-02-2012

We have had this since we installed the Nexus 1010s but I want to get some help to clarify why.

We have 2 Nexus 1010 in HA. One is connected to a Nexus 5596UP vPC pair in one DC. The other connected to a second Nexus 5596UP vPC pair in the other DC. All 4 5596UP run 5.0(3)N1(1). Nexus 1010 run 4.2(1)SP1(3)

We see this message frequently in the logs of the Nexus 5596 switches

2012 Jan 14 17:40:25 NRK-BA01-SW1 %CDP-5-NEIGHBOR_REMOVED: CDP Neighbor NTM-DC-VSA-01(QCI1533A662) on port Ethernet1/30

has been removed

2012 Jan 14 17:40:25 NRK-BA01-SW1 %CDP-5-NEIGHBOR_REMOVED: CDP Neighbor NTM-DC-VSA-01(QCI1533A662) on port Ethernet1/32

has been removed

2012 Jan 14 17:40:25 NRK-BA01-SW1 %CDP-5-NEIGHBOR_REMOVED: CDP Neighbor NTM-DC-VSA-01(QCI1533A662) on port Ethernet1/31

has been removed

2012 Jan 14 17:40:25 NRK-BA01-SW1 %CDP-5-NEIGHBOR_ADDED: Device NTM-DC-VSA-01(QCI1533A662) discovered of type Nexus1010

with port eth0 on incoming port Ethernet1/30 with ip addr 255.255.255.255 and mgmt ip <no MGMT IP Addr Received>

2012 Jan 14 17:40:25 NRK-BA01-SW1 %CDP-5-NEIGHBOR_ADDED: Device NTM-DC-VSA-01(QCI1533A662) discovered of type Nexus1010

with port eth3 on incoming port Ethernet1/32 with ip addr 255.255.255.255 and mgmt ip <no MGMT IP Addr Received>

2012 Jan 14 17:40:25 NRK-BA01-SW1 %CDP-5-NEIGHBOR_ADDED: Device NTM-DC-VSA-01(QCI1533A662) discovered of type Nexus1010

with port eth5 on incoming port Ethernet1/31 with ip addr 255.255.255.255 and mgmt ip <no MGMT IP Addr Received>

I do not see the message at the same time in both switches of a vPC pair.

krun_shah · ‎02-02-2012

Do you have management IP configured for Nexus 1010 ?? Alternatively you can launch kvm and turn off cdp on nexus 1010 using "no cdp enable" configuration command till you configure management IP on it.

sachinraja · ‎02-02-2012

I have the same issue , even if the management IP is configured.

We were asked to change the CDP timer in Nexus 1010 to 120 secs to resolve this issue. I have still not tried it.. May be you can try it if your setup is still not in production..

Regards

sokarlsson · ‎02-02-2012

Thanks for replying. Yes the 1010 do have a management ip address

NTM-DC-VSA-01# show run int mgmt0

interface mgmt0

ip address 10.16.4.20/24

I have checked the cdp status and timers

show cdp interface mgmt0

Interface Index :83886080
Interface mgmt0:
    Operational status: up
    Config status: enabled
    Refresh time: 60
    Hold time: 180

The other weird thing is that the only CDP neigh the 1010 see is the VSM that resides on it.

NTM-DC-VSA-01# show cdp neigh det
----------------------------------------
Device ID:NTM-DC-VSM-01(1635496009132078588)
System Name: NTM-DC-VSM-01

Interface address(es):
IPv4 Address: 10.16.4.21
Platform: Nexus1000V, Capabilities: Switch IGMP Filtering Supports-STP-Dispute
Interface: mgmt0, Port ID (outgoing port): mgmt0
Holdtime: 177 sec

Version:
Cisco Nexus Operating System (NX-OS) Software, Version 4.2(1)SV1(4a)

Advertisement Version: 2
Duplex: full

MTU: 1500
Physical Location: snmplocation
Mgmt address(es):
IPv4 Address: 10.16.4.21

This is what we normally see on one of the Nex5596UP in the vPC pair this 1010 connects to

LNK-DE27-SW1# show cdp neigh int eth1/30 det
----------------------------------------
Device ID:NTM-DC-VSM-01(1635496009132078588)
System Name:NTM-DC-VSM-01
Interface address(es):
IPv4 Address: 10.16.4.21
Platform: Nexus1000V, Capabilities: Switch IGMP Filtering Supports-STP-Dispute
Interface: Ethernet1/30, Port ID (outgoing port): mgmt0
Holdtime: 154 sec

Version:
Cisco Nexus Operating System (NX-OS) Software, Version 4.2(1)SV1(4a)

Advertisement Version: 2
Duplex: full
Physical Location: snmplocation
Mgmt address(es):
IPv4 Address: 10.16.4.21

----------------------------------------
Device ID:NTM-DC-VSA-01(QCI1533A7Q3)
System Name:
Interface address(es):
IPv4 Address: 255.255.255.255
Platform: Nexus1010, Capabilities: Host
Interface: Ethernet1/30, Port ID (outgoing port): eth0
Holdtime: 167 sec

Version:
Cisco Nexus Operating System (NX-OS) Software, Version 4.2(1)SP1(3)

Advertisement Version: 2
Duplex: full

----------------------------------------
Device ID:NTM-DC-VSA-01(1610440479132074267)
System Name:NTM-DC-VSA-01
Interface address(es):
IPv4 Address: 10.16.4.20
Platform: Nexus1010, Capabilities: Switch IGMP Filtering Supports-STP-Dispute
Interface: Ethernet1/30, Port ID (outgoing port): mgmt0
Holdtime: 167 sec

Version:
Cisco Nexus Operating System (NX-OS) Software, Version 4.2(1)SP1(3)

Advertisement Version: 2
Duplex: full
Physical Location: snmplocation
Mgmt address(es):
IPv4 Address: 10.16.4.20

10.16.4.20 is the mgmt0 of the VSM virtual blade. I have no idea where "interface eth0" is located on the VSA since there is no such interface.

NTM-DC-VSA-01# show interface brief

--------------------------------------------------------------------------------
Port VRF Status IP Address Speed MTU
--------------------------------------------------------------------------------
mgmt0 -- up 10.16.4.20 1000 1500

--------------------------------------------------------------------------------
Port VRF Status IP Address Speed MTU
--------------------------------------------------------------------------------
control0 -- up -- 1000 1500

sokarlsson · ‎02-02-2012

sashinraja

Is that 120sec timer (refresh) or hold timer? Increasing the refresh timer sounds backwards.

krun_shah · ‎02-02-2012

this looks liek problem with Nexus 1010 Management Software version 4.2(1)SP1(3) as you can see in show cdp neigh int eth 1/30 detail the port shows IPv4 address of hidden eth0 interface is 255.255.255.255 which is invalid. What is the uplink type that you use to connect to Nexus 5500?? Try disabling CDP and re-enable it.

sokarlsson · ‎02-02-2012

kruh_shah

Yes we are using 4.2(1)SP1(3). We are running uplink type 3

I will start with trying to disable and enable CDP first. After that it's time to fix the broken primary Nexus 1010.(See below)

sachinraja · ‎02-02-2012

Hi Karlsson

How many VSM's do you have with Nexus 1010 ?

Can you send the show module from the VSM's ? There is a chance that the VSM is built from the standby unit, and due to that the Nexus 7k fails CDP with interfaces going to standby unit since all configs are done through active unit.. We had one of the VSM's which rebooted and flipped over to standby 1010. Since then the CDP fails with standby unit since this unit does not have any mgmt ip configured..

Regards

Raj

sokarlsson · ‎02-02-2012

Actually at the moment we are in a situation that is rather broken. The Primary 1010 has crashed after a power outage and the secondary 1010 does not look fully healthy either. Could be that.

NTM-DC-VSA-01# show mod

Mod Ports Module-Type Model Status

--- ----- -------------------------------- ------------------ ------------

1 0 Virtual Supervisor Module powered-up

2 0 Nexus 1010 (Virtual Services App Nexus1010 active *

Mod Sw Hw

--- ---------------- ------------------------------------------------

2 4.2(1)SP1(3) 0.0

Mod MAC-Address(es) Serial-Num

--- -------------------------------------- ----------

2 00-19-07-6c-5a-a8 to 00-19-07-6c-62-a8 NA

Mod Server-IP Server-UUID Server-Name

--- --------------- ------------------------------------ --------------------

2 10.16.4.20 NA NA

NTM-DC-VSA-01# show mod 1

Mod Ports Module-Type Model Status

--- ----- -------------------------------- ------------------ ------------

1 0 Virtual Supervisor Module powered-up

Mod Sw Hw

--- ---------------- ------------------------------------------------

Mod MAC-Address(es) Serial-Num

--- -------------------------------------- ----------

Mod Server-IP Server-UUID Server-Name

--- --------------- ------------------------------------ --------------------

I'm going to reinstall the Primary Nexus 1010 tomorrow. However we did see these messages on the Nexus 5596s attached to the secondary 1010 even before the Primary crashed.

sachinraja · ‎02-02-2012

Yes.. Give it a shot.. Once it registers back to primary 1010 you should be fine..

By the way, if primary is crashed now, are you able to login to the iLo port of secondary, and do a "connect host" and see if you see the mgmt IP flipped to the secondary? If not, cdp will fail.. Try to see if you are able to get onto 'conf t' on secondary now..

Raj

sokarlsson · ‎02-02-2012

I can log into the VSA and VSM as usual. The mgmt interface is active on the secondary.
This for sure must be the whole point of having a HA cluster with the secondary becoming active if the primary fails?

I will get back to both of you with the results.

sokarlsson · ‎02-02-2012

OK this is going to be a bit long winded so bare with me.............

Disabling and enabling CDP on the VSA gave me the following results to begin with in the two Nex 5596's

NTM-VSA-001# no cdp en

LNK-DE27-SW1#

2012 Feb 2 22:55:49 LNK-DE27-SW1 %CDP-5-NEIGHBOR_REMOVED: CDP Neighbor NTM-DC-VSA-01(1610440479132074267) on port Ethernet1/30 has been removed

LNK-DE27-SW2#

2012 Feb 2 22:55:49 LNK-DE27-SW2 %CDP-5-NEIGHBOR_REMOVED: CDP Neighbor NTM-DC-VSA-01(1610440479132074267) on port Ethernet1/32 has been removed

The Nex 5596's still see the VSA as a CDP neigh on the ports for Ctrl and Packet.

2,5 min later CDP time out on the ports for Ctrl and Packet

2012 Feb 2 22:58:23 LNK-DE27-SW1 %CDP-5-NEIGHBOR_REMOVED: CDP Neighbor NTM-DC-VSA-01(QCI1533A7Q3) on port Ethernet1/32 has been removed

2012 Feb 2 22:58:23 LNK-DE27-SW1 %CDP-5-NEIGHBOR_REMOVED: CDP Neighbor NTM-DC-VSA-01(QCI1533A7Q3) on port Ethernet1/31 has been removed

2012 Feb 2 22:58:23 LNK-DE27-SW2 %CDP-5-NEIGHBOR_REMOVED: CDP Neighbor NTM-DC-VSA-01(QCI1533A7Q3) on port Ethernet1/32 has been removed

2012 Feb 2 22:58:23 LNK-DE27-SW2 %CDP-5-NEIGHBOR_REMOVED: CDP Neighbor NTM-DC-VSA-01(QCI1533A7Q3) on port Ethernet1/31 has been removed

I now made sure the VSA CDP entries had timed out + waited another 3min to be sure no CDP entries from the VSA popped up.

Controlling the CDP neighbors on the Nex 5596's now showed eachother, the upstream VSS system + the VSM on all ports connected to ESX hosts with installed VEMs =)

I now switched CDP off on the mgmt0 and the control0 interfaces of the 1010.
Time to switch CDP on globally to see if something happens though it should not and guess what.....

LNK-DE27-SW2# 2012 Feb 2 23:08:28 LNK-DE27-SW2 %CDP-5-NEIGHBOR_ADDED: Device NTM-DC-VSA-01(1610440479132074267) discovered of type Nexus1010 with port control0 on incoming port Ethernet1/32 with ip addr 10.16.4.20 and mgmt ip 10.16.4.20

As it turns out swtching CDP on globally in the 1010 also turns it on for interface control0. However the CDP entry only shows up on one of four Ctrl and Packet interfaces on the 5596. Only one. This just ask for more investigation so I try to switch CDP off on interface control0 on the VSA but that does not work. Apparently one can disable it locally on mgmt0 but not on control0.

Interface control0:

Operational status: up

CDP global config status: disabled

Config status: enabled

Refresh time: 60

Hold time: 180

NTM-DC-VSA-01# show cdp interface mgmt 0

Interface Index :83886080

Interface mgmt0:

Operational status: up

CDP global config status: disabled

Config status: disabled

Refresh time: 60

Hold time: 180

So what happens when if one first switch on CDP on globally while it's disabled on mgmt0 and after a while enable it on mgmt0

enable CDP globally forcing it enabled on control 0:

LNK-DE27-SW2# 2012 Feb 2 23:22:07 LNK-DE27-SW2 %CDP-5-NEIGHBOR_ADDED: Device NTM-DC-VSA-01(1610440479132074267) discovered of type Nexus1010 with port control0 on incoming port Ethernet1/32 with ip addr 10.16.4.20 and mgmt ip 10.16.4.20

CDP discoverd on one 5596 on a Ctrl and Packet interface

Then enable CDP on mgmt0

2012 Feb 2 23:22:37 LNK-DE27-SW1 %CDP-5-NEIGHBOR_ADDED: Device NTM-DC-VSA-01(1610440479132074267) discovered of type Nexus1010 with port mgmt0 on incoming port Ethernet1/30 with ip addr 10.16.4.20 and mgmt ip 10.16.4.20

2012 Feb 2 23:22:37 LNK-DE27-SW1 %CDP-5-NEIGHBOR_ADDED: Device NTM-DC-VSA-01(QCI1533A7Q3) discovered of type Nexus1010 with port eth0 on incoming port Ethernet1/30 with ip addr 255.255.255.255 and mgmt ip

2012 Feb 2 23:22:37 LNK-DE27-SW1 %CDP-5-NEIGHBOR_ADDED: Device NTM-DC-VSA-01(QCI1533A7Q3) discovered of type Nexus1010 with port eth3 on incoming port Ethernet1/32 with ip addr 255.255.255.255 and mgmt ip

2012 Feb 2 23:22:37 LNK-DE27-SW1 %CDP-5-NEIGHBOR_ADDED: Device NTM-DC-VSA-01(QCI1533A7Q3) discovered of type Nexus1010 with port eth5 on incoming port Ethernet1/31 with ip addr 255.255.255.255 and mgmt ip

LNK-DE27-SW2# 2012 Feb 2 23:22:37 LNK-DE27-SW2 %CDP-5-NEIGHBOR_ADDED: Device NTM-DC-VSA-01(QCI1533A7Q3) discovered of type Nexus1010 with port eth2 on incoming port Ethernet1/32 with ip addr 255.255.255.255 and mgmt ip

2012 Feb 2 23:22:37 LNK-DE27-SW2 %CDP-5-NEIGHBOR_ADDED: Device NTM-DC-VSA-01(QCI1533A7Q3) discovered of type Nexus1010 with port eth4 on incoming port Ethernet1/31 with ip addr 255.255.255.255 and mgmt ip

After this it goes back to how it was before. Every 180sec the CDP neighbors that are related to CDP on eth0-5 is removed and instantly added. The entries related to control0 DO NOT get removed and added.

For now I'm going to disable CDP to get rid of the anoying messages in the log. Lowering the refresh to 30 sec do not affect CDP from eth0-5

jeff_bhz · ‎01-23-2019

I had a similar problem, same message but in a port-channel. A door in the channel group presented some runts and was flapping. Shutdown on the door and physical analysis. Problem solved.

Nexus 5596UP repeating CDP "device discovered of type Nexus 1010"