Nexus 5548 Link is operationally individual

[Robert Wortmann]

Hey,

we have 2 FlexPod (DC1 and DC2) with two Nexus 5548 on each side. We connected each Nexus on DC1 with the one on DC2 with 2 ports (Ethernet 1/21,1/22) in the port-channel 15. All worked fine until the day one SFP module was broken and we changed it. After a few minutes we noticed that a few Esxi hosts weren't connected anymore. A "show interface ethernet 1/22" showed the state "Link not connected", the "show port-channel sum" showed this:

15    Po15(SU)    Eth      LACP      Eth1/21(P)   Eth1/22(D)

I also noticed a syslog:

2013 Jul 22 12:10:36 mez: %ETH_PORT_CHANNEL-4-PORT_INDIVIDUAL: port Ethernet1/22 is operationally individual

After a few minutes the port-channel was fully  up again and we could the Esxi hosts. Shutting down the ports and "no shutdown" them didn't work either.

But a few minutes later the old problem came up.

Now it changes between these two states every 15 minutes. I think it is a problem with the lacp channel, but the config seems fine. I hope you can help me - if you need any further information, just ask!

Regards Robert

[Reza Sharifi]

Hi,

Have you tried deleting the portchannel and rebuilding it?

Also, are the links betwen the 5ks configured as vPC peer-link?

Can you post the portchannel and the physical ports configs?

HTH

[Robert Wortmann]

Yes I also tried this - same result.

No they're just a normal vPC.

dc1-n5k-02:

interface Ethernet1/22

  description dc2-n5k-01

  shutdown

  switchport mode trunk

  channel-group 15 mode active

interface port-channel15

  description DC Connection

  switchport mode trunk

  spanning-tree port type network

  vpc 15

dc2-n5k-01:

interface Ethernet1/22

  description dc1-n5k-02

  shutdown

  switchport mode trunk

  channel-group 15 mode active

interface port-channel15

  description DC Connection

  switchport mode trunk

  spanning-tree port type network

  vpc 15

"Shutdown" is because of the disconnections of the ESXi Hosts.

Regards

Robert

[Kevin Dorrell]

I would look at whether your ESXi hosts are correctly configured for LACP bundling.

Kevin Dorrell

Luxembourg

[Robert Wortmann]

I also checked this - ESXi are are configured correctly.

Regards

Robert

[kdebrouw]

Hi Robert,

Could you share the output of show lacp counters interface port-channel15?

If we would be seeing a lot of difference in the RX and TX of LACPDU's we can assume 3 things, either we're not receiving them and we have a link problem, they're getting dropped in the path towards the CPU by copp or rate-limiting on the asic or the CPU has not enough time to process them because we have another high priority process consuming cpu cycles.

Kristof