Troubleshooting Nexus switchports

[Tim Glen]

Hi All,

I have two Nexus 5596-UP both running NX-OS 7.0(6)N1(1).  Both of these have an N55-M16P in slots 2 & 3. They are vPC peers.

Here is the problem I’m experiencing.  There are ports on both switches that do not come up when they are connected to another device (switch or server or looped to another switchport on the same switch)

This problem affects 6 switch ports.

Nexus-A eth2/10,   

Nexus-B eth2/10

Nexus-B eth 3/11-14

These ports never link up.   I’m wondering if this is a HW issue, but I think it’s odd that eth 2/10 on BOTH switches doesn’t work.

The TAC engineer and myself have reviewed the ‘show accounting log,  show logging logfile and show system internal ethpm event-history interface (affected interfaces).  

I’m requesting a maintenance window to reseat the modules but I have little confidence that reseating them will fix the issue.     

For a while I assumed it may be CSCus09929 but the TAC engineer states it’s not.

Does anyone have any ideas or helpful troubleshooting steps ?

Thanks! 

Tim

[Reza Sharifi]

Hi Tim,

I don't think reseating the modules will fix the issue as you have 2 switches with the same issue.

What is TAC's solution for this?

 Thanks,

[Tim Glen]

TAC is requesting I reseat the modules and if the problem persists they will replace them.    I may be missing something but I find it really odd that its port 2/10 on both switches and ports 3/11-14 on another.    

I've been unable to find any docs that discuss which ports share ASIC's.   I read a little about how the 5500 allows 8 ports to share a Unified Port Controller but the document didn't go deeper into which UPC's control which ports.   Ports 11-14 just seems odd,    if it was 13 - 16 (last 4) id be more conformable with hardware failure.  

Any tips would be greatly appreciated.

Thanks!

Tim

[Tim Glen]

Last week I pulled the module out of the Nexus-B switch , then the Nexus experienced a Kernel Panic and rebooted itself.   This was caused by bug CSCtz05620 which isn't public at this time but details are below. Separately, it's HIGHLY frustrating that this bug isn't public.  I'd assume that this is a Catastrophic bug and assuming that the 5x00 line of switches & the N55-M16P are both very popular there is likely a fair number of folks that have the possibility of running into this.    @Cisco, make this public. 

Symptom:
A Nexus 5k switch may experience a kernel panics When UGEM (n55-M16P) is pulled out from the chassis.

Conditions:
Module OIR (pulled out) from N5k Switch.

Workaround:
Power off GEM before pulling it out.

SWITCH# config t
SWITCH(config)# poweroff module x

Obviously that was unexpected since the Nexus is supposed to support OIR for this card but the bug is pretty clear. Strangely though, the ports worked after the reboot. 

The other switch Nexus-A still had eth2/10 3/15 & 3/16 that were not working.   I rebooted that switch last night and to my surprise after the reboot they are working now.   

All I have to say is,  odd....