Nexus FEX to UCS CIMC connection droping.

[mahensin]

We are having issues here in our data center.

Background :  We have migrated all of the Catalyst switches with Nexus and FEX. 

Now we have all of the CIMC ports on UCS standalone servers ( UCS 220 and UCS 240 M3 and M4s connecting to the NEXUS FEX ( 2248 ) in different RACKS.

Problem Description: The CIMC ports keep on dropping at some random intervals. 

When I login to the Nexus, I find that ports are up and connected.

The only issue I find is it does not get any input packets from the server.

The only way to recover it is to bounce the port (shut/no shut).

It stays on for some time and then again runs into the same problem. 

Already tried:  We already tried to change the speed/duplex settings on the switch.

Servers CIMC are running in dedicated mode with static IP from CIMC.

I would appreciate if someone shares their experience if they ran into the same issue and have any recovery steps available. 

Thank you.

[Kirk J]

Greetings.

Do you notice any CRC errors for the ports?

I've seen some cases where you need to hard code the speed duplex on the CIMC side to 1000/100/full, while the CIMC and switch port disagree on the speed and the duplex, resulting probably 90% packet loss, lots of CRC errors, etc.

Did you happen to notice if the duplex got negotiated at half?

The Obfl file in the cimc tech support will note the speed/duplex it thinks it negotiated at.

If everything is hardcoded to speed/duplex, then set to auto/auto, or vs versa.

 If you CIMC firmware is fairly old, it might only show 100mb as max speed (its a 1Gb port), or not let you manually set speed/duplex.

Thanks,

Kirk...