Fabric Interconnect // CRC errors on IO module Fabric ports ( Fabric-A ) only

[Mohammad Al-Ammouri]

Hello team,

We have a lot of CRC errors on the UCS manger / IO modules Fabric ports connected to FI-A only.

CRC errors increasing on Fabric ports of IO modules in all chassis ( A-side only) causing threshold errors. 

 FI-A , trying different cables and swapping the IO modules ( A and B) haven't resolved the issue ( still errors on even new IO module connected to FI-A).

Just wanted to check if anyone faced similar issue before replacing the FI.

Thanks in advance

[Walter Dey]

Q. how many links between IOM and FI ?

Q. how long are the twinax cables cables ? Cisco branded ?

Q. did you try other 10G ports on the FI ?

[Mohammad Al-Ammouri]

Hi Walter,

We have 8 x Chassis and we have CRC errors on all IOM-A fabric ports on all chassis ( Four from each chassis).

cables are 5 meters long.

[Keny Perez]

Can you paste a sample of the "sh interface x/y" command for the interfaces experiencing the issue?

-Kenny

[Mohammad Al-Ammouri]

Hello Team,

It turned out that one of the Ethernet uplink ports has much more CRC errors compared to IO modules fabric ports.. Once this port disbaled, CRC erros stopped on IO modules ports.

it could be Twinax cable issue on this specific uplink port.

But is this exprcted behavior ?

Thanks All for your help.

[Keny Perez]

If those CRC errors were Stomped CRCs then yes, as those frames would be sent/marked downstream to be discarded.

-Kenny

[Walter Dey]

http://thenetworksherpa.com/cut-through-corruption-and-crc-stomping/

explains CRC errors in cut thorugh switching !

Q. what is the FI Northbound connectivity ? copper or fibre ?

[junior.beatty]

I am running into a similar situation.  Chassis 1 is connected to FI A ports 3 and 4 via IOM A.  Both of these ports are showing CRC errors.  Chassis 1 connected to FI B ports 3 and 4 via IOM B do not have any CRC errors.  After some testing I think it is a physical issue, either the twinax cables or the IOM module itself.  I do have an open TAC case.  I have requested a new IOM and two twinax cables to replace what is currently there.  Unfortunately the data center is about 4 hours away, so have to schedule time to get there to swap IOM and cables.  Just curious as to what others think.

[Keny Perez]

Most important thing will be to confirm the CRC counters are going up so that you know if this is happening now or if it is old information.

TAC will also be able to see if the errors are "Stomp" or "not-stomp"

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus5000/sw/troubleshooting/guide/N5K_Troubleshooting_Guide/n5K_ts_l2.html#pgfId-1029225

HTH,

-Kenny

[junior.beatty]

CRC's are still incrementing on a consistent basis.

[Keny Perez]

Have they told you if they are stomp or not-stomp CRC errors?

-Kenny

[junior.beatty]

No not yet.  I provided show techs, figured they should be able to figure out the issue from the show techs.  Thanks for the info.

[Keny Perez]

np, keep us updated.

-Kenny

[junior.beatty]

Yep, no problem with that.

[Walter Dey]

Hi Mohammad

Q. Is it correct ? you have 8 chassis, and this problem is only seen on one FI ?

Q. are all ports on the FI facing the chassis IOM server ports ?

Q. Which model of FI ? can you please post the configuration ?

This must be a problem with the FI, not the IOM ?