C3 discards on a few cisco Fi / Cisco UCS 6332

[Vaikis]

Any suggestions what to check on cisco FI level to avoid c3 discards which we see in a few our Cisco UCS 6332 FI's ?  2 pairs runs without any errors and other 2 pairs reporting c3 via snmp.  According storage team all configurations are identical on there end. 

[Kirk J]

Greetings.

Would be helpful to know what kind of ports (i.e. Server, Uplink) and what the other device on the other side of the link is registering.

Is this only specific ports, or is it spread out among most of the ports?

Can you post the output of:

show int eth x/y 

show queuing int eth x/y

You can also see some CRC type error counters if something in your environment is trying to transmit jumbo frames and the frames are not tagged with a cos group that allows for the larger frames.

Kirk...

[Vaikis]

We catch c3 discards with some netapp monitoring tool, all discards are seen on vfc interface which is a FCoE port-channel created from 4x40G ports going towards SAN switch. With the same identical setup on other FI pair we don't see such errors at all. No any differences between storage back-end on both pairs.

Attached interface outputs used for a FCoE port-channel towards SAN switch.

[Kirk J]

what does the output from

show int count error 

show?

Any server facing ports showing any errors?

What is the average utilization, especially for inbound/RX on the UCSM FCOE related ports?

Are these Rack servers or Blade servers?  If blade servers, may need to take a look at IOMs and see if we have any drops/tail drops, or lots of pause frames, etc.

Input drops frequently indicates over subscription/congestion.

Kirk...

[Vaikis]

---

@Kirk J wrote:

what does the output from

show int count error 

show?

Any server facing ports showing any errors?

What is the average utilization, especially for inbound/RX on the UCSM FCOE related ports?

Are these Rack servers or Blade servers?  If blade servers, may need to take a look at IOMs and see if we have any drops/tail drops, or lots of pause frames, etc.

Input drops frequently indicates over subscription/congestion.

Kirk...

---

I have compared between FIs with no C3 discards and with

sh queuing interface on FCoE ports going towards storage backed switch  I see 0 dropped:

Pkts dropped by RX thresholds : 0
Bytes dropped by RX thresholds : 0

on FI with C3 discards:

Pkts dropped by RX thresholds : 22400199
Bytes dropped by RX thresholds : 12439472460

[Vaikis]

issue was QoS System Class count, when doing FcoE with Brocade SAN switch it works without C3 discards when you have 2 Classes defined and with more issues start to appear.

[Steven Tardy]

7 year old bug. You'd think UCSM would have a fault for this by now instead of sending a SYSLOG which 50% of UCS customers don't configure much less review.

CSCuh72875 :: FCoE multi-hop performance degraded because of QoS config mismatch