04-08-2020 11:17 PM - edited 04-08-2020 11:39 PM
Any suggestions what to check on cisco FI level to avoid c3 discards which we see in a few our Cisco UCS 6332 FI's ? 2 pairs runs without any errors and other 2 pairs reporting c3 via snmp. According storage team all configurations are identical on there end.
04-09-2020 04:57 AM - edited 04-09-2020 05:47 PM
Greetings.
Would be helpful to know what kind of ports (i.e. Server, Uplink) and what the other device on the other side of the link is registering.
Is this only specific ports, or is it spread out among most of the ports?
Can you post the output of:
show int eth x/y
show queuing int eth x/y
You can also see some CRC type error counters if something in your environment is trying to transmit jumbo frames and the frames are not tagged with a cos group that allows for the larger frames.
Kirk...
04-09-2020 12:45 PM
We catch c3 discards with some netapp monitoring tool, all discards are seen on vfc interface which is a FCoE port-channel created from 4x40G ports going towards SAN switch. With the same identical setup on other FI pair we don't see such errors at all. No any differences between storage back-end on both pairs.
Attached interface outputs used for a FCoE port-channel towards SAN switch.
04-10-2020 11:58 AM - edited 04-10-2020 12:16 PM
what does the output from
show int count error
show?
Any server facing ports showing any errors?
What is the average utilization, especially for inbound/RX on the UCSM FCOE related ports?
Are these Rack servers or Blade servers? If blade servers, may need to take a look at IOMs and see if we have any drops/tail drops, or lots of pause frames, etc.
Input drops frequently indicates over subscription/congestion.
Kirk...
04-14-2020 04:58 AM
@Kirk J wrote:what does the output from
show int count error
show?
Any server facing ports showing any errors?
What is the average utilization, especially for inbound/RX on the UCSM FCOE related ports?
Are these Rack servers or Blade servers? If blade servers, may need to take a look at IOMs and see if we have any drops/tail drops, or lots of pause frames, etc.
Input drops frequently indicates over subscription/congestion.
Kirk...
I have compared between FIs with no C3 discards and with
sh queuing interface on FCoE ports going towards storage backed switch I see 0 dropped:
Pkts dropped by RX thresholds : 0
Bytes dropped by RX thresholds : 0
on FI with C3 discards:
Pkts dropped by RX thresholds : 22400199
Bytes dropped by RX thresholds : 12439472460
11-12-2020 10:56 PM
issue was QoS System Class count, when doing FcoE with Brocade SAN switch it works without C3 discards when you have 2 Classes defined and with more issues start to appear.
11-13-2020 05:28 AM
7 year old bug. You'd think UCSM would have a fault for this by now instead of sending a SYSLOG which 50% of UCS customers don't configure much less review.
CSCuh72875 :: FCoE multi-hop performance degraded because of QoS config mismatch
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide