2960x Total output drops and output buffer failures

[Evgenii Boniakevich]

Hi!

I have 2 stacked WS-C2960X-48TS-L 15.2(7)E1. 

And there are many interfaces with "Total output drops" :

GigabitEthernet1/0/10 is up, line protocol is up (connected) 
  Hardware is Gigabit Ethernet, address is a4b2.392c.4f8a (bia a4b2.392c.4f8a)
  MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec, 
     reliability 255/255, txload 8/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX
  input flow-control is off, output flow-control is unsupported 
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input never, output 00:00:00, output hang never
  Last clearing of "show interface" counters 4d06h
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 3582434
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 110000 bits/sec, 126 packets/sec
  5 minute output rate 31658000 bits/sec, 3126 packets/sec
     393083385 packets input, 423203301438 bytes, 0 no buffer
     Received 85827 broadcasts (72266 multicasts)
     0 runts, 0 giants, 0 throttles 
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 72266 multicast, 0 pause input
     0 input packets with dribble condition detected
     1956456254 packets output, 2659504553824 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 pause output
     0 output buffer failures, 0 output buffers swapped out

SW-EST-SRV-01#show int gigabitEthernet 1/0/10 counters errors            

Port        Align-Err     FCS-Err    Xmit-Err     Rcv-Err  UnderSize  OutDiscards 
Gi1/0/10            0           0           0           0          0      3582434 

Port      Single-Col  Multi-Col   Late-Col  Excess-Col  Carri-Sen      Runts     Giants 
Gi1/0/10           0          0          0           0          0          0          0

This interfaces are connected to Windows servers and Hyper-V virtual machines. 

According to https://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-switches/12027-53.html#shwop its happening because the output queue sometimes is full. 

There is also said:

"The port starts to drop the packets when the Tx buffer is full and thus increases the underruns and the output buffer failure counters. The increase in the output buffer failure counters can be a sign that the ports are run at an inferior speed and/or duplex, or there is too much traffic that goes through the port."

But i haven't "output buffer failure" and "underruns". Is this mean that there are another reasons of "Total output drops"?

Thank you!

[Joseph W. Doherty]

". . . or there is too much traffic that goes through the port."

That's the most common cause.  Often a problem on the Catalyst (at least on the older) 3K and 2K switches (the latter have even less buffer resources than the former), especially if QoS is enabled and running with its default configuration.

Your 2960X is often not a good choice for "busy" server ports.  Best for "typical" user hosts.

Sometimes you can "tweak" the config and/or arrange ports to mitigate this kind of issue.  Unsure about your 2K switch, but on, for example, the 3560/3750 physical buffer RAM (the same amount?) is provided to banks of 24 copper ports and to all the (2 or 4) uplink ports.  (I recall [?] the 2K switches only have one bank for RAM for all the ports, which also might have only been 2k.)

[Leo Laohoo]

On top of what @Joseph W. Doherty has to say, what happens if the link speed gets dropped to 100 Mbps, does the Total Output Drops increment?

[Evgenii Boniakevich]

Thank you for replies!

Leo Laohoo, yes, Total Output Drops increase anyway

Joseph W. Doherty, yes, some ports are connected to backup and file servers and they can be over utilized.

But some of interfaces are connected to Supermicro IPMI, IP KVM and LAN USB HUBs. They are all 100 Mbps, but there can't be too much traffic through the port. And they drops packets every 2-3 min. What's strange, total output drops on such interfaces are almost the same - about 500000 packets after last cleaning. And two different ports connected to different servers IPMIs has exactlly the same drops - 525598.

QOS disabled. All ports are full duplex.

Could there be any other reason other than full port unilization?

[Joseph W. Doherty]

"Could there be any other reason other than full port unilization?"

Yes, might be some kind of bug.

Also might be some kind of traffic storm, such as broadcast and/or multicast that could flood your low usage ports.