Output error Cisco C1000

[jamesdevinnobles]

Good day, I have a cisco C1000 switch, and it is showing output errors. There are no other errors on the interface. this is happening on several interfaces. I was thinking the buffers were overflowing but there are no errors on output buffer failures.

Also QOS is not enabled on this switch

MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:10, output 00:00:00, output hang never
Last clearing of "show interface" counters 4d16h
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 120137
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 54000 bits/sec, 14 packets/sec
5 minute output rate 35000 bits/sec, 26 packets/sec
98079045 packets input, 3905032892 bytes, 0 no buffer
Received 23915 broadcasts (13530 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 13530 multicast, 0 pause input
0 input packets with dribble condition detected
59530207 packets output, 1300128913 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out

Any help would be greatly appreciated.

[Flavio Miranda]

@jamesdevinnobles 

 I dont see output errors but I do see output drops. what is connected to this switch?

"Output drops can also be caused by a mismatch between the speed of the interface and the speed of the connected device. For example, if a 10 Gbps interface is connected to a device that only supports a data transfer rate of 1 Gbps, the interface can drop packets if it is unable to slow down the data rate to match the capabilities of the device."

[Joseph W. Doherty]

Like @Flavio Miranda , I see output drops, which is most often caused by overflowing logical queue limits during microbursts (insufficient information to say whether that is the case for you).

It appears the C1000 switch is still very much based on 2960 architecture, with some improvements.

Unclear whether prior 2960 configure and show commands all retain for mitigation of such an issue, which, I believe, can only be done with QoS enabled.  (NB: the default QoS configuration will often drop even more than the non-QoS configuration, but a custom QoS configuration can be adjusted to, often, minimize drops.)

[jamesdevinnobles]

The attached device is a NAS with a 1 gig NIC card. Although this isn't the only port seeing the issue. i am also seeing the output drops on my trunk links between switches all switches are c1000. Also the amount of people that access the NAS is less than 15 people a day and i have 10 total switches onsite.

[Flavio Miranda]

I would clear the counters as this one is 4d16h and observe. Is there any issue going on or are you just investigating ?

[jamesdevinnobles]

A little of both, there was a complaint of slowness. The NAS was rebooted, and it seems to have gone away for now. I am just doing due diligence and to have a better understanding of these drops.

[Joseph W. Doherty]

NAS host and inter switch interfaces are just the kind of interfaces where you often see output drops.

Years ago, had a pair of iSCSI hosts on 3750G gig interfaces.  They were showing multiple drops per second until I adjusted logical WTD limits.  After doing that, drops fell to a few per day.

The C1000 doesn't really have hardware resources to deal with busy hosts, like NAS.

[Joseph W. Doherty]

FYI - I was just skimming the IOS command reference manual for QoS.  From what I see, "under the hood", architecture appears to be very similar to 2960 series, but advanced QoS configuration commands not available.  I.e. Might be unable to mitigate your drops via configuration changes.

[Joseph W. Doherty]

For the interfaces with high drops, try the command:

show mls qos interface interface-id statistics

(It might not work without QoS enabled, which I don't recommend [at this time]).

[jamesdevinnobles]

GigabitEthernet1/0/10 (All statistics are in packets)

Queue : Egress Packets/Threshold id
----------------------------------

queue: threshold1 threshold2
-----------------------------------

Queue[1] 0 0

Queue[2] 0 2406305

Queue[3] 0 0

Queue[4] 0 1778099541

Queue : Egress PacketsDroped/Threshold id
----------------------------------

queue: threshold1 threshold2
-----------------------------------

Queue[1] 0 0

Queue[2] 0 0

Queue[3] 0 0

Queue[4] 0 240670

Policer: Inprofile: 0 OutofProfile: 0

[Joseph W. Doherty]

What's surprising, with QoS disabled, two queues show hits.

What's not surprising, queue showing drops shows many packets being enqueued.

BTW, what IOS version is running?

[jamesdevinnobles]

15.2(7)E1

[Joseph W. Doherty]

---

@jamesdevinnobles wrote:

15.2(7)E1

---

I don't believe upgrading would impact your issue, but personally, you might consider moving to the current E11, if you can.  Release notes do describe the bug fixes over the years.

An interesting note in the QoS Limitations section is:

• Some switch queues are disabled if the buffer size or threshold level is set too low with the mls qos queue-set output global configuration command. The ratio of buffer size to threshold level should be greater than 10 to avoid disabling the queue.

  The workaround is to choose compatible buffer sizes and threshold levels. (CSCea76893)

Interesting because mls qos queue-set output global configuration command is not documented in the command reference, which is the command used to adjust WTD settings.