Re: Troubleshooting network Out Discards errors

Beaurr · ‎02-27-2023

Hello,

I have some users who complain about slowness in their outlook client and sometimes elsewhere.

I'm not sure it's from the network but I'm asked to watch.

In fact, I rather think of slowness on outlook (not really elsewhere) but users tend to exaggerate things

We have a new antivirus and I feel like it's since we installed it on our mail servers (but I don't manage that part). But my boss asks me to check the network

When I do a show interface counters errors, I see High Number of Out Discards

S1#sh interfaces counters errors

Port Align-Err FCS-Err Xmit-Err Rcv-Err UnderSize OutDiscards
Gi1/0/1 0 0 0 0 0 0
Gi1/0/2 0 0 0 0 0 0
Gi1/0/3 0 0 0 0 0 0
Gi1/0/4 0 0 0 0 0 0
Gi1/0/5 0 0 0 0 0 0
Gi1/0/6 0 0 0 0 0 0
Gi1/0/7 0 0 0 0 0 0
Gi1/0/8 0 0 0 0 0 0
Gi1/0/9 0 0 0 0 0 1473
Gi1/0/10 0 0 0 0 0 0
Gi1/0/11 0 0 0 0 0 0
Gi1/0/12 0 0 0 0 0 0
Gi1/0/13 0 0 0 0 0 0
Gi1/0/14 0 0 0 0 0 0
Gi1/0/15 0 0 0 0 0 0
Gi1/0/16 0 0 0 0 0 0
Gi1/0/17 0 0 0 0 0 0
Gi1/0/18 0 0 0 0 0 0
Gi1/0/19 0 0 0 0 0 0
Gi1/0/20 0 0 0 0 0 0
Gi1/0/21 0 0 0 0 0 0
Gi1/0/22 0 0 0 0 0 0
Gi1/0/23 0 0 0 0 0 0
Gi1/0/24 0 0 0 0 0 0
Gi1/0/25 0 0 0 0 0 0
Gi1/0/26 0 0 0 0 0 0
Gi1/0/27 0 0 0 0 0 0
Gi1/0/28 0 0 0 0 0 0
Gi1/0/29 0 0 0 0 0 0
Gi1/0/30 0 0 0 0 0 0
Gi1/0/31 0 0 0 0 0 0
Gi1/0/32 0 0 0 0 0 0
Gi1/0/33 0 0 0 0 0 0
Gi1/0/34 0 0 0 0 0 0
Gi1/0/35 0 0 0 0 0 12
Gi1/0/36 0 0 0 0 0 0
Gi1/0/37 0 0 0 0 0 0
Gi1/0/38 0 0 0 0 0 0
Gi1/0/39 0 0 0 0 0 2435
Gi1/0/40 0 0 0 0 0 0
Gi1/0/41 0 0 0 0 0 0
Gi1/0/42 0 0 0 0 0 0
Gi1/0/43 0 0 0 0 0 0
Gi1/0/44 0 0 0 0 0 0
Gi1/0/45 0 0 0 0 0 0
Gi1/0/46 0 0 0 0 0 0
Gi1/0/47 0 0 0 0 0 0
Gi1/0/48 0 0 0 0 0 0
Gi1/0/49 0 0 0 0 0 0
Gi1/0/50 0 0 0 0 0 0
Gi2/0/1 0 0 0 0 0 0
Gi2/0/2 0 0 0 0 0 0
Gi2/0/3 0 0 0 0 0 0
Gi2/0/4 0 0 0 0 0 0
Gi2/0/5 0 0 0 0 0 0
Gi2/0/6 0 0 0 0 0 0
Gi2/0/7 0 0 0 0 0 0
Gi2/0/8 0 0 0 0 0 0
Gi2/0/9 0 0 0 0 0 0
Gi2/0/10 0 0 0 0 0 0
Gi2/0/11 0 0 0 0 0 0
Gi2/0/12 0 0 0 0 0 0
Gi2/0/13 0 0 0 0 0 0
Gi2/0/14 0 0 0 0 0 0
Gi2/0/15 0 0 0 0 0 0
Gi2/0/16 0 0 0 0 0 0
Gi2/0/17 0 0 0 0 0 2440
Gi2/0/18 0 0 0 0 0 0
Gi2/0/19 0 0 0 0 0 0
Gi2/0/20 0 0 0 0 0 0
Gi2/0/21 0 0 0 0 0 0
Gi2/0/22 0 0 0 0 0 0
Gi2/0/23 0 0 0 0 0 2338
Gi2/0/24 0 0 0 0 0 0
Gi2/0/25 0 0 0 0 0 0
Gi2/0/26 0 0 0 0 0 0
Gi2/0/27 0 0 0 0 0 0
Gi2/0/28 0 0 0 0 0 0
Gi2/0/29 0 0 0 0 0 0
Gi2/0/30 0 0 0 0 0 0
Gi2/0/31 0 0 0 0 0 0
Gi2/0/32 0 0 0 0 0 0
Gi2/0/33 0 0 0 0 0 0
Gi2/0/34 0 0 0 0 0 0
Gi2/0/35 0 0 0 0 0 0
Gi2/0/36 0 0 0 0 0 0
Gi2/0/37 0 0 0 0 0 0
Gi2/0/38 0 0 0 0 0 0
Gi2/0/39 0 0 0 0 0 0
Gi2/0/40 0 0 0 0 0 0
Gi2/0/41 0 0 0 0 0 0
Gi2/0/42 0 0 0 0 0 0
Gi2/0/43 0 0 0 0 0 0
Gi2/0/44 0 0 0 0 0 0
Gi2/0/45 0 0 0 0 0 0
Gi2/0/46 0 0 0 0 0 0
Gi2/0/47 0 0 0 0 0 0
Gi2/0/48 0 0 0 0 0 0
Gi2/0/49 0 0 0 0 0 0
Te2/0/2 0 0 0 0 0 0
Gi3/0/1 0 0 0 0 0 0
Gi3/0/2 0 0 0 0 0 0
Gi3/0/3 0 0 0 0 0 0
Gi3/0/4 0 0 0 0 0 0
Gi3/0/5 0 0 0 0 0 0
Gi3/0/6 0 0 0 0 0 0
Gi3/0/7 0 0 0 0 0 0
Gi3/0/8 0 0 0 0 0 0
Gi3/0/9 0 0 0 0 0 0
Gi3/0/10 0 0 0 0 0 0
Gi3/0/11 0 0 0 0 0 0
Gi3/0/12 0 0 0 0 0 0
Gi3/0/13 0 0 0 0 0 0
Gi3/0/14 0 0 0 0 0 0
Gi3/0/15 0 0 0 0 0 0
Gi3/0/16 0 0 0 0 0 0
Gi3/0/17 0 0 0 0 0 0
Gi3/0/18 0 0 0 0 0 0
Gi3/0/19 0 0 0 0 0 0
Gi3/0/20 0 0 0 0 0 0
Gi3/0/21 0 0 0 0 0 0
Gi3/0/22 0 0 0 0 0 0
Gi3/0/23 0 0 0 0 0 0
Gi3/0/24 0 0 0 0 0 0
Gi3/0/25 0 0 0 0 0 2402
Gi3/0/26 0 0 0 0 0 0
Gi3/0/27 0 0 0 0 0 0
Gi3/0/28 0 0 0 0 0 0
Gi3/0/29 0 0 0 0 0 0
Gi3/0/30 0 0 0 0 0 0
Gi3/0/31 0 0 0 0 0 0
Gi3/0/32 0 0 0 0 0 0
Gi3/0/33 0 0 0 0 0 0
Gi3/0/34 0 0 0 0 0 0
Gi3/0/35 0 0 0 0 0 0
Gi3/0/36 0 0 0 0 0 0
Gi3/0/37 0 0 0 0 0 0
Gi3/0/38 0 0 0 0 0 0
Gi3/0/39 0 0 0 0 0 2439

....

Port Single-Col Multi-Col Late-Col Excess-Col Carri-Sen Runts Giants
Gi1/0/1 0 0 0 0 0 0 0
Gi1/0/2 0 0 0 0 0 0 0
Gi1/0/3 0 0 0 0 0 0 0
Gi1/0/4 0 0 0 0 0 0 0
Gi1/0/5 0 0 0 0 0 0 0
Gi1/0/6 0 0 0 0 0 0 0
Gi1/0/7 0 0 0 0 0 0 0
Gi1/0/8 0 0 0 0 0 0 0
Gi1/0/9 0 0 0 0 0 0 0
Gi1/0/10 0 0 0 0 0 0 0
Gi1/0/11 0 0 0 0 0 0 0
Gi1/0/12 0 0 0 0 0 0 0
Gi1/0/13 0 0 0 0 0 0 0
Gi1/0/14 0 0 0 0 0 0 0
Gi1/0/15 0 0 0 0 0 0 0
Gi1/0/16 0 0 0 0 0 0 0
Gi1/0/17 0 0 0 0 0 0 0
Gi1/0/18 0 0 0 0 0 0 0
Gi1/0/19 0 0 0 0 0 0 0
Gi1/0/20 0 0 0 0 0 0 0
Gi1/0/21 0 0 0 0 0 0 0
Gi1/0/22 0 0 0 0 0 0 0
Gi1/0/23 0 0 0 0 0 0 0
Gi1/0/24 0 0 0 0 0 0 0
Gi1/0/25 0 0 0 0 0 0 0
Gi1/0/26 0 0 0 0 0 0 0
Gi1/0/27 0 0 0 0 0 0 0
Gi1/0/28 0 0 0 0 0 0 0
Gi1/0/29 0 0 0 0 0 0 0
Gi1/0/30 0 0 0 0 0 0 0
Gi1/0/31 0 0 0 0 0 0 0
Gi1/0/32 0 0 0 0 0 0 0
Gi1/0/33 0 0 0 0 0 0 0
Gi1/0/34 0 0 0 0 0 0 0
Gi1/0/35 0 0 0 0 0 0 0
Gi1/0/36 0 0 0 0 0 0 0
Gi1/0/37 0 0 0 0 0 0 0
Gi1/0/38 0 0 0 0 0 0 0
Gi1/0/39 0 0 0 0 0 0 0
Gi1/0/40 0 0 0 0 0 0 0
Gi1/0/41 0 0 0 0 0 0 0
Gi1/0/42 0 0 0 0 0 0 0
Gi1/0/43 0 0 0 0 0 0 0
Gi1/0/44 0 0 0 0 0 0 0
Gi1/0/45 0 0 0 0 0 0 0
Gi1/0/46 0 0 0 0 0 0 0
Gi1/0/47 0 0 0 0 0 0 0
Gi1/0/48 0 0 0 0 0 0 0

I did a clear counters 2 hours before. And we can already see the outdiscards increasing on certain ports.

most connected equipment is gigabit

Port configuration example :

interface GigabitEthernet5/0/31
description Info
switchport access vlan 210
switchport mode access
srr-queue bandwidth share 1 30 35 5
priority-queue out
mls qos trust cos
auto qos trust
spanning-tree portfast

I specify that I have other ports, which have exactly the same configuration, with a pc connected to it, up, and which have no outdiscards errors.

S1#sh int Gi5/0/31
GigabitEthernet5/0/31 is up, line protocol is up (connected)
Hardware is Gigabit Ethernet, address is 2c73.a097.809f (bia 2c73.a097.809f)
Description: Poste Info
MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, media type is 10/100/1000BaseTX
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:11:59, output 00:00:24, output hang never
Last clearing of "show interface" counters 01:21:57
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 6259
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 84000 bits/sec, 32 packets/sec
319174 packets input, 50254464 bytes, 0 no buffer
Received 152 broadcasts (151 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 151 multicast, 0 pause input
0 input packets with dribble condition detected
784475 packets output, 963445118 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out

S1#sh running-config | i mls
mls qos map cos-dscp 0 8 16 24 32 46 48 56
mls qos srr-queue output cos-map queue 1 threshold 3 4 5
mls qos srr-queue output cos-map queue 2 threshold 1 2
mls qos srr-queue output cos-map queue 2 threshold 2 3
mls qos srr-queue output cos-map queue 2 threshold 3 6 7
mls qos srr-queue output cos-map queue 3 threshold 3 0
mls qos srr-queue output cos-map queue 4 threshold 3 1
mls qos srr-queue output dscp-map queue 1 threshold 3 32 33 40 41 42 43 44 45
mls qos srr-queue output dscp-map queue 1 threshold 3 46 47
mls qos srr-queue output dscp-map queue 2 threshold 1 16 17 18 19 20 21 22 23
mls qos srr-queue output dscp-map queue 2 threshold 1 26 27 28 29 30 31 34 35
mls qos srr-queue output dscp-map queue 2 threshold 1 36 37 38 39
mls qos srr-queue output dscp-map queue 2 threshold 2 24
mls qos srr-queue output dscp-map queue 2 threshold 3 48 49 50 51 52 53 54 55
mls qos srr-queue output dscp-map queue 2 threshold 3 56 57 58 59 60 61 62 63
mls qos srr-queue output dscp-map queue 3 threshold 3 0 1 2 3 4 5 6 7
mls qos srr-queue output dscp-map queue 4 threshold 1 8 9 11 13 15
mls qos srr-queue output dscp-map queue 4 threshold 2 10 12 14
mls qos queue-set output 1 threshold 1 100 100 50 200
mls qos queue-set output 1 threshold 2 125 125 100 400
mls qos queue-set output 1 threshold 3 100 100 100 400
mls qos queue-set output 1 threshold 4 60 150 50 200
mls qos queue-set output 1 buffers 15 25 40 20
mls qos

PCs and phones are in different vlans and in most cases on different ports with a few exceptions

I didn't configure the ports like that. I do not really master the good practices of the QOS, therefore, I left these configurations

For these outdiscards, are there any other commands I can check my network health?

Thanks

MHM Cisco World · ‎02-27-2023

sh platform hardware fed switch x qos queue config interface x/x <<- please share this

Beaurr · ‎02-28-2023

@MHM Cisco World wrote:

S1#sh platform hardware fed switch x qos queue config interface Gi5/0/31
^
% Invalid input detected at '^' marker.

S1#sh platform ?
acl Access Control List (ACL) Manager Information
authentication Show switch auth info
backup Show Platform Backup
configuration show config manager related info
dl Show Dynamically loaded modules Info
dot1x Dot1X
dtm Drop Table Manager show commands
etherchannel Platform EtherChannel information
fnf fnf platform commands
forward Show forwarding info
frontend-controller Show frontend controller info
host-access-table host access table info
hulc-fnf fnf platform commands
ip platform ip specific show commands
ipc Show Platform IPC Info
ipv6 platform ipv6 specific show commands
layer4op Layer4 operators Information
mac-address-table show platform mac address table Info
messaging show information regarding messages
monitor platform monitor info
mvlan Control Plane Policing show command
mvr MVR Details
phy Show platform phy functionality
pm Show Platform Port Manager Info
port-asic Show port-asic data structures
port-security Show platform port-security information
qos Qos Mgr platform commands
resource-manager Resource Manager
snmp Show platform SNMP info
spanning-tree Show platform spanning tree info
stack Show data related to stack behavior/status
status Show Platform Status Info
stm SRAM/TCAM Manager
stp-instance Show STP instance
tb Show Platform Trust Boundary Info
tcam Show TCAM Driver
vlan VLAN module information

Switch Ports Model SW Version SW Image
------ ----- ----- ---------- ----------
* 1 52 WS-C2960X-48LPD-L 15.2(2)E7 C2960X-UNIVERSALK9-M
2 52 WS-C2960X-48FPD-L 15.2(2)E7 C2960X-UNIVERSALK9-M
3 52 WS-C2960X-48FPD-L 15.2(2)E7 C2960X-UNIVERSALK9-M
4 52 WS-C2960X-48FPD-L 15.2(2)E7 C2960X-UNIVERSALK9-M
5 52 WS-C2960X-48FPD-L 15.2(2)E7 C2960X-UNIVERSALK9-M

Joseph W. Doherty · ‎02-27-2023

"When I do a show interface counters errors, I see High Number of Out Discards"

BTW, "high", if you don't know, is not considered by its absolute value, but by its relative value. I.e. what's the percentage of drops to outputted packets.

The old rule of thumb was 1% or less was okay. Personally, for interactive traffic, and especially real-time traffic, like VoIP, I start to be concerned when its more that 0.1%, sometimes even less is an issue. BTW, although zero drops looks nice, it might not be the ideal either (laugh, if fact, in some cases, zero drops can indicate non-optimal usage).

"For these outdiscards, are there any other commands I can check my network health?"

Very possibly, but it would help much if you also describe the exact device and its exact software (and licenses?) being used.

That noted, from what you've posted, is this a 2960, 3560 or 3750 switch?

If so, there are commands to see exactly where in the QoS structure drops are happening.

The usual cause of unnecessary drops is WTD 1 and 2 and full queue drops limits, using too small values. Often just increasing those often eliminates many drops.

I.e.

We likely will need to adjust these statements:

mls qos queue-set output 1 threshold 1 100 100 50 200
mls qos queue-set output 1 threshold 2 125 125 100 400
mls qos queue-set output 1 threshold 3 100 100 100 400
mls qos queue-set output 1 threshold 4 60 150 50 200

Beaurr · ‎02-28-2023

Switch Ports Model SW Version SW Image
------ ----- ----- ---------- ----------
* 1 52 WS-C2960X-48LPD-L 15.2(2)E7 C2960X-UNIVERSALK9-M
2 52 WS-C2960X-48FPD-L 15.2(2)E7 C2960X-UNIVERSALK9-M
3 52 WS-C2960X-48FPD-L 15.2(2)E7 C2960X-UNIVERSALK9-M
4 52 WS-C2960X-48FPD-L 15.2(2)E7 C2960X-UNIVERSALK9-M
5 52 WS-C2960X-48FPD-L 15.2(2)E7 C2960X-UNIVERSALK9-M

Joseph W. Doherty · ‎02-28-2023

Here's the Cisco TechNote for troubleshooting 2960(3560/3750) output drops.

Getting more stats about the drops is done with command noted in the TechNote:

show mls qos int <interface> statistics

The prior should provide better details about drops (and queuing congestion).

Often, as this TechNote mentions, and which I earlier did too, adjust the queue limits can mitigate drops (it almost always decreases them).

Interfaces with a really severe situation of dropping packets, often need much more extensive QoS tuning, which I believe this TechNote does not adequately deal with, but I suspect yours will not need that level of attention.

There was a great "document", published on these forums, which really explained queue/buffer tuning for the 2960/3560/3750, at least it was one that I could make sense out of. Cannot find it, might have recently been archived.

Did find: https://community.cisco.com/t5/networking-knowledge-base/increased-total-output-drops-outdiscards-by-qos-for-desktop/ta-p/3156674

Which mentions TAC often recommends:

mls qos queue-set output 1 threshold 1 3200 3200 100 3200
mls qos queue-set output 1 threshold 2 3200 3200 100 3200
mls qos queue-set output 1 threshold 3 3200 3200 100 3200
mls qos queue-set output 1 threshold 4 3200 3200 100 3200

But we can looks at your more detailed stats first.