11-17-2021 03:23 AM
Hi,
Our management station gives an notification about high output drops on interfaces.
After some debugging I have found the following information but am unsure what to make of it. And if this is an actual issue or just microbust drops.
This is on a Cat 9400 switch.
high amount of OQD on interface 1/0/22:
en02-abh022#show int summary *: interface is up IHQ: pkts in input hold queue IQD: pkts dropped from input queue OHQ: pkts in output hold queue OQD: pkts dropped from output queue RXBS: rx rate (bits/sec) RXPS: rx rate (pkts/sec) TXBS: tx rate (bits/sec) TXPS: tx rate (pkts/sec) TRTL: throttle count Interface IHQ IQD OHQ OQD RXBS RXPS TXBS TXPS TRTL ----------------------------------------------------------------------------------------------------------------- GigabitEthernet1/0/18 0 0 0 0 0 0 0 0 0 GigabitEthernet1/0/19 0 0 0 184456 0 0 0 0 0 * GigabitEthernet1/0/20 0 0 0 77949 213000 46 154000 44 0 GigabitEthernet1/0/21 0 0 0 2701 0 0 0 0 0 * GigabitEthernet1/0/22 0 0 0 86482 68000 13 313000 39 0
No output errors on the interface:
#show interface GigabitEthernet1/0/22
GigabitEthernet1/0/22 is up, line protocol is up (connected)
Hardware is Gigabit Ethernet, address is 5c71.0d16.97f5 (bia 5c71.0d16.97f5)
MTU 9100 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX
input flow-control is off, output flow-control is unsupported
Auto-MDIX on (operational: on)
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:01:28, output 00:00:01, output hang never
Last clearing of "show interface" counters 2w5d
Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 86494
Queueing strategy: Class-based queueing
Output queue: 0/40 (size/max)
5 minute input rate 93000 bits/sec, 33 packets/sec
5 minute output rate 702000 bits/sec, 80 packets/sec
7541536 packets input, 4175452690 bytes, 0 no buffer
Received 13989 broadcasts (11168 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 11168 multicast, 0 pause input
0 input packets with dribble condition detected
13028188 packets output, 10316245654 bytes, 0 underruns
Output 434509 broadcasts (0 multicasts)
0 output errors, 0 collisions, 2 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped outThe service policy outgoing is showing default class drops:
#show policy-map interface GigabitEthernet1/0/22
GigabitEthernet1/0/22
Service-policy output: ACCESS_OUT
<output omitted>
Class-map: BULK-DATA (match-any)
0 packets
Match: dscp af13 (14)
Queueing
(total drops) 0
(bytes output) 0
bandwidth remaining 5%
Class-map: class-default (match-any)
0 packets
Match: any
Queueing
(total drops) 131929956
(bytes output) 9918062185
bandwidth remaining 20%taildropping:
#show platform hard fed active qos queue stats interface GigabitEthernet1/0/22
----------------------------------------------------------------------------------------------
AQM Global counters
GlobalHardLimit: 9500 | GlobalHardBufCount: 0
GlobalSoftLimit: 35476 | GlobalSoftBufCount: 0
----------------------------------------------------------------------------------------------
High Watermark Soft Buffers: Port Monitor Disabled
----------------------------------------------------------------------------------------------
Asic:0 Core:0 DATA Port:42 Hardware Enqueue Counters
----------------------------------------------------------------------------------------------
Q Buffers Enqueue-TH0 Enqueue-TH1 Enqueue-TH2 Qpolicer
(Count) (Bytes) (Bytes) (Bytes) (Bytes)
-- ------- -------------------- -------------------- -------------------- --------------------
0 0 0 316867966 12289408 0
1 0 0 0 14246585 0
2 0 0 0 13536 0
3 0 0 0 8001743 0
4 0 0 0 0 0
5 0 0 0 6732895248 0
6 0 0 0 0 0
7 0 0 0 0 0
Asic:0 Core:0 DATA Port:42 Hardware Drop Counters
--------------------------------------------------------------------------------------------------------------------------------
Q Drop-TH0 Drop-TH1 Drop-TH2 SBufDrop QebDrop QpolicerDrop
(Bytes) (Bytes) (Bytes) (Bytes) (Bytes) (Bytes)
-- -------------------- -------------------- -------------------- -------------------- -------------------- --------------------
0 0 0 0 0 0 0
1 0 0 0 0 0 0
2 0 0 0 0 0 0
3 0 0 0 0 0 0
4 0 0 0 0 0 0
5 0 0 84117614 0 0 0
6 0 0 0 0 0 0
7 0 0 0 0 0 0How do I proceed in the tshoot from here?
11-17-2021 03:41 AM
Hello,
your first action to take with output drops would be to configure:
qos queue-softmax-multiplier 1200
in global configuration mode. Clear the counters and check if the output drops decrease...
11-17-2021 03:43 AM - edited 11-17-2021 03:44 AM
This is confusing me, you're saying your class-default is dropping but I'm seeing class 5 being dropped. Just commenting so I can keep an eye on the thread and learn something.
11-17-2021 06:40 AM
As far as I can see most traffic is added to queue 5:
#show platform hardware fed active qos queue label2qmap qmap-egress-data interface gi 1/0/22
Egress DATA Queue Mapping - Asic/Core/Port: 0/0/42
===============================================================================
Label Q Threshold VQ | Label Q Threshold VQ | Label Q Threshold VQ
===== === ========= === | ===== === ========= === | ===== === ========= ===
0 5 2 2 1 5 2 2 2 5 2 2
3 5 2 2 4 5 2 2 5 5 2 2
6 5 2 2 7 5 2 2 8 5 2 2
9 5 2 2 10 5 2 2 11 5 2 2
12 5 2 2 13 5 2 2 14 5 2 2
15 4 2 2 16 5 2 2 17 5 2 2
18 5 2 2 19 3 2 2 20 5 2 2
21 5 2 2 22 5 2 2 23 5 2 2
24 5 2 2 25 2 2 2 26 5 2 2
27 2 2 2 28 5 2 2 29 2 2 2
30 5 2 2 31 2 2 2 32 5 2 2
33 1 2 2 34 5 2 2 35 1 2 2
36 5 2 2 37 1 2 2 38 5 2 2
39 1 2 2 40 5 2 2 41 5 2 2
42 5 2 2 43 5 2 2 44 5 2 2
45 5 2 2 46 5 2 2 47 0 1 2
48 5 2 2 49 5 2 2 50 5 2 2
51 5 2 2 52 5 2 2 53 5 2 2
54 5 2 2 55 5 2 2 56 5 2 2
57 5 2 2 58 5 2 2 59 5 2 2
60 5 2 2 61 5 2 2 62 5 2 2
63 5 2 2 64 5 2 2 https://people.ucsc.edu/~warner/Bufs/9000-QoS-and-Queuing.pdf
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide