Hello folks,
I've got an issue with a new Catalyst 9300 deployment for a customer. We've moved routing from a dedicated router to the 9300 and the 9300 has taken the QoS for the site. We're using standard Auto QoS, trusting DSCP, but I'm seeing significant output drops on the WAN bound interface. Can anyone help shed some light on to what is going on? The stats don't seem to add up correctly on the port-map. The WAN providers interface is nailed to 100/Full and the Auto QoS is set to run in percentages of the interface and has dutifully set it's figures to a 100 Mbps reference bandwidth. I've upgraded the switches from 16.09.01 to 16.12.01, but that didn't clear the problem. I can't quite understand from the show policy-map what the drops in the priority-queue 1 are from, as it doesn't seem to be matching anything and there's no traffic on the site marked that should hit the priority-queue.
Here is a show interface:-
GigabitEthernet1/0/1 is up, line protocol is up (connected)
Hardware is Gigabit Ethernet, address is c4b3.6a05.7c81 (bia c4b3.6a05.7c81)
Description: Network::WAN
MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,
reliability 255/255, txload 2/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Carrier delay is 0 msec
Full-duplex, 100Mb/s, media type is 10/100/1000BaseTX
input flow-control is on, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:01, output 00:00:08, output hang never
Last clearing of "show interface" counters never
Input queue: 0/200000/0/0 (size/max/drops/flushes); Total output drops: 35615
Queueing strategy: Class-based queueing
Output queue: 0/200000 (size/max)
30 second input rate 34000 bits/sec, 34 packets/sec
30 second output rate 966000 bits/sec, 102 packets/sec
120258 packets input, 66898060 bytes, 0 no buffer
Received 2703 broadcasts (2702 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 2702 multicast, 0 pause input
0 input packets with dribble condition detected
285906 packets output, 310294634 bytes, 0 underruns
0 output errors, 0 collisions, 2 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out
Show policy-map int g1/0/1:-
GigabitEthernet1/0/1
Service-policy input: AutoQos-4.0-Trust-Dscp-Input-Policy
Class-map: class-default (match-any)
291057 packets
Match: any
QoS Set
dscp dscp table AutoQos-4.0-Trust-Dscp-Table
Service-policy output: AutoQos-4.0-Output-Policy
queue stats for all priority classes:
Queueing
priority level 1
(total drops) 50316915
(bytes output) 52157956
Class-map: AutoQos-4.0-Output-Priority-Queue (match-any)
0 packets
Match: dscp cs4 (32) cs5 (40) ef (46)
Match: cos 5
Priority: 30% (30000 kbps), burst bytes 750000,
Priority Level: 1
Class-map: AutoQos-4.0-Output-Control-Mgmt-Queue (match-any)
0 packets
Match: dscp cs2 (16) cs3 (24) cs6 (48) cs7 (56)
Match: cos 3
Queueing
queue-limit dscp 16 percent 80
queue-limit dscp 24 percent 90
queue-limit dscp 48 percent 100
queue-limit dscp 56 percent 100
(total drops) 0
(bytes output) 915388
bandwidth remaining 10%
queue-buffers ratio 10
Class-map: AutoQos-4.0-Output-Multimedia-Conf-Queue (match-any)
0 packets
Match: dscp af41 (34) af42 (36) af43 (38)
Match: cos 4
Queueing
(total drops) 0
(bytes output) 303
bandwidth remaining 10%
queue-buffers ratio 10
Class-map: AutoQos-4.0-Output-Trans-Data-Queue (match-any)
0 packets
Match: dscp af21 (18) af22 (20) af23 (22)
Match: cos 2
Queueing
(total drops) 0
(bytes output) 0
bandwidth remaining 10%
queue-buffers ratio 10
Class-map: AutoQos-4.0-Output-Bulk-Data-Queue (match-any)
0 packets
Match: dscp af11 (10) af12 (12) af13 (14)
Match: cos 1
Queueing
(total drops) 0
(bytes output) 0
bandwidth remaining 4%
queue-buffers ratio 10
Class-map: AutoQos-4.0-Output-Scavenger-Queue (match-any)
0 packets
Match: dscp cs1 (8)
Queueing
(total drops) 0
(bytes output) 0
bandwidth remaining 1%
queue-buffers ratio 10
Class-map: AutoQos-4.0-Output-Multimedia-Strm-Queue (match-any)
0 packets
Match: dscp af31 (26) af32 (28) af33 (30)
Queueing
(total drops) 0
(bytes output) 0
bandwidth remaining 10%
queue-buffers ratio 10
Class-map: class-default (match-any)
0 packets
Match: any
Queueing
(total drops) 0
(bytes output) 259417210
bandwidth remaining 25%
queue-buffers ratio 25
Show run int g1/0/1:-
interface GigabitEthernet1/0/1
description Network::WAN
switchport access vlan 900
switchport trunk native vlan 999
switchport mode access
switchport nonegotiate
bandwidth 100000
bandwidth qos-reference 100000
ip flow monitor FNF-monitor-SolarWinds-in input
ip flow monitor FNF-monitor-SolarWinds-out output
ip arp inspection trust
load-interval 30
carrier-delay msec 0
speed 100
duplex full
snmp ifindex persist
storm-control broadcast level 50.00
storm-control multicast level 50.00
storm-control action trap
auto qos trust dscp
spanning-tree portfast trunk
service-policy input AutoQos-4.0-Trust-Dscp-Input-Policy
service-policy output AutoQos-4.0-Output-Policy
hold-queue 200000 in
hold-queue 200000 out
ip dhcp snooping trust
end
Any insight or thoughts, greatly appreciated.
Best, Leigh
