12-07-2011 08:39 PM - edited 03-07-2019 03:47 AM
Hey all,
We are receiving a large amount of output drops on 4 interfaces that connect to microwave wireless.
This is playing up with our Management system. Saw a bug report but that was for 3750, not the IOS version but was a similar issue.
Switch Ports Model SW Version SW Image
------ ----- ----- ---------- ----------
* 1 30 WS-C3560E-24TD 12.2(46)SE C3560E-UNIVERSALK9-M
The output errors tend to happen at the same time. All ports are trunks however there are other trunks on the switch that are not having these issues.
GigabitEthernet0/4 is up, line protocol is up (connected)
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 53717
GigabitEthernet0/5 is up, line protocol is up (connected)
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 53732
GigabitEthernet0/6 is up, line protocol is up (connected)
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 50007
GigabitEthernet0/7 is up, line protocol is up (connected)
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 54484
Gi 0/4 is configured:
switchport trunk encapsulation dot1q
switchport mode trunk
srr-queue bandwidth share 10 10 60 20
queue-set 2
priority-queue out
mls qos trust dscp
auto qos voip trust
We have other switches running a similar config and connected to microwave equipment without these errors.
Looking into the issue we noticed that the actual drop counts in ASIC are not increasing., port 2 (gi0/4) show, the rest similar
RTR1#sh platform port-asic stats drop port 2
Port-asic Port Drop Statistics - Summary
========================================
RxQueue Drop Statistics Slice0
RxQueue 0 Drop Stats Slice0: 0
RxQueue 1 Drop Stats Slice0: 0
RxQueue 2 Drop Stats Slice0: 0
RxQueue 3 Drop Stats Slice0: 0
RxQueue Drop Statistics Slice1
RxQueue 0 Drop Stats Slice1: 0
RxQueue 1 Drop Stats Slice1: 0
RxQueue 2 Drop Stats Slice1: 0
RxQueue 3 Drop Stats Slice1: 0
Port 0 TxQueue Drop Stats: 0
Port 1 TxQueue Drop Stats: 0
Port 2 TxQueue Drop Stats: 0
Port 3 TxQueue Drop Stats: 0
Port 4 TxQueue Drop Stats: 0
Port 5 TxQueue Drop Stats: 0
Port 6 TxQueue Drop Stats: 0
Port 7 TxQueue Drop Stats: 0
Port 8 TxQueue Drop Stats: 0
Port 9 TxQueue Drop Stats: 0
Port 10 TxQueue Drop Stats: 0
Port 11 TxQueue Drop Stats: 0
Port 12 TxQueue Drop Stats: 0
Port 13 TxQueue Drop Stats: 0
Port 14 TxQueue Drop Stats: 0
Port 15 TxQueue Drop Stats: 0
Port 16 TxQueue Drop Stats: 0
Port 17 TxQueue Drop Stats: 0
Port 18 TxQueue Drop Stats: 0
Port 19 TxQueue Drop Stats: 0
Port 20 TxQueue Drop Stats: 0
Port 21 TxQueue Drop Stats: 0
Port 22 TxQueue Drop Stats: 0
Port 23 TxQueue Drop Stats: 0
Port 24 TxQueue Drop Stats: 0
Port 25 TxQueue Drop Stats: 0
Port 26 TxQueue Drop Stats: 0
Port 27 TxQueue Drop Stats: 0
Supervisor TxQueue Drop Statistics
Queue 0: 0
Queue 1: 0
Queue 2: 0
Queue 3: 0
Queue 4: 0
Queue 5: 0
Queue 6: 0
Queue 7: 0
Queue 8: 0
Queue 9: 0
Queue 10: 0
Queue 11: 0
Queue 12: 0
Queue 13: 0
Queue 14: 0
Queue 15: 0
Port-asic Port Drop Statistics - Details
========================================
RxQueue Drop Statistics Slice0
Queue 0 DropThreshold 0 0 DropThreshold 1 0 DropThreshold 2 0
Queue 1 DropThreshold 0 0 DropThreshold 1 0 DropThreshold 2 0
Queue 2 DropThreshold 0 0 DropThreshold 1 0 DropThreshold 2 0
Queue 3 DropThreshold 0 0 DropThreshold 1 0 DropThreshold 2 0
RxQueue Drop Statistics Slice1
Queue 0
DropThreshold 0 0
DropThreshold 1 0
DropThreshold 2 0
Queue 1
DropThreshold 0 0
DropThreshold 1 0
DropThreshold 2 0
Queue 2
DropThreshold 0 0
DropThreshold 1 0
DropThreshold 2 0
Queue 3
DropThreshold 0 0
DropThreshold 1 0
DropThreshold 2 0
Port 2 TxQueue Drop Statistics
Queue 0
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 1
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 2
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 3
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 4
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 5
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 6
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 7
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Supervisor TxQueue Drop Statistics
Queue 0: 0
Queue 1: 0
Queue 2: 0
Queue 3: 0
Queue 4: 0
Queue 5: 0
Queue 6: 0
Queue 7: 0
Queue 8: 0
Queue 9: 0
Queue 10: 0
Queue 11: 0
Queue 12: 0
Queue 13: 0
Queue 14: 0
Queue 15: 0
Any help would be appreciated.
Solved! Go to Solution.
12-20-2011 07:14 PM
Hi,
It seems you oversubscribe queue 3. Port Asic command showing that:
Port 2 TxQueue Drop Statistics
Queue 0
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 1
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 13555
Queue 2
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 3
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 49495569
As per your config queue 2 has much bigger share:
srr-queue bandwidth share 10 10 60 20
You can check the following commands to see which DSCP-QOS are matching that queue and possibly then adjust it:
sho mls qos int gi0/4 stat
show mls qos maps dscp-cos
sh mls qos maps cos-output-q
Nik
12-07-2011 09:44 PM
Hello,
Ports can experience bursty traffic and drop due to no buffers. Please attach following logs:
show ver
show buffer
show platform pm if-number
sh controllers ethernet-controller port-asic statistics
for all affected ports:
show int gix/y switching
show queueing int gix/y
sho controller ethernet gix/Y
show int gix/y counter
Nik
12-07-2011 10:22 PM
12-07-2011 10:24 PM
12-13-2011 06:19 AM
Thanks,
Can you please also capture this command:
sh platform port-asic stats drop asic 1 Port 2
Please also try this command once more "show int gix/y switching". Don't expect that to come out with question mark - that is hidden. Just copy paste as is.
Thanks
Nik
12-14-2011 07:25 PM
12-20-2011 07:14 PM
Hi,
It seems you oversubscribe queue 3. Port Asic command showing that:
Port 2 TxQueue Drop Statistics
Queue 0
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 1
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 13555
Queue 2
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 3
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 49495569
As per your config queue 2 has much bigger share:
srr-queue bandwidth share 10 10 60 20
You can check the following commands to see which DSCP-QOS are matching that queue and possibly then adjust it:
sho mls qos int gi0/4 stat
show mls qos maps dscp-cos
sh mls qos maps cos-output-q
Nik
01-04-2012 01:34 PM
Thanks for your assistance Nik.
It was troubling me what was using the links and causing congestion, as they are not typically heavily utilised, especially at night. I setup a packet capture and noticed that there was a large amount of traffic going down the trunk that should not have been.
Seems we ended up having an issue with incorrect vtp and there were incorrect vlans on the remote site.
After removing the vlans, the problem has been resolved.
On the plus side I learnt a lot about QoS.
Cheers
Chris
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide