02-20-2013 07:32 AM - edited 03-04-2019 07:05 PM
On one of our gigabit interfaces we see output drops, the traffic is comming from a TenGig interface. The outgoing gig intface has a 5min avg load of 300-500Mbit. When i set the snmp monitoring to 1 sec avg i see some micro bursts. So output drops could indeed occure. But since the bursts are so small is it possible to solve this? Is QoS the way to manage this or are there faster solutions?
QoS is currently disabled.
When i do a "show buffers" i see some misses but no failures. So buffers/memory is not a issue?
The packets are dropped in Q3 Weight. So Q1 and Q2 have already been used and are full?
sh platform port-asic stats drop gigabitEthernet 0/11
Interface Gi0/11 TxQueue Drop Statistics
Queue 0
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 1
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 2
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 3
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 236111
Reading :
https://supportforums.cisco.com/docs/DOC-8093#_Toc191205668
enlarge the thresholds seems a solution.
I have no idea what the impact is on the switch(switch memory) and the other switch ports.
We have a 3560X and ios 12.2(55)SE3
I hope someone could assist.
02-20-2013 10:23 AM
Disclaimer
The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.
Liability Disclaimer
In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.
Posting
The packets are dropped in Q3 Weight. So Q1 and Q2 have already been used and are full?
If QoS is disabled, there's only one egress per port.
What might be possible with activation of QoS you can tune resources. I.e. you might be able to "borrow" buffer resources reserved for other ports to better deal with microbursts. Of course, this could impact (negatively) performance on other ports.
02-21-2013 02:01 AM
Thanks for you answer
"If QoS is disabled, there's only one egress per port."
This suggests to me otherwise:
sh platform port-asic stats enqueue gigabitEthernet 0/11
Interface Gi0/11 TxQueue Enqueue Statistics
Queue 0
Weight 0 Frames 2
Weight 1 Frames 0
Weight 2 Frames 0
Queue 1
Weight 0 Frames 0
Weight 1 Frames 159529759
Weight 2 Frames 558551
Queue 2
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 3
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 695042334
"I.e. you might be able to "borrow" buffer resources reserved for other ports to better deal with microbursts."
If i have only 5 ports enabled on the switch do i have more resources to play with?
02-21-2013 02:36 AM
Disclaimer
The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.
Liability Disclaimer
In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.
Posting
"If QoS is disabled, there's only one egress per port."This suggests to me otherwise:
sh platform port-asic stats enqueue gigabitEthernet 0/11
Interface Gi0/11 TxQueue Enqueue Statistics
Queue 0
Weight 0 Frames 2
Weight 1 Frames 0
Weight 2 Frames 0
Queue 1
Weight 0 Frames 0
Weight 1 Frames 159529759
Weight 2 Frames 558551
Queue 2
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 3
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 695042334
Interesting! I agree it does. So if you do a show mls qos, it shows disabled? If it does show disabled, QoS was never enabled on this device? (I'm wondering whether stats would "survive" a change in QoS mode.) Otherwise, I'm also wondering (especially because of Q1 Wt1) whether WTD is still in effect, recorded per what would be a queue. If none of these, something to go to TAC with. (Might also be a stats recording bug.)
"I.e. you might be able to "borrow" buffer resources reserved for other ports to better deal with microbursts."If i have only 5 ports enabled on the switch do i have more resources to play with?
Perhaps. What I'm thinking is you much reduce the buffer allocation per port, putting more buffers into the common pool and then hopefully a busy port will draw excess buffers from that pool. (Your original reference has about the best explanation of buffer management I've read.)
What I'm unsure about is whether, for a microburst, a port can draw buffers from the common pool fast enough to not drop packets. If that is an issue, the alternative might be to assign most of the common pool to port reserved buffers. As my understanding is this would apply to all the ports, there would be less buffer resources available per your 5 ports but in a microburst there shouldn't be any delay in obtaining the port reserved buffers.
Also keep in mind, from 10g to gig, some dropping is often normal to indicate to 10g (TCP) flows you don't have 10g of bandwidth. Counter intuitively, increasing buffer (queue) resources can increase the overall drop rate. What you can also do is calculate BDP, divide in half, and take 90% (10g - g) speed difference for the maximum optimal buffer requirement.
02-22-2013 02:08 AM
#sh mls qos
QoS is disabled
QoS ip packet dscp rewrite is enabled
QoS was never enabled on the switch.
If i do a sh platform port-asic stats enqueue without the interface id, and scroll to port 11 i see no enqueued frames. (if port 11 is gi0/11, but only port 0 is showing qeueud frames) Could be indeed a IOS bug. I have no support contract to contact tac....
I do however think the drops on Gi0/11 are real, because they increase like the used bandwidth.
BDP = 10Gb * 2ms (the delay between the 10g switches?) =2500000b /2 and take 90% or do you mean take 10%?
Thanks for your efford.
02-22-2013 02:42 AM
Disclaimer
The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.
Liability Disclaimer
In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.
Posting
You would calculate BDP from the congested interface to the end-point of the traffic. Then divide that value in half and then multiply by .9. That should provide about the maximum buffer space needed on the interface.
Ideally, you would also calculate the BDP between the two hosts, and set the receiving host's TCP RWIN to that value.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide