Re: output drops with interface utilization = 20% !!! in Sw 2960

mohamed louhab · ‎05-20-2013

hi ,

i noticed the "output drops" increasing in interface gi 1/0/1 in my switch 2960-S although the utilization of my interface is between 20% and 50 %

SW__A#sh interfaces gigabitEthernet 1/0/1

GigabitEthernet1/0/1 is up, line protocol is up (connected)

MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,

reliability 255/255, txload 93/255, rxload 3/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX

input flow-control is off, output flow-control is unsupported

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:00, output 00:00:00, output hang never

Last clearing of "show interface" counters 00:10:00

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 78484

Queueing strategy: fifo

Output queue: 0/40 (size/max)

30 second input rate 14786000 bits/sec, 22426 packets/sec

30 second output rate 365630000 bits/sec, 42896 packets/sec

6584758 packets input, 575696417 bytes, 0 no buffer

SW__A#sh controllers gigabitEthernet 1/0/1 utilization

Receive Bandwidth Percentage Utilization : 3

Transmit Bandwidth Percentage Utilization : 50

SW_A#sh interfaces gigabitEthernet 1/0/1 | include Total output drops

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 585722

SW_A#sh controllers gigabitEthernet 1/0/1 utilization

Receive Bandwidth Percentage Utilization : 1

Transmit Bandwidth Percentage Utilization : 24

The IOS version is :

Switch Ports Model SW Version SW Image

------ ----- ----- ---------- ----------

* 1 28 WS-C2960S-24TD-L 12.2(55)SE5 C2960S-UNIVERSALK9-M

what is the cause of this issue ??

thank you

Joseph W. Doherty · ‎05-20-2013

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

Typically transient congestion (or microbursts). Typically not "visible" across even a 30 second load average. Worst case, transient drops will slow traffic such that it brings your effective bandwidth utilization down. I.e. a high drop rate will very much slow rate adaptive flows.

If this is the cause, what can you do? Possible solutions include: providing more bandwidth, providing more buffering resources, better drop management, TCP RWIN tuning.

mohamed louhab · ‎05-20-2013

Hi Joeseph ,

how i can providing more buffer ressources, or drop management ?

paolo bevilacqua · ‎05-20-2013

Is there configured any QoS/MLS on the switch?

mohamed louhab · ‎05-20-2013

Hi Paolo ,

no , the QOS is disable in the Switch. really this issue drive me crazy !!!

thanks

paolo bevilacqua · ‎05-20-2013

Is no QoS or custom queuing is configured, then Joseph explaination is correct and drops are due to micro-bursts.

mohamed louhab · ‎05-20-2013

Hi ,

can you explain more about micro-bursts and how i can solve this issue.

thank you

paolo bevilacqua · ‎05-20-2013

That has been correctly explained by Joseph above already, or you can use the search function for more reading.

mohamed louhab · ‎05-20-2013

Hi ,

thank all for your help .

how i can increase the buffer in the interfaces and or drop management ?

if i enable the QOS perhaps resolve this problem ?

how about a bug in IOS ??

thank you

paolo bevilacqua · ‎05-20-2013

Add another interface for etherchannel.

Joseph W. Doherty · ‎05-20-2013

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

mohamed louhab wrote:

Hi Paolo ,

no , the QOS is disable in the Switch. really this issue drive me crazy !!!

thanks

From one of your later postings, it appears QoS is enabled, otherwise would expect to see different WTD drops for different queues.

So you're sure "mls qos" isn't in your config?

Leo Laohoo · ‎05-20-2013

SW__A#sh controllers gigabitEthernet 1/0/1 utilization

Receive Bandwidth Percentage Utilization : 3

Transmit Bandwidth Percentage Utilization : 50

The command you are using, "sh controller util", takes an on-the-spot snapshot of the utilization of the specific link. In your case 50%.

Basically, this means that the switch is sending 50% of the link's capabilities (about 500 Mbps) to whatever is downstream. However, the downstream device is possibly sending "Pause" frames back. The backed-up packets get put into a buffer or a bucket until it gets full and overflows, hence, you are seeing output drops.

Configure the link to 100 Mbps, command: speed auto 10 100, and observe if output drops occur.

Can you also post the output to the command "sh controller eth Gi 1/0/1", please?

mohamed louhab · ‎05-20-2013

Hi ,

thank you Paolo ,

@leolaohoo , but i have drops in the interfaces although the utilization is 20% !!

we suffer a problem of copy betwen my two sites

that's why after my trubelshhoting I suspect this issue causing this problem.

i don't know if it or not interesting i put this :

SW_A#show platform port-asic stats drop gigabitEthernet 1/0/1

Interface Gi1/0/1 TxQueue Drop Statistics

Queue 0

Weight 0 Frames 0

Weight 1 Frames 0

Weight 2 Frames 0

Queue 1

Weight 0 Frames 1314629

Weight 1 Frames 0

Weight 2 Frames 0

Queue 2

Weight 0 Frames 10

Weight 1 Frames 0

Weight 2 Frames 0

Queue 3

Weight 0 Frames 0

Weight 1 Frames 0

Weight 2 Frames 250824

Queue 4

Weight 0 Frames 0

Weight 1 Frames 0

Weight 2 Frames 0

Queue 5

Weight 0 Frames 0

Weight 1 Frames 0

Weight 2 Frames 0

Queue 6

Weight 0 Frames 0

Weight 1 Frames 0

Weight 2 Frames 0

Queue 7

Weight 0 Frames 0

Weight 1 Frames 0

Weight 2 Frames 0

for the result of "sh controller eth Gi1/0/1" i will send to you tomorrow

thank you

Leo Laohoo · ‎05-20-2013

but i have drops in the interfaces although the utilization is 20% !!

I have, in our network, seen your case of output drops. The fastest way to eliminate WHERE the problem is is to drop the speed from 1 Gbps to 100 Mbps.

Once we've proven to the people who owns and/or manage the servers that output drops from the switch to the server has decrease do the server people start talking seriously (instead of saying, "It's a network fault").

paolo bevilacqua · ‎05-20-2013

Basically, this means that the switch is sending 50% of the link's capabilities (about 500 Mbps) to whatever is downstream. However, the downstream device is possibly sending "Pause" frames back. The backed-up packets get put into a buffer or a bucket until it gets full and overflows, hence, you are seeing output drops.

Even if the other switch is sending pause frames, the 2960S is ignoring them, as the show interface is telling us:

input flow-control is off, output flow-control is unsupported

output drops with interface utilization = 20% !!! in Sw 2960-S