Total output drops- is it a switch issue?

[Loc Nguyen]

i,

We have an issue between Zic and camera on the same switch. It  is Cisco 2960XR.

Below is the detail.

Honestly, it looks normal to me. We have some output drops on the ZIC interfaces but it is very small to me. Could you advise?

ZIC1:

TX-gb-lane-sw#show int g1/0/23

GigabitEthernet1/0/23 is up, line protocol is up (connected)

  Hardware is Gigabit Ethernet, address is 00bf.779f.a817 (bia 00bf.779f.a817)

  MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,

     reliability 255/255, txload 5/255, rxload 6/255

  Encapsulation ARPA, loopback not set

  Keepalive set (10 sec)

  Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX

  input flow-control is off, output flow-control is unsupported

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input never, output 00:00:01, output hang never

  Last clearing of "show interface" counters never

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 17429

 Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 23882000 bits/sec, 3354 packets/sec

  5 minute output rate 20954000 bits/sec, 3196 packets/sec

     14905244402 packets input, 11254112198153 bytes, 0 no buffer

     Received 436924 broadcasts (212198 multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog, 212198 multicast, 0 pause input

     0 input packets with dribble condition detected

     15670673071 packets output, 12699980256885 bytes, 0 underruns

     0 output errors, 0 collisions, 1 interface resets

     0 unknown protocol drops

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier, 0 pause output

     0 output buffer failures, 0 output buffers swapped out

TX-gb-lane-sw#

ZIC 2

TX-gb-lane-sw#show int g2/0/23

GigabitEthernet2/0/23 is up, line protocol is up (connected)

  Hardware is Gigabit Ethernet, address is 00bf.779f.b217 (bia 00bf.779f.b217)

  MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,

     reliability 255/255, txload 5/255, rxload 1/255

  Encapsulation ARPA, loopback not set

  Keepalive set (10 sec)

  Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX

  input flow-control is off, output flow-control is unsupported

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input never, output 00:00:25, output hang never

  Last clearing of "show interface" counters never

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 19462

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 7816000 bits/sec, 2000 packets/sec

  5 minute output rate 21437000 bits/sec, 2762 packets/sec

     10477765106 packets input, 4325453623761 bytes, 0 no buffer

     Received 586359 broadcasts (212148 multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog, 212148 multicast, 0 pause input

     0 input packets with dribble condition detected

     14713862133 packets output, 13495212115127 bytes, 0 underruns

     0 output errors, 0 collisions, 1 interface resets

     0 unknown protocol drops

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier, 0 pause output

     0 output buffer failures, 0 output buffers swapped out

TX-gb-lane-sw#

Camera 1:

TX-gb-lane-sw#show int g1/0/23 stats

GigabitEthernet1/0/23

          Switching path    Pkts In   Chars In   Pkts Out  Chars Out

               Processor          0          0        106       8580

             Route cache          0          0          0          0

                   Total          0          0        106       8580

TX-gb-lane-sw#show int g2/0/23 stats

GigabitEthernet2/0/23

          Switching path    Pkts In   Chars In   Pkts Out  Chars Out

               Processor          0          0          4       2052

             Route cache          0          0          0          0

                   Total          0          0          4       2052

TX-gb-lane-sw#

F1:

TX-gb-lane-sw#show int g1/0/11

GigabitEthernet1/0/11 is up, line protocol is up (connected)

  Hardware is Gigabit Ethernet, address is 00bf.779f.a80b (bia 00bf.779f.a80b)

  MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,

     reliability 255/255, txload 1/255, rxload 1/255

  Encapsulation ARPA, loopback not set

  Keepalive set (10 sec)

  Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX

  input flow-control is off, output flow-control is unsupported

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input never, output 00:00:00, output hang never

  Last clearing of "show interface" counters never

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 454000 bits/sec, 51 packets/sec

  5 minute output rate 10000 bits/sec, 19 packets/sec

     237712641 packets input, 313834113872 bytes, 0 no buffer

     Received 16661 broadcasts (60 multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog, 60 multicast, 0 pause input

     0 input packets with dribble condition detected

     178065519 packets output, 77619947496 bytes, 0 underruns

     0 output errors, 0 collisions, 1 interface resets

     0 unknown protocol drops

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier, 0 pause output

     0 output buffer failures, 0 output buffers swapped out

TX-gb-lane-sw#

Camera 2:

TX-gb-lane-sw#show int g1/0/36

GigabitEthernet1/0/36 is up, line protocol is up (connected)

  Hardware is Gigabit Ethernet, address is 00bf.779f.a824 (bia 00bf.779f.a824)

  MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,

     reliability 255/255, txload 1/255, rxload 1/255

  Encapsulation ARPA, loopback not set

  Keepalive set (10 sec)

  Full-duplex, 100Mb/s, media type is 10/100/1000BaseTX

  input flow-control is off, output flow-control is unsupported

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input never, output 00:00:00, output hang never

  Last clearing of "show interface" counters never

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 444000 bits/sec, 49 packets/sec

  5 minute output rate 22000 bits/sec, 38 packets/sec

     77091999 packets input, 96465951570 bytes, 0 no buffer

     Received 15556 broadcasts (5906 multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog, 5906 multicast, 0 pause input

     0 input packets with dribble condition detected

     76344598 packets output, 6159064306 bytes, 0 underruns

     0 output errors, 0 collisions, 1 interface resets

     0 unknown protocol drops

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier, 0 pause output

     0 output buffer failures, 0 output buffers swapped out

TX-gb-lane-sw#

Thank you 

Loc

[Georg Pauwen]

Hello,

what model (e.g. 3750) is your switch ?

[Loc Nguyen]

Hi Georg, it is Cisco 2960XR.

[Georg Pauwen]

Hello,

if not already configured, enable 'mls qos' globally, clear the counters, and monitor if the output drops decrease or disappear.

[Loc Nguyen]

Hi Georg,

Could you explain why 'mls qos' works here?

Thanks

Loc

[Joseph W. Doherty]

BTW, if MLS QoS isn't already enabled, on a 2960 series switch, enabling it, using its defaults, might increase drops.

Although, if MLS QoS is already enabled, and you remove it, again with default settings, drops often decrease, however you lose QoS which you might be using.

As Giuseppe notes, your overall drop rate is very low, so low, such a low drop percentage is normally not a problem, but, that's only true if that percentage is about the same across your whole time period. What you really want to know, is whether you're having occasional high drop rates, during short time periods.  That can only been seen with overtime monitoring your drop rate, during short time intervals (like a second) looking for "spikes".  If such spikes exceed, about, 1%, then you might be having enough drops that remediation might be considered.

However, even with drop spikes, you need to understand network applications needs too.  Bulk file transfers, may have a less than optimal "goodput" rate, but that's often a non-issue.  Highly sensitive on-line applications, might be though.

[balaji.bandi]

You have mentioned Switch model  - 2960XR, can you post the below information to understand better

show version

sh controllers ethernet-controller x/x

show process CPU sorted | ex 0.00

show run

[Giuseppe Larosa]

Hello @Loc Nguyen ,

when looking at this kind of issues you need to compare the total output drops to the total output packets.

There is no practical impact in your case

19462 output drops

14713862133 packets output

this leads to a drop probability of :  19462 /(19462 + 14713862133 ) = 1.322696521264211e-6

means 1.32 packets dropped every million of packets this has no impact on TCP and also on UDP traffic

Your switch is not an issue.

Hope to help

Giuseppe

[Loc Nguyen]

Thanks!

[Georg Pauwen]

Hello,

there is a way to fine tune the individual queues. With 'mls qos' enabled globally, post the output of the commands below. As Joseph mentioned, enabling mls qos can increase output drops, because it enables the SRR scheduler in hardware and makes queues share bandwidth in a round-robin fashion.

show mls qos interface gigabitethernet1/0/23 statistics
show mls qos interface gigabitethernet2/0/23 statistics

show mls qos maps

[Joseph W. Doherty]

BTW, on the 2960 (and 3560/3750) switches, enabling QoS, using default settings, more often increases drops because of the way QoS "reserves" buffers per port egress queue rather than how those port egress queues share bandwidth.  How sharing bandwidth, between those queues, impacts drops, generally, is more about "shifting" what traffic is dropped.  I.e. overall port drop rate might not change much.  Again, this is with default settings.  As Georg correctly notes, enabled QoS allows fine tuning, including buffer management.  Optimal fine tuning might result in the lowest possible drop rate.

[faawa35252]

AMP does not provide integration with Active Directory in the intervening time, however there's a script available, wherein you may sync Active product page ofreviews articleDirectory (LDAP) Groups with AMP corporations, please take a glance

[JerryPro]

From my real time experience output drops can cause by heavy traffic. Long time ago I have faced similar problem & I have changed port & also Ethernet Cable but didn't solved the problem. Then I have reduce some traffic from that interface.