cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
3092
Views
0
Helpful
10
Replies

Lots of output drops - 3750 Stack

LucaSalvatore_2
Level 1
Level 1

We have a stack of 4 x 3750G-48PS-S which we use as an access layer switch in our network.

These are new switches, currently only about 10 ports are active as we have not migrated everyone accross yet.

I have noticed that on all active interfaces there are alot of out drops being reccorded.  All interfaces have a VoIP phone connected to the PC. QoS is enabled globally but there is no specifc QoS config on each interface.

The switchports are all configured the same, as shown below:

interface GigabitEthernet1/0/1
switchport access vlan 401
switchport mode access
switchport voice vlan 301
switchport port-security maximum 2
switchport port-security
switchport port-security aging time 2
switchport port-security aging type inactivity
spanning-tree portfast
spanning-tree bpduguard enable
spanning-tree guard root

These are the only errors showing up, there are no colisions, input or output errors, or anything else..

Any ideas why this is happening?

10 Replies 10

IAN WHITMORE
Level 4
Level 4

Are the speed and duplex negotiating properly? You could try fixing it. If that doesn't help you should definately configure QoS on the ports for VoIP.

Output queue drops are caused by congestion.

Regards,

Ian

Yes all ports are at 100Mb full.

There is very little congestion on this switch, its currently got maybe 10 people connected to it.  All interfaces show very little utilization, less that 5%.

If i disable QoS globally on the switch (no mls qos) the output drops stop.

Can you post the output for "show platform port-asic stats drop" -  when the qos is enabled.

Here it is:

switch#sh platform port-asic stats drop

Port-asic Port Drop Statistics - Summary
========================================
  RxQueue 0 Drop Stats: 0
  RxQueue 1 Drop Stats: 0
  RxQueue 2 Drop Stats: 0
  RxQueue 3 Drop Stats: 0

  Port  0 TxQueue Drop Stats: 0
  Port  1 TxQueue Drop Stats: 0
  Port  2 TxQueue Drop Stats: 0
  Port  3 TxQueue Drop Stats: 0

  Supervisor TxQueue Drop Statistics
    Queue  0: 0
    Queue  1: 0
    Queue  2: 0
    Queue  3: 0
    Queue  4: 0
    Queue  5: 0
    Queue  6: 0
    Queue  7: 0
    Queue  8: 0
    Queue  9: 0
    Queue 10: 0
    Queue 11: 0
    Queue 12: 0
    Queue 13: 0
    Queue 14: 0
    Queue 15: 0

Port-asic Port Drop Statistics - Details
========================================
  RxQueue Drop Statistics
    Queue 0
    Weight 0 Frames: 0
    Weight 1 Frames: 0
    Weight 2 Frames: 0
    Queue 1
    Weight 0 Frames: 0
    Weight 1 Frames: 0
    Weight 2 Frames: 0
    Queue 2
    Weight 0 Frames: 0
    Weight 1 Frames: 0
    Weight 2 Frames: 0
    Queue 3
    Weight 0 Frames: 0
    Weight 1 Frames: 0
    Weight 2 Frames: 0

  Port 0 TxQueue Drop Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0

  Port 1 TxQueue Drop Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0

  Port 2 TxQueue Drop Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0

  Port 3 TxQueue Drop Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
  Supervisor TxQueue Drop Statistics
    Queue  0: 0
    Queue  1: 0
    Queue  2: 0
    Queue  3: 0
    Queue  4: 0
    Queue  5: 0
    Queue  6: 0
    Queue  7: 0
    Queue  8: 0
    Queue  9: 0
    Queue 10: 0
    Queue 11: 0
    Queue 12: 0
    Queue 13: 0
    Queue 14: 0
    Queue 15: 0
switch#

That doesn't seem to show anything.... But here is an example from a show interface

GigabitEthernet1/0/22 is up, line protocol is up (connected)
  Hardware is Gigabit Ethernet, address is 68bd.abce.8616 (bia 68bd.abce.8616)
  MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  Full-duplex, 100Mb/s, media type is 10/100/1000BaseTX
  input flow-control is off, output flow-control is unsupported
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input never, output 00:00:00, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 2853
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 17000 bits/sec, 24 packets/sec
  5 minute output rate 549000 bits/sec, 129 packets/sec
     407925 packets input, 63472475 bytes, 0 no buffer
     Received 1311 broadcasts (0 multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 0 multicast, 0 pause input
     0 input packets with dribble condition detected
     1787040 packets output, 884800310 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 PAUSE output
     0 output buffer failures, 0 output buffers swapped out

Hey Luca,

The drop command is specific to the switch and should report a drop if interface is seeing output drops. May need to check which switch is the Master. Eg. If Switch2 is master, then these stats will be from switch2.

You can do:

#remote command all [command string]  <<--- all switches

#remote command [switch number] [command string]

It'll look something like:

#remote command 1 sh platform port-asic stats drop

Eugene

Switch 1 is the master, which is the switch i entered the command on.

Anyway, here is the output from everything


AUADFLEES100#remote command all sh platform port-asic stats drop
Switch : 1 : (Master)
---------------------


Port-asic Port Drop Statistics - Summary
========================================
  RxQueue 0 Drop Stats: 0
  RxQueue 1 Drop Stats: 0
  RxQueue 2 Drop Stats: 0
  RxQueue 3 Drop Stats: 0

  Port  0 TxQueue Drop Stats: 0
  Port  1 TxQueue Drop Stats: 0
  Port  2 TxQueue Drop Stats: 0
  Port  3 TxQueue Drop Stats: 0

  Supervisor TxQueue Drop Statistics
    Queue  0: 0
    Queue  1: 0
    Queue  2: 0
    Queue  3: 0
    Queue  4: 0
    Queue  5: 0
    Queue  6: 0
    Queue  7: 0
    Queue  8: 0
    Queue  9: 0
    Queue 10: 0
    Queue 11: 0
    Queue 12: 0
    Queue 13: 0
    Queue 14: 0
    Queue 15: 0

Port-asic Port Drop Statistics - Details
========================================
  RxQueue Drop Statistics
    Queue 0
    Weight 0 Frames: 0
    Weight 1 Frames: 0
    Weight 2 Frames: 0
    Queue 1
    Weight 0 Frames: 0
    Weight 1 Frames: 0
    Weight 2 Frames: 0
    Queue 2
    Weight 0 Frames: 0
    Weight 1 Frames: 0
    Weight 2 Frames: 0
    Queue 3
    Weight 0 Frames: 0
    Weight 1 Frames: 0
    Weight 2 Frames: 0

  Port 0 TxQueue Drop Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0

  Port 1 TxQueue Drop Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0

  Port 2 TxQueue Drop Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0

  Port 3 TxQueue Drop Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
  Supervisor TxQueue Drop Statistics
    Queue  0: 0
    Queue  1: 0
    Queue  2: 0
    Queue  3: 0
    Queue  4: 0
    Queue  5: 0
    Queue  6: 0
    Queue  7: 0
    Queue  8: 0
    Queue  9: 0
    Queue 10: 0
    Queue 11: 0
    Queue 12: 0
    Queue 13: 0
    Queue 14: 0
    Queue 15: 0
Switch : 2 :
------------


Port-asic Port Drop Statistics - Summary
========================================
  RxQueue 0 Drop Stats: 0
  RxQueue 1 Drop Stats: 0
  RxQueue 2 Drop Stats: 0
  RxQueue 3 Drop Stats: 0

  Port  0 TxQueue Drop Stats: 0
  Port  1 TxQueue Drop Stats: 0
  Port  2 TxQueue Drop Stats: 0
  Port  3 TxQueue Drop Stats: 0

  Supervisor TxQueue Drop Statistics
    Queue  0: 0
    Queue  1: 0
    Queue  2: 0
    Queue  3: 0
    Queue  4: 0
    Queue  5: 0
    Queue  6: 0
    Queue  7: 0
    Queue  8: 0
    Queue  9: 0
    Queue 10: 0
    Queue 11: 0
    Queue 12: 0
    Queue 13: 0
    Queue 14: 0
    Queue 15: 0

Port-asic Port Drop Statistics - Details
========================================
  RxQueue Drop Statistics
    Queue 0
    Weight 0 Frames: 0
    Weight 1 Frames: 0
    Weight 2 Frames: 0
    Queue 1
    Weight 0 Frames: 0
    Weight 1 Frames: 0
    Weight 2 Frames: 0
    Queue 2
    Weight 0 Frames: 0
    Weight 1 Frames: 0
    Weight 2 Frames: 0
    Queue 3
    Weight 0 Frames: 0
    Weight 1 Frames: 0
    Weight 2 Frames: 0

  Port 0 TxQueue Drop Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0

  Port 1 TxQueue Drop Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0

  Port 2 TxQueue Drop Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0

  Port 3 TxQueue Drop Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
  Supervisor TxQueue Drop Statistics
    Queue  0: 0
    Queue  1: 0
    Queue  2: 0
    Queue  3: 0
    Queue  4: 0
    Queue  5: 0
    Queue  6: 0
    Queue  7: 0
    Queue  8: 0
    Queue  9: 0
    Queue 10: 0
    Queue 11: 0
    Queue 12: 0
    Queue 13: 0
    Queue 14: 0
    Queue 15: 0
Switch : 3 :
------------


Port-asic Port Drop Statistics - Summary
========================================
  RxQueue 0 Drop Stats: 0
  RxQueue 1 Drop Stats: 0
  RxQueue 2 Drop Stats: 0
  RxQueue 3 Drop Stats: 0

  Port  0 TxQueue Drop Stats: 0
  Port  1 TxQueue Drop Stats: 0
  Port  2 TxQueue Drop Stats: 0
  Port  3 TxQueue Drop Stats: 0

  Supervisor TxQueue Drop Statistics
    Queue  0: 0
    Queue  1: 0
    Queue  2: 0
    Queue  3: 0
    Queue  4: 0
    Queue  5: 0
    Queue  6: 0
    Queue  7: 0
    Queue  8: 0
    Queue  9: 0
    Queue 10: 0
    Queue 11: 0
    Queue 12: 0
    Queue 13: 0
    Queue 14: 0
    Queue 15: 0

Port-asic Port Drop Statistics - Details
========================================
  RxQueue Drop Statistics
    Queue 0
    Weight 0 Frames: 0
    Weight 1 Frames: 0
    Weight 2 Frames: 0
    Queue 1
    Weight 0 Frames: 0
    Weight 1 Frames: 0
    Weight 2 Frames: 0
    Queue 2
    Weight 0 Frames: 0
    Weight 1 Frames: 0
    Weight 2 Frames: 0
    Queue 3
    Weight 0 Frames: 0
    Weight 1 Frames: 0
    Weight 2 Frames: 0

  Port 0 TxQueue Drop Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0

  Port 1 TxQueue Drop Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0

  Port 2 TxQueue Drop Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0

  Port 3 TxQueue Drop Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
  Supervisor TxQueue Drop Statistics
    Queue  0: 0
    Queue  1: 0
    Queue  2: 0
    Queue  3: 0
    Queue  4: 0
    Queue  5: 0
    Queue  6: 0
    Queue  7: 0
    Queue  8: 0
    Queue  9: 0
    Queue 10: 0
    Queue 11: 0
    Queue 12: 0
    Queue 13: 0
    Queue 14: 0
    Queue 15: 0
Switch : 4 :
------------


Port-asic Port Drop Statistics - Summary
========================================
  RxQueue 0 Drop Stats: 0
  RxQueue 1 Drop Stats: 0
  RxQueue 2 Drop Stats: 0
  RxQueue 3 Drop Stats: 0

  Port  0 TxQueue Drop Stats: 0
  Port  1 TxQueue Drop Stats: 0
  Port  2 TxQueue Drop Stats: 0
  Port  3 TxQueue Drop Stats: 0

  Supervisor TxQueue Drop Statistics
    Queue  0: 0
    Queue  1: 0
    Queue  2: 0
    Queue  3: 0
    Queue  4: 0
    Queue  5: 0
    Queue  6: 0
    Queue  7: 0
    Queue  8: 0
    Queue  9: 0
    Queue 10: 0
    Queue 11: 0
    Queue 12: 0
    Queue 13: 0
    Queue 14: 0
    Queue 15: 0

Port-asic Port Drop Statistics - Details
========================================
  RxQueue Drop Statistics
    Queue 0
    Weight 0 Frames: 0
    Weight 1 Frames: 0
    Weight 2 Frames: 0
    Queue 1
    Weight 0 Frames: 0
    Weight 1 Frames: 0
    Weight 2 Frames: 0
    Queue 2
    Weight 0 Frames: 0
    Weight 1 Frames: 0
    Weight 2 Frames: 0
    Queue 3
    Weight 0 Frames: 0
    Weight 1 Frames: 0
    Weight 2 Frames: 0

  Port 0 TxQueue Drop Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0

  Port 1 TxQueue Drop Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0

  Port 2 TxQueue Drop Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0

  Port 3 TxQueue Drop Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
  Supervisor TxQueue Drop Statistics
    Queue  0: 0
    Queue  1: 0
    Queue  2: 0
    Queue  3: 0
    Queue  4: 0
    Queue  5: 0
    Queue  6: 0
    Queue  7: 0
    Queue  8: 0
    Queue  9: 0
    Queue 10: 0
    Queue 11: 0
    Queue 12: 0
    Queue 13: 0
    Queue 14: 0
    Queue 15: 0

AUADFLEES100#

Hi Luca,

This seems unusual so I'm going to do a sanity check and ask a few questions. The platform asic stats should report drops in queues if you're seening drops on the interface.

I like to use

#term exec prom time   <<--- at the prompt. Basically prints a timestamp with every command

1. are the output drops on the interface continuously incremeting? Eg. clear counter and monitor

2. Are the controller outputs reporting the same thing? eg.

#sh ether controller [inter]  <<-- checking for consistency

#sh int counter error <<-- making sure output drops are recorded consistently

3. What version of IOS are you running?

4. Focus the output a little

#sh platform pm if-number <<--- look for g1/0/1

under port it should be ASIC/Port

#sh plat port-asic stat drop port x asic y   <<---- gets the drop stats for that port.

#sh plat port-asic stat enqueue port x asic y  <<--- check that queue stats are infact incrementing, so always capture multiple snapshots.

If stats arent incrementing then maybe the command output has issues. We're basically looking for the inconsistency.

Oh, one last question. I noticed that your host name has changed from switch to AUADFLEES10. Was this done recently? If not, are we looking at the right switch?

Eugene

hi Eugene

1) Yes output drops are incrementing all the time.  This happens on every active interface but i will just focus on Gi1/0/1

2)

AUADFLEES100#show controllers ethernet-controller gi1/0/1
Load for five secs: 13%/0%; one minute: 11%; five minutes: 11%
Time source is NTP, 13:51:27.057 EDST Thu Mar 17 2011


     Transmit GigabitEthernet1/0/1            Receive
    772623400 Bytes                        125906003 Bytes                   
       763679 Unicast frames                  408194 Unicast frames          
       475833 Multicast frames                     7 Multicast frames        
        34816 Broadcast frames                  1329 Broadcast frames        
            0 Too old frames               125216490 Unicast bytes           
            0 Deferred frames                    448 Multicast bytes         
            0 MTU exceeded frames             356126 Broadcast bytes         
            0 1 collision frames                   0 Alignment errors        
            0 2 collision frames                   0 FCS errors              
            0 3 collision frames                   0 Oversize frames         
            0 4 collision frames                   0 Undersize frames        
            0 5 collision frames                   0 Collision fragments     
            0 6 collision frames      
            0 7 collision frames               32136 Minimum size frames     
            0 8 collision frames              234538 65 to 127 byte frames   
            0 9 collision frames               62472 128 to 255 byte frames  
            0 10 collision frames               7326 256 to 511 byte frames  
            0 11 collision frames              10180 512 to 1023 byte frames 
            0 12 collision frames              62871 1024 to 1518 byte frames
            0 13 collision frames                  0 Overrun frames          
            0 14 collision frames                  0 Pause frames            
            0 15 collision frames     
            0 Excessive collisions                 1 Symbol error frames     
            0 Late collisions                      0 Invalid frames, too large
            0 VLAN discard frames                  8 Valid frames, too large 
            0 Excess defer frames                  0 Invalid frames, too small
        69072 64 byte frames                       0 Valid frames, too small 
       574328 127 byte frames         
        19791 255 byte frames                      0 Too old frames          
        63041 511 byte frames                      0 Valid oversize frames   
       141265 1023 byte frames                     0 System FCS error frames 
       406830 1518 byte frames                     0 RxPortFifoFull drop frame
            1 Too large frames        
            0 Good (1 coll) frames    
            0 Good (>1 coll) frames  

AUADFLEES100#sh int count errors           
Load for five secs: 11%/0%; one minute: 11%; five minutes: 11%
Time source is NTP, 13:54:12.255 EDST Thu Mar 17 2011


Port        Align-Err     FCS-Err    Xmit-Err     Rcv-Err  UnderSize  OutDiscards
Gi1/0/1             0           0           0           1          0         1934
Gi1/0/2             0           0           0           0          0            0

Port      Single-Col  Multi-Col   Late-Col  Excess-Col  Carri-Sen      Runts     Giants
Gi1/0/1            0          0          0           0          0          0          0
Gi1/0/2            0          0          0           0          0          0          0

3) IOS verison is 12.2(53)SE2

4) AUADFLEES100#sh platform pm if-numbers
Load for five secs: 12%/0%; one minute: 11%; five minutes: 11%
Time source is NTP, 13:55:33.047 EDST Thu Mar 17 2011


interface gid  gpn  lpn  port slot unit slun port-type lpn-idb gpn-idb
----------------------------------------------------------------------
Gi1/0/1   1    1    1    6/3  1    1    1    local     Yes     Yes

AUADFLEES100#sh platform port-asic stats enqueue port 6 asic 3
Load for five secs: 10%/0%; one minute: 11%; five minutes: 11%
Time source is NTP, 13:57:25.430 EDST Thu Mar 17 2011


Port-asic Port Enqueue Statistics - Summary
===========================================
  RxQueue 0 Enqueue Stats: 52
  RxQueue 1 Enqueue Stats: 1748
  RxQueue 2 Enqueue Stats: 2103325
  RxQueue 3 Enqueue Stats: 0

  Port  0 TxQueue Enqueue Stats:          2
  Port  1 TxQueue Enqueue Stats:         46
  Port  2 TxQueue Enqueue Stats:          2
  Port  3 TxQueue Enqueue Stats:    7697082

Port-asic Port Enqueue Statistics - Details
===========================================
  RxQueue Enqueue Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 52
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 1748
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 2103325
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0

  Port 6 TxQueue Enqueue Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0

and again

AUADFLEES100#sh platform port-asic stats enqueue port 6 asic 3
Load for five secs: 10%/0%; one minute: 11%; five minutes: 11%
Time source is NTP, 13:58:40.743 EDST Thu Mar 17 2011


Port-asic Port Enqueue Statistics - Summary
===========================================
  RxQueue 0 Enqueue Stats: 52
  RxQueue 1 Enqueue Stats: 1748
  RxQueue 2 Enqueue Stats: 2103331
  RxQueue 3 Enqueue Stats: 0

  Port  0 TxQueue Enqueue Stats:          2
  Port  1 TxQueue Enqueue Stats:         46
  Port  2 TxQueue Enqueue Stats:          2
  Port  3 TxQueue Enqueue Stats:    7697175

Port-asic Port Enqueue Statistics - Details
===========================================
  RxQueue Enqueue Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 52
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 1748
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 2103331
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0

  Port 6 TxQueue Enqueue Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0

The switch name is AUADFLEES100.  Might have been a typo with the name before as it has not changed.

Thanks for the help so far.  I also have a TAC case raised for this.

No problems Luca, Fills my time in management meetings but don't tell the Mangers

From the enqueue captures -> it should be ASIC/port -> ASIC = 6, port = 3 -> #sh plat port-asic stat enqueue port 3 asic 6

However in your version of IOS, you can simply issue the interface. They made it easier for us after we complained!

#sh plat port-asic stat enqueue g1/0/1

#sh plat port-asic stat drop g1/0/1

Has there been any impact to voice quality etc reported? Any connectivity type issues?

If no network issues, I'm thinking there's something wrong with one of the counters but not sure which one.

Normally, we'd depend on the port-asic level counters. Could you check that the enqueue stats for g1/0/1 actually increment in the TX queues? This would mean enqueue is working.

I feel that the inconsistency is the #sh plat port-asic stat drop counters are not reporting drops while the s#h interface & #sh int counter is reporting output drops. In my opinion, they should mirror up pretty close.

The other thought is, you could try and do some testing on a free port with a very basic config - no port security etc, just access. In times where things are unusual, it could be some type of feature interaction

See what TAC comes up with

Eugene

One more thing I forgot to mention this -  The statement "If i disable QoS globally on the switch (no mls qos) the output drops stop"

indicates that output drops are likely to be caused by the QoS configuration. When you enable QoS, the queues are carved up by default and it becomes normal to see drops in the low priority queues, even at low traffic rates due to "micro bursting" of TCP traffic.

Which queue is defined by the Mapping and there are defaults:

http://www.cisco.com/en/US/docs/switches/lan/catalyst3750/software/release/12.2_55_se/configuration/guide/swqos.html#wp1021252

From a configuration point of view, it will be defined as q1,q2,q3,q4

Then from the ASIC point of view it will be  - tx queue starting 0,1,2,3  (zero based)

#sh mls qos int g1/0/1 stat <<-- tells you the marked traffic is seen on the interface

Enqueue -> tells us which queues are seeing traffic and we can identify which one has more

Drops -> tells us which queue is dropping -> we should be able to define whether it is low priority traffic etc

Ideally, we don't see drops in the priority queue (cos5, DSCP46 - q1) and it would be normal to see drops in CoS0/DSCP0 queues (unmarked data traffic).

HTH

Eugene

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Review Cisco Networking products for a $25 gift card