3750X swiches droping packings on the uplink interface

[Eric Lindsey]

We have a remote site that is using 3750X switches as layer 2 switches back to our home site.  The uplink port is showing dropped packets but the utilization on the link is never about 10%.  We have a 100Mb circuit to this site.  Our speed tests and iperf tests are not showing any issues that we can see.  However the port is still droping packets.  It is not dropping at a high rate but they are dropping.          

switch#sh platform port-asic stats drop gi1/1/4

  Interface Gi1/1/4 TxQueue Drop Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 52876
      Weight 1 Frames 2
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 1330874
    Queue 4
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 5
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 6
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 7
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
switch#

Any thoughts on this issue?  Is there a way to capture these dropped packets to see what they are?  We do have VOIP phones at the site and are using Qos.

[Joseph W. Doherty]

Disclaimer

The   Author of this posting offers the information contained within this   posting without consideration and with the reader's understanding that   there's no implied or expressed suitability or fitness for any purpose.   Information provided is for informational purposes only and should not   be construed as rendering professional advice of any kind. Usage of  this  posting's information is solely at reader's own risk.

Liability Disclaimer

In   no event shall Author be liable for any damages whatsoever (including,   without limitation, damages for loss of use, data or profit) arising  out  of the use or inability to use the posting's information even if  Author  has been advised of the possibility of such damage.

Posting

If you're dropping packets, you're utilization is going above 100%.

Your measured utilization is an average utilization over some time period (often in minutes).  It's not uncommon for short time transient bursts (fancy term, micro bursts) isn't seen during measured usage.

Tuning of your QoS parameters may mitigate these drops.  Disabling QoS may also mitigate these drops and yet still be sufficient for your VoIP needs, as you're running at LAN bandwidths.

[Eric Lindsey]

We have talked to the provider in the middle and even they can't see what would be dropping. We have wireshark traces running but do not see any giant amounts of data live.

Sent from Cisco Technical Support iPhone App

[Leo Laohoo]

We have talked to the provider in the middle and even they can't see what would be dropping.

Talking to your provider is your last resort.  Asking them to help you is, frankly, is a waste of your time.  They are there to make money and not hold-your-hand.

Ok, so G1/1/4 is the link towards your ISP?  Please post the output to the follwoing commands:

1.  sh interface G1/1/4;

2.  sh controll e G1/1/4;

3.  sh controll util; and

4.  sh version

[Eric Lindsey]

WhiteBirch-2nd-Floor-Stack#sh inter gi1/1/4
GigabitEthernet1/1/4 is up, line protocol is up (connected)
  Hardware is Gigabit Ethernet, address is 7081.0533.d134 (bia 7081.0533.d134)
  Description: Ntelos Uplink
  MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,
     reliability 255/255, txload 10/255, rxload 7/255
  Encapsulation ARPA, loopback not set
  Keepalive not set
  Full-duplex, 100Mb/s, link type is auto, media type is 10/100/1000BaseTX SFP
  input flow-control is off, output flow-control is unsupported
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input never, output never, output hang never
  Last clearing of "show interface" counters 2d17h
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 8901
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 2833000 bits/sec, 999 packets/sec
  5 minute output rate 4038000 bits/sec, 1114 packets/sec
     47325860 packets input, 30504151950 bytes, 0 no buffer
     Received 62560 broadcasts (51133 multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 51133 multicast, 0 pause input
     0 input packets with dribble condition detected
     35422706 packets output, 11462577005 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 pause output
     0 output buffer failures, 0 output buffers swapped out
WhiteBirch-2nd-Floor-Stack#

WhiteBirch-2nd-Floor-Stack#show controll e gi1/1/4

     Transmit GigabitEthernet1/1/4            Receive
    879147660 Bytes                        227920817 Bytes
    419755981 Unicast frames              2816115621 Unicast frames
     35470387 Multicast frames               5917617 Multicast frames
     47987691 Broadcast frames                592719 Broadcast frames
            0 Too old frames              4018471356 Unicast bytes
            0 Deferred frames              461577066 Multicast bytes
            0 MTU exceeded frames           42839691 Broadcast bytes
            0 1 collision frames                   0 Alignment errors
            0 2 collision frames                   0 FCS errors
            0 3 collision frames                   0 Oversize frames
            0 4 collision frames                   0 Undersize frames
            0 5 collision frames                   0 Collision fragments
            0 6 collision frames
            0 7 collision frames          2800044823 Minimum size frames
            0 8 collision frames          1079739152 65 to 127 byte frames
            0 9 collision frames          1197004097 128 to 255 byte frames
            0 10 collision frames          339619576 256 to 511 byte frames
            0 11 collision frames          367567392 512 to 1023 byte frames
            0 12 collision frames         1333618213 1024 to 1518 byte frames
            0 13 collision frames                  0 Overrun frames
            0 14 collision frames                  0 Pause frames
            0 15 collision frames
            0 Excessive collisions                 0 Symbol error frames
            0 Late collisions                      0 Invalid frames, too large
            0 VLAN discard frames                  0 Valid frames, too large
            0 Excess defer frames                  0 Invalid frames, too small
   3198835225 64 byte frames                       0 Valid frames, too small
   4169705153 127 byte frames
     13132247 255 byte frames                      0 Too old frames
    284093150 511 byte frames                      0 Valid oversize frames
    149731275 1023 byte frames                     0 System FCS error frames
   1277651601 1518 byte frames                     0 RxPortFifoFull drop frame
            0 Too large frames
            0 Good (1 coll) frames
            0 Good (>1 coll) frames

WhiteBirch-2nd-Floor-Stack#

WhiteBirch-2nd-Floor-Stack#show controllers utilization; and
Port       Receive Utilization  Transmit Utilization
Gi1/0/1            0                    0
Gi1/0/2            0                    0
Gi1/0/3            0                    0
Gi1/0/4            0                    0
Gi1/0/5            0                    0
Gi1/0/6            0                    0
Gi1/0/7            0                    0
Gi1/0/8            0                    0
Gi1/0/9            0                    0
Gi1/0/10           0                    0
Gi1/0/11           0                    0
Gi1/0/12           0                    0
Gi1/0/13           0                    0
Gi1/0/14           0                    0
Gi1/0/15           0                    0
Gi1/0/16           0                    0
Gi1/0/17           0                    0
Gi1/0/18           0                    0
Gi1/0/19           0                    0
Gi1/0/20           0                    0
Gi1/0/21           0                    0
Gi1/0/22           0                    0
Gi1/0/23           0                    0
Gi1/0/24           0                    0
Gi1/0/25           0                    0
Gi1/0/26           0                    0
Gi1/0/27           0                    0
Gi1/0/28           0                    0
Gi1/0/29           0                    0
Gi1/0/30           0                    0
Gi1/0/31           0                    0
Gi1/0/32           0                    0
Gi1/0/33           0                    0
Gi1/0/34           0                    0
Gi1/0/35           0                    0
Gi1/0/36           0                    0
Gi1/0/37           0                    0
Gi1/0/38           0                    0
Gi1/0/39           0                    0
Gi1/0/40           0                    0
Gi1/0/41           0                    0
Gi1/0/42           0                    0
Gi1/0/43           0                    0
Gi1/0/44           0                    0
Gi1/0/45           0                    0
Gi1/0/46           0                    0
Gi1/0/47           0                    0
Gi1/0/48           0                    0
Gi1/1/1            0                    0
Gi1/1/2            0                    0
Gi1/1/3            0                    0
Gi1/1/4            2                    5
Te1/1/1            0                    0
Te1/1/2            0                    0
Gi2/0/1            0                    0
Gi2/0/2            0                    0
Gi2/0/3            0                    0
Gi2/0/4            0                    0
Gi2/0/5            0                    0
Gi2/0/6            0                    0
Gi2/0/7            0                    0
Gi2/0/8            0                    0
Gi2/0/9            0                    0
Gi2/0/10           0                    0
Gi2/0/11           0                    0
Gi2/0/12           0                    0
Gi2/0/13           0                    0
Gi2/0/14           0                    0
Gi2/0/15           0                    0
Gi2/0/16           0                    0
Gi2/0/17           0                    0
Gi2/0/18           0                    0
Gi2/0/19           0                    0
Gi2/0/20           0                    0
Gi2/0/21           0                    0
Gi2/0/22           0                    0
Gi2/0/23           0                    0
Gi2/0/24           0                    0
Gi2/1/1            0                    0
Gi2/1/2            0                    0
Gi2/1/3            0                    0
Gi2/1/4            0                    0
Te2/1/1            0                    0
Te2/1/2            0                    0

Total Ports : 84
Switch Receive Bandwidth Percentage Utilization  : 0
Switch Transmit Bandwidth Percentage Utilization : 0

Stack Ring Percentage Utilization : 0

WhiteBirch-2nd-Floor-Stack#

WhiteBirch-2nd-Floor-Stack#sh version
Cisco IOS Software, C3750E Software (C3750E-UNIVERSALK9-M), Version 12.2(58)SE2, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2011 by Cisco Systems, Inc.
Compiled Thu 21-Jul-11 01:23 by prod_rel_team

ROM: Bootstrap program is C3750E boot loader
BOOTLDR: C3750E Boot Loader (C3750X-HBOOT-M) Version 12.2(53r)SE2, RELEASE SOFTWARE (fc1)

WhiteBirch-2nd-Floor-Stack uptime is 45 weeks, 2 days, 4 hours, 21 minutes
System returned to ROM by power-on
System restarted at 03:27:20 EDT Sat Jun 30 2012
System image file is "flash:/c3750e-universalk9-mz.122-58.SE2/c3750e-universalk9-mz.122-58.SE2.bin"

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

License Level: ipbase
License Type: Permanent
Next reload license Level: ipbase

cisco WS-C3750X-48 (PowerPC405) processor (revision A0) with 262144K bytes of memory.
Processor board ID FDO1530P1HN
Last reset from power-on
1 Virtual Ethernet interface
1 FastEthernet interface
80 Gigabit Ethernet interfaces
4 Ten Gigabit Ethernet interfaces
The password-recovery mechanism is enabled.

512K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address       : 70:81:05:33:D1:00
Motherboard assembly number     : 73-12552-05
Motherboard serial number       : FDO15310VE3
Model revision number           : A0
Motherboard revision number     : B0
Model number                    : WS-C3750X-48T-S
Daughterboard assembly number   : 800-32727-01
Daughterboard serial number     : FDO15310EM8
System serial number            : FDO1530P1HN
Top Assembly Part Number        : 800-31326-02
Top Assembly Revision Number    : C0
Version ID                      : V02
CLEI Code Number                : COMJT00ARB
Hardware Board Revision Number  : 0x03

Switch Ports Model              SW Version            SW Image
------ ----- -----              ----------            ----------
*    1 54    WS-C3750X-48       12.2(58)SE2           C3750E-UNIVERSALK9-M
     2 30    WS-C3750X-24P      12.2(58)SE2           C3750E-UNIVERSALK9-M

Switch 02
---------
Switch Uptime                   : 45 weeks, 2 days, 4 hours, 2 minutes
Base ethernet MAC Address       : 70:81:05:28:32:80
Motherboard assembly number     : 73-12551-05
Motherboard serial number       : FDO15302BNM
Model revision number           : A0
Motherboard revision number     : C0
Model number                    : WS-C3750X-24P-S
Daughterboard assembly number   : 800-32727-01
Daughterboard serial number     : FDO153026DV
System serial number            : FDO1530R12G
Top assembly part number        : 800-31325-02
Top assembly revision number    : C0
Version ID                      : V02
CLEI Code Number                : COMJR00ARB
License Level                   : ipbase
License Type                    : Permanent
Next reboot licensing Level     : ipbase

Configuration register is 0xF

WhiteBirch-2nd-Floor-Stack#

[Leo Laohoo · Accepted solution]

Thanks Eric.

The output for the "sh controller e G 1/1/14" and "sh interface G 1/1/14" confirms you do not have any line issue.  I can confirm that the interface is showing signs of output drops.

If I get your opening thread, the link goes to your ISP which has configured a 100 Mbps link.  You also have a 100BaseTX SFP.

I would recommend you try the following to test:

1.  Configure QoS;

2.  Drop the Ethernet link to a 10 Mbps and see if output drops occur.  If it doesn't occur, then there's a possibility that your ISP did not configure their end properly; and

3.  Possible IOS bug.