cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
3892
Views
0
Helpful
8
Replies

Large numbers of Total output drops

mcguirebob
Level 1
Level 1

We have 2 switches, each with 3 Gig interfaces that are used as the VLAN for data replication between 3 nodes of an HA SAN (with gig nics).  These ports are showing a large number of Total output drops in the show interface output.  What things can I try to narrow down where the issues are coming from?  Here's is the output of one of the interfaces:

SWSAN1#show interfaces GigabitEthernet 0/2

GigabitEthernet0/2 is up, line  protocol is up (connected)

  Hardware is Gigabit Ethernet, address is 0014.a80e.2382 (bia 0014.a80e.2382)

  Description: SAN1_BPlane

  MTU 1500 bytes, BW 1000000 Kbit, DLY 10  usec,

     reliability 255/255, txload 2/255, rxload 2/255

  Encapsulation ARPA, loopback not  set

  Keepalive set  (10 sec)

  Full-duplex, 1000Mb/s, media type is  10/100/1000BaseTX

  input flow-control is off, output flow-control is unsupported

  ARP type: ARPA, ARP Timeout  04:00:00

  Last input never,  output 00:00:00, output hang never

  Last clearing of "show interface"  counters never

  Input queue: 0/75/0/0  (size/max/drops/flushes); Total output drops:  3871100

  Queueing strategy: fifo

  Output queue: 0/40  (size/max)

  5 minute input rate 10183000 bits/sec,  1448 packets/sec

  5 minute output rate 11561000 bits/sec,  1510 packets/sec

     1493229684 packets input,  1329967163222 bytes, 0 no buffer

     Received 7240 broadcasts (3  multicasts)

     0 runts, 0 giants, 0  throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog, 3 multicast, 0 pause  input

     0 input packets with dribble condition  detected

     1638884964 packets output,  1837773160862 bytes, 0 underruns

     0 output errors, 0 collisions, 0  interface resets

    0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier, 0 PAUSE  output

     0 output buffer failures, 0 output  buffers swapped out

Thanks in advance

8 Replies 8

spremkumar
Level 9
Level 9

Hi

Are you seeing continous increments within the output drop counter? can you check clearint the counters once? also cna you paste the config of this interface? are you seeing anything in the logs with respect to  this interface? drops occurs mainly due to insufficient queue size to hold the data.

regds

Hi,

Yes, on the one interface that this system is using out of the 6, the drops continue to occur after clearing the counters, and again, this is the only interface that is showing anything on the 5 minute input and output rates.

interface config:

interface GigabitEthernet0/2

description SAN1_BPlane

switchport access vlan 100

switchport mode access

flowcontrol receive on

spanning-tree portfast

SWSAN2#show interfaces gigabitEthernet 0/2
GigabitEthernet0/2 is up, line protocol is up (connected)
  Hardware is Gigabit Ethernet, address is 0014.a82e.aa02 (bia 0014.a82e.aa02)
  Description: FICPASAN1_BPlane
  MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
     reliability 255/255, txload 4/255, rxload 6/255
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX
  input flow-control is on, output flow-control is unsupported
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input never, output 00:00:01, output hang never
  Last clearing of "show interface" counters 00:08:35
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 1594
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 26124000 bits/sec, 2965 packets/sec
  5 minute output rate 16630000 bits/sec, 2566 packets/sec
     1953356 packets input, 2072245740 bytes, 0 no buffer
     Received 0 broadcasts (0 multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 0 multicast, 0 pause input
     0 input packets with dribble condition detected
     1780461 packets output, 1551855955 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 PAUSE output
     0 output buffer failures, 0 output buffers swapped out

mcguirebob
Level 1
Level 1

Now a couple of the other interfaces are starting to see some traffic, and the drops are incrementing on those also.  The configuration for all of the interfaces is the same:

SWSAN2#show interfaces gigabitEthernet 0/4
GigabitEthernet0/4 is up, line protocol is up (connected)
  Hardware is Gigabit Ethernet, address is 0014.a82e.aa04 (bia 0014.a82e.aa04)
  Description: SAN2_BPlane
  MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
     reliability 255/255, txload 3/255, rxload 2/255
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX
  input flow-control is on, output flow-control is unsupported
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input never, output 00:00:00, output hang never
  Last clearing of "show interface" counters 00:16:08
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 74
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 11233000 bits/sec, 1632 packets/sec
  5 minute output rate 14599000 bits/sec, 1765 packets/sec
     1715069 packets input, 1361564336 bytes, 0 no buffer
     Received 0 broadcasts (0 multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 0 multicast, 0 pause input
     0 input packets with dribble condition detected
     1976729 packets output, 2143226538 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 PAUSE output
     0 output buffer failures, 0 output buffers swapped out

To be able to troubleshoot output drops, i would require the following outputs

if it is a modular switch, i would require the following outputs, if its not a module switch, i would need show version and output #2 and 3

1. show mod

2. show int | inc line | drops

3. show int | inc line | rate

Capture output 2 and 3 twice in successsion

What i want to see is the rate at which the drops are incrementing

SWSAN2#show version              
Cisco IOS Software, C3560 Software (C3560-IPBASEK9-M), Version 12.2(53)SE2, RELEASE SOFTWARE (fc3)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2010 by Cisco Systems, Inc.
Compiled Wed 21-Apr-10 05:33 by prod_rel_team
Image text-base: 0x01000000, data-base: 0x02B00000

ROM: Bootstrap program is C3560 boot loader
BOOTLDR: C3560 Boot Loader (C3560-HBOOT-M) Version 12.2(44)SE5, RELEASE SOFTWARE (fc1)

SWSAN2 uptime is 22 weeks, 1 day, 9 hours, 36 minutes
System returned to ROM by power-on
System restarted at 06:31:06 EDT Tue Aug 10 2010
System image file is "flash:c3560-ipbasek9-mz.122-53.SE2/c3560-ipbasek9-mz.122-53.SE2.bin"


This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

cisco WS-C3560G-24TS (PowerPC405) processor (revision C0) with 131072K bytes of memory.
Processor board ID FOC0924U2RA
Last reset from power-on
2 Virtual Ethernet interfaces
28 Gigabit Ethernet interfaces
The password-recovery mechanism is enabled.

512K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address       : 00:14:A8:2E:AA:00
Motherboard assembly number     : 73-9708-04
Power supply part number        : 341-0098-01
Motherboard serial number       : FOC09245D0J
Power supply serial number      : DCA09210JX8
Model revision number           : C0
Motherboard revision number     : A0
Model number                    : WS-C3560G-24TS-S
System serial number            : FOC0924U2RA
SFP Module assembly part number : 73-7757-03
SFP Module revision Number      : A0
SFP Module serial number        : CAT09220WNP
Top Assembly Part Number        : 800-25864-02
Top Assembly Revision Number    : A0
Version ID                      : 02
CLEI Code Number                : CNMW200ARB
Hardware Board Revision Number  : 0x05


Switch Ports Model              SW Version            SW Image                
------ ----- -----              ----------            ----------              
*    1 28    WS-C3560G-24TS     12.2(53)SE2           C3560-IPBASEK9-M       


Configuration register is 0xF

FICPASWSAN2#show int | inc line | drops
Vlan1 is up, line protocol is up
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Vlan102 is up, line protocol is up
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/1 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/2 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 1786
GigabitEthernet0/3 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/4 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 1046
GigabitEthernet0/5 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/6 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 990
GigabitEthernet0/7 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/8 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/9 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/10 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/11 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/12 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/13 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/14 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/15 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/16 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/17 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/18 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/19 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/20 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/21 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/22 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/23 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/24 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/25 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/26 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/27 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/28 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Port-channel2 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

FICPASWSAN2#show int | inc line | rate
Vlan1 is up, line protocol is up
  5 minute input rate 3000 bits/sec, 5 packets/sec
  5 minute output rate 1000 bits/sec, 1 packets/sec
Vlan102 is up, line protocol is up
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/1 is up, line protocol is up (connected)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/2 is up, line protocol is up (connected)
  5 minute input rate 20399000 bits/sec, 2129 packets/sec
  5 minute output rate 7659000 bits/sec, 1595 packets/sec
GigabitEthernet0/3 is up, line protocol is up (connected)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/4 is up, line protocol is up (connected)
  5 minute input rate 4262000 bits/sec, 1002 packets/sec
  5 minute output rate 13334000 bits/sec, 1386 packets/sec
GigabitEthernet0/5 is up, line protocol is up (connected)
  5 minute input rate 138000 bits/sec, 3 packets/sec
  5 minute output rate 16000 bits/sec, 8 packets/sec
GigabitEthernet0/6 is up, line protocol is up (connected)
  5 minute input rate 8016000 bits/sec, 1200 packets/sec
  5 minute output rate 11662000 bits/sec, 1352 packets/sec
GigabitEthernet0/7 is up, line protocol is up (connected)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 4000 bits/sec, 5 packets/sec
GigabitEthernet0/8 is down, line protocol is down (notconnect)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/9 is up, line protocol is up (connected)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/10 is down, line protocol is down (notconnect)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/11 is down, line protocol is down (notconnect)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/12 is down, line protocol is down (notconnect)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/13 is up, line protocol is up (connected)
  5 minute input rate 1422000 bits/sec, 119 packets/sec
  5 minute output rate 112000 bits/sec, 45 packets/sec
GigabitEthernet0/14 is up, line protocol is up (connected)
  5 minute input rate 176000 bits/sec, 6 packets/sec
  5 minute output rate 255000 bits/sec, 8 packets/sec
GigabitEthernet0/15 is up, line protocol is up (connected)
  5 minute input rate 417000 bits/sec, 32 packets/sec
  5 minute output rate 3000 bits/sec, 4 packets/sec
GigabitEthernet0/16 is up, line protocol is up (connected)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/17 is up, line protocol is up (connected)
  5 minute input rate 544000 bits/sec, 117 packets/sec
  5 minute output rate 2968000 bits/sec, 260 packets/sec
GigabitEthernet0/18 is down, line protocol is down (notconnect)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/19 is up, line protocol is up (connected)
  5 minute input rate 24000 bits/sec, 5 packets/sec
  5 minute output rate 182000 bits/sec, 10 packets/sec
GigabitEthernet0/20 is down, line protocol is down (notconnect)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/21 is up, line protocol is up (connected)
  5 minute input rate 78000 bits/sec, 45 packets/sec
  5 minute output rate 1650000 bits/sec, 138 packets/sec
GigabitEthernet0/22 is up, line protocol is up (connected)
  5 minute input rate 192000 bits/sec, 12 packets/sec
  5 minute output rate 285000 bits/sec, 96 packets/sec
GigabitEthernet0/23 is up, line protocol is up (connected)
  5 minute input rate 2603000 bits/sec, 223 packets/sec
  5 minute output rate 137000 bits/sec, 4 packets/sec
GigabitEthernet0/24 is down, line protocol is down (notconnect)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/25 is down, line protocol is down (notconnect)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/26 is down, line protocol is down (notconnect)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/27 is down, line protocol is down (notconnect)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/28 is down, line protocol is down (notconnect)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
Port-channel2 is up, line protocol is up (connected)
  5 minute input rate 2820000 bits/sec, 260 packets/sec
  5 minute output rate 437000 bits/sec, 115 packets/sec

FICPASWSAN2#show int | inc line | drops
Vlan1 is up, line protocol is up
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Vlan102 is up, line protocol is up
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/1 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/2 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 1786
GigabitEthernet0/3 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/4 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 1046
GigabitEthernet0/5 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/6 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 990
GigabitEthernet0/7 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/8 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/9 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/10 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/11 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/12 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/13 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/14 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/15 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/16 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/17 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/18 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/19 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/20 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/21 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/22 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/23 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/24 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/25 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/26 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/27 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet0/28 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Port-channel2 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

FICPASWSAN2#show int | inc line | rate
Vlan1 is up, line protocol is up
  5 minute input rate 1000 bits/sec, 2 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
Vlan102 is up, line protocol is up
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/1 is up, line protocol is up (connected)
  5 minute input rate 3000 bits/sec, 1 packets/sec
  5 minute output rate 3000 bits/sec, 1 packets/sec
GigabitEthernet0/2 is up, line protocol is up (connected)
  5 minute input rate 19853000 bits/sec, 2053 packets/sec
  5 minute output rate 7125000 bits/sec, 1516 packets/sec
GigabitEthernet0/3 is up, line protocol is up (connected)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/4 is up, line protocol is up (connected)
  5 minute input rate 4083000 bits/sec, 946 packets/sec
  5 minute output rate 12587000 bits/sec, 1307 packets/sec
GigabitEthernet0/5 is up, line protocol is up (connected)
  5 minute input rate 265000 bits/sec, 19 packets/sec
  5 minute output rate 24000 bits/sec, 18 packets/sec
GigabitEthernet0/6 is up, line protocol is up (connected)
  5 minute input rate 7448000 bits/sec, 1148 packets/sec
  5 minute output rate 11658000 bits/sec, 1325 packets/sec
GigabitEthernet0/7 is up, line protocol is up (connected)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 2000 bits/sec, 2 packets/sec
GigabitEthernet0/8 is down, line protocol is down (notconnect)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/9 is up, line protocol is up (connected)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/10 is down, line protocol is down (notconnect)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/11 is down, line protocol is down (notconnect)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/12 is down, line protocol is down (notconnect)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/13 is up, line protocol is up (connected)
  5 minute input rate 1276000 bits/sec, 105 packets/sec
  5 minute output rate 101000 bits/sec, 37 packets/sec
GigabitEthernet0/14 is up, line protocol is up (connected)
  5 minute input rate 803000 bits/sec, 62 packets/sec
  5 minute output rate 219000 bits/sec, 12 packets/sec
GigabitEthernet0/15 is up, line protocol is up (connected)
  5 minute input rate 376000 bits/sec, 26 packets/sec
  5 minute output rate 4000 bits/sec, 5 packets/sec
GigabitEthernet0/16 is up, line protocol is up (connected)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/17 is up, line protocol is up (connected)
  5 minute input rate 478000 bits/sec, 102 packets/sec
  5 minute output rate 3088000 bits/sec, 267 packets/sec
GigabitEthernet0/18 is down, line protocol is down (notconnect)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/19 is up, line protocol is up (connected)
  5 minute input rate 23000 bits/sec, 8 packets/sec
  5 minute output rate 174000 bits/sec, 13 packets/sec
GigabitEthernet0/20 is down, line protocol is down (notconnect)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/21 is up, line protocol is up (connected)
  5 minute input rate 68000 bits/sec, 40 packets/sec
  5 minute output rate 1477000 bits/sec, 123 packets/sec
GigabitEthernet0/22 is up, line protocol is up (connected)
  5 minute input rate 161000 bits/sec, 21 packets/sec
  5 minute output rate 276000 bits/sec, 79 packets/sec
GigabitEthernet0/23 is up, line protocol is up (connected)
  5 minute input rate 2130000 bits/sec, 180 packets/sec
  5 minute output rate 238000 bits/sec, 19 packets/sec
GigabitEthernet0/24 is down, line protocol is down (notconnect)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/25 is down, line protocol is down (notconnect)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/26 is down, line protocol is down (notconnect)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/27 is down, line protocol is down (notconnect)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
GigabitEthernet0/28 is down, line protocol is down (notconnect)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
Port-channel2 is up, line protocol is up (connected)
  5 minute input rate 2313000 bits/sec, 223 packets/sec
  5 minute output rate 532000 bits/sec, 109 packets/sec

The output drops are the number of packets dropped because the output queue is full. A
common cause of this might be traffic from a high bandwidth link being switched to a lower
bandwidth link or traffic from multiple inbound links being switched to a single outbound
link. For example, if a large amount of bursty traffic comes in on a gigabit interface and
is switched out to a 100Mbps interface, this might cause output drops to increment on the
100Mbps interface. This is because the output queue on that interface is overwhelmed by
the excess traffic due to the speed mismatch between the inbound and outbound bandwidths.

What kind of traffic are you running?

Do you have havey multicast traffic running across these links?

Please change the "load-interval 30 " to minimal under the switch interfaces to get more granular input and
output rate from the interfaces.

What IOS version are you running and which switch?

It might also be a bug but need to check the details just in case.

Cheers,
-amit singh

Yes, however, both the interface and the nic is 1000Mbit, and

the setup is pretty simple - 2 3560 switches, 3 ports on each (in question), all on the same VLAN, 1 port on each switch connects to one NIC on each of the 3 SAN nodes

The interfaces are showing less than 5% utilization.

Not sure of the traffic except that it's replication traffic from CentOS using GPFS filesystems.

show version is posted above.

can you explain what you mean about changing the load-interval 30?

Thanks

Gurpreet Kochar
Level 1
Level 1

The output drops are not incrementing rapidly, it gives me an impression that they are being caused by bursty traffic

The interface monitors the average traffic rate over the interval of 5 minutes.

What if the burst occures during the few seconds of those 5 minutes over which the traffic rate was measured, the interface would never be able to show you the exact details

The traffic may peak for lets say 3-5 seconds and exceed the threshold of the inteface and at that moment the traffic is dropped.

By reducing the load-interval to 30seconds you are monitoring the traffic rate on the interface over an interval of 30secs rather than 5 minutes.

This would give you a better picture but not a precise view.