cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1341
Views
0
Helpful
4
Replies
Highlighted

output drop issue...

Hi Guys,

I am having a issue where i see huge output drops on trunk .....basically i have a access layer switch which is conected to core , core's trunk is gig \n access layer has 100 meg....instead of seeing errors on 100meg i see it on gig interface....i have host A conected to access layer which polls servers connectwed behind another core switch (say host B) ...every now and then host A idrop packet and send false alert out for unpoll server as down....now i traced path and done ping test from HOST A to HOST B .....from Host A to get to  (PING) servers it takes Max 25 ms for 30000 pings ( min is 1ms, avergae is 2msec)...for HIOSt B to get HOST A min 1msec , average 3msec  and max 100msec for 1000 pings.....i even traced the path back ( mac to port tracing )and HOST hits one main core switch and then destination and HOST B hits two core switches and then HOST A....i see out put drops on GIG trunk of core...

CORE switch trunk

GigabitEthernet3/37 is up, line protocol is up (connected)

  Hardware is C6k 1000Mb 802.3, address is 001e.13dd.e5b4 (bia 001e.13dd.e5b4)

  Description: trunk to it area switch

  MTU 1500 bytes, BW 100000 Kbit, DLY 10 usec,

     reliability 255/255, txload 5/255, rxload 1/255

  Encapsulation ARPA, loopback not set

  Keepalive set (10 sec)

  Full-duplex, 100Mb/s

  input flow-control is off, output flow-control is off

  Clock mode is auto

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input 00:00:34, output 00:00:50, output hang never

  Last clearing of "show interface" counters 1w4d

  Input queue: 0/2000/22/0 (size/max/drops/flushes); Total output drops: 318966450

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 124000 bits/sec, 208 packets/sec

  5 minute output rate 2335000 bits/sec, 844 packets/sec

     423686111 packets input, 75694893840 bytes, 0 no buffer

     Received 771518 broadcasts (707699 multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 11 overrun, 0 ignored

     0 watchdog, 0 multicast, 0 pause input

     0 input packets with dribble condition detected

     2977282761 packets output, 2042066847687 bytes, 0 underruns

     2 output errors, 0 collisions, 0 interface resets

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier, 0 PAUSE output

     0 output buffer failures, 0 output buffers swapped out

SHow interface counter error

Port        Align-Err    FCS-Err   Xmit-Err    Rcv-Err UnderSize OutDiscards

Gi3/37              0          0          2          0         0   318966450

SHow ip traffic on core

IP statistics:
  Rcvd:  154315762 total, 982345507 local destination
         107 format errors, 1704 checksum errors, 65838464 bad hop count
         0 unknown protocol, 22423094 not a gateway
         0 security failures, 0 bad options, 26827831 with options
  Opts:  41092 end, 0 nop, 0 basic security, 0 loose source route
         0 timestamp, 0 extended security, 8 record route
         0 stream ID, 0 strict source route, 26786739 alert, 0 cipso, 0 ump
         0 other
  Frags: 300 reassembled, 0 timeouts, 0 couldn't reassemble
         204750 fragmented, 0 couldn't fragment
  Bcast: 565095059 received, 2084899 sent
  Mcast: 396305142 received, 326552877 sent
  Sent:  304134024 generated, 2119036473 forwarded
  Drop:  272133326 encapsulation failed, 0 unresolved, 0 no adjacency
         0 no route, 0 unicast RPF, 12466 forced drop
  Drop:  81 packets with source IP address zero

ICMP statistics:
  Rcvd: 0 format errors, 33700 checksum errors, 2196 redirects, 346621 unreachable
        13450588 echo, 8187 echo reply, 0 mask requests, 0 mask replies, 0 quench
        0 parameter, 0 timestamp, 0 info request, 0 other
        284 irdp solicitations, 0 irdp advertisements
  Sent: 23639 redirects, 11737 unreachable, 8383 echo, 13450505 echo reply
        0 mask requests, 0 mask replies, 0 quench, 0 timestamp
        0 info reply, 2668767 time exceeded, 0 parameter problem
        0 irdp solicitations, 0 irdp advertisements

BGP statistics:
  Rcvd: 0 total, 0 opens, 0 notifications, 0 updates
        0 keepalives, 0 route-refresh, 0 unrecognized
  Sent: 0 total, 0 opens, 0 notifications, 0 updates
        0 keepalives, 0 route-refresh

PIMv2 statistics: Sent/Received
  Total: 55361949/147580009, 0 checksum errors, 0 format errors
  Registers: 0/0 (0 non-rp, 0 non-sm-group), Register Stops: 0/0,  Hellos: 9646881/9648389
  Join/Prunes: 0/0, Asserts: 45715068/137931620, grafts: 0/0
  Bootstraps: 0/0, Candidate_RP_Advertisements: 0/0
  State-Refresh: 0/0

IGMP statistics: Sent/Received
  Total: 4916042/30310848, Format errors: 0/0, Checksum errors: 0/0
  Host Queries: 4765648/3, Host Reports: 150394/27250088, Host Leaves: 0/1045333
  DVMRP: 0/0, PIM: 0/0

UDP statistics:
  Rcvd: 789512758 total, 81 checksum errors, 68072949 no port
  Sent: 226593422 total, 1337189545 forwarded broadcasts

TCP statistics:
  Rcvd: 1100399 total, 0 checksum errors, 2336 no port
  Sent: 1099626 total

OSPF statistics:
  Rcvd: 0 total, 0 checksum errors
        0 hello, 0 database desc, 0 link state req
        0 link state updates, 0 link state acks

  Sent: 0 total

EGP statistics:
  Rcvd: 0 total, 0 format errors, 0 checksum errors, 0 no listener
  Sent: 0 total

Probe statistics:
  Rcvd: 0 address requests, 0 address replies
        0 proxy name requests, 0 where-is requests, 0 other
  Sent: 0 address requests, 0 address replies (0 proxy)
        0 proxy name replies, 0 where-is replies

IP-EIGRP statistics:
  Rcvd: 0 total
  Sent: 0 total

ARP statistics:
  Rcvd: 476570723 requests, 31382270 replies, 40485 reverse, 0 other
  Sent: 57319280 requests, 80166745 replies (79900818 proxy), 0 reverse
  Drop due to input queue full: 179517

Access layer switch ip traffci

IP statistics:
  Rcvd:  20444807 total, 9491341 local destination
         0 format errors, 0 checksum errors, 10526295 bad hop count
         0 unknown protocol, 4422 not a gateway
         0 security failures, 0 bad options, 6782252 with options
  Opts:  40968 end, 0 nop, 0 basic security, 0 loose source route
         0 timestamp, 0 extended security, 0 record route
         0 stream ID, 0 strict source route, 6741284 alert, 0 cipso, 0 ump
         0 other
  Frags: 5998 reassembled, 1 timeouts, 0 couldn't reassemble
         0 fragmented, 0 couldn't fragment
  Bcast: 1211668 received, 19 sent
  Mcast: 0 received, 0 sent
  Sent:  7814147 generated, 2 forwarded
  Drop:  153 encapsulation failed, 0 unresolved, 0 no adjacency
         0 no route, 0 unicast RPF, 0 forced drop
         0 options denied, 0 source IP address zero

ICMP statistics:
  Rcvd: 0 format errors, 0 checksum errors, 0 redirects, 18470 unreachable
        7071061 echo, 128440 echo reply, 0 mask requests, 0 mask replies, 0 quench
        0 parameter, 0 timestamp, 0 info request, 0 other
        0 irdp solicitations, 0 irdp advertisements
  Sent: 0 redirects, 8 unreachable, 128495 echo, 7071061 echo reply
        0 mask requests, 0 mask replies, 0 quench, 0 timestamp
        0 info reply, 1 time exceeded, 0 parameter problem
        0 irdp solicitations, 0 irdp advertisements

TCP statistics:
  Rcvd: 442950 total, 0 checksum errors, 844 no port
  Sent: 522227 total

UDP statistics:
  Rcvd: 1830400 total, 0 checksum errors, 1202334 no port
  Sent: 92514 total, 2976 forwarded broadcasts

IP-EIGRP statistics:
  Rcvd: 0 total
  Sent: 0 total

ARP statistics:
  Rcvd: 12419723 requests, 13914426 replies, 0 reverse, 0 other
  Sent: 3113 requests, 2782 replies (16 proxy), 0 reverse
  Drop due to input queue full: 0

FastEthernet1/0/1 is up, line protocol is up (connected)

  Hardware is Fast Ethernet, address is 001f.9d4f.e003 (bia 001f.9d4f.e003)

  Description: floorport main

  MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,

     reliability 255/255, txload 1/255, rxload 6/255

  Encapsulation ARPA, loopback not set

  Keepalive set (10 sec)

  Full-duplex, 100Mb/s, media type is 10/100BaseTX

  input flow-control is off, output flow-control is unsupported

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input 00:00:00, output 00:00:00, output hang never

  Last clearing of "show interface" counters 1w5d

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 2674000 bits/sec, 897 packets/sec

  5 minute output rate 128000 bits/sec, 215 packets/sec

     3096311877 packets input, 2122061396037 bytes, 0 no buffer

     Received 324515796 broadcasts (0 multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog, 248145891 multicast, 0 pause input

     0 input packets with dribble condition detected

Can any one advise please ....iam think of upgrading from 100 meg to gig but i still should see these errors on 100 trunk not core....and also all the cores got 10gig connection between each other .....

4 REPLIES 4
Highlighted
VIP Mentor

Hello
Output drops usually indicate congestion onI the link. however from the readouts you have posted don't show any high utilisation on the interfaces.

I do see you have default mtu of 1500 - possibility this needs to higher to support the multiple vlan tags of the dot1q trunks

Do you have any Qos applied on these switches?

Res
Paul

Sent from Cisco Technical Support iPad App



kind regards
Paul

Please rate and mark posts accordingly if you have found any of the information provided useful.
It will hopefully assist others with similar issues in the future
Highlighted

hi paul,

thanks for your reply, yes auto qos has been configured ....as far as utilization is concerned access layer switch has high voulme of traffic during peak hours, mind out put is from ooh.... but y dnt i see any drops on 100 meg trunk? ....

Highlighted

Ok, output drops means that your interface is pushing as much data as it can but the remote end device is sending back a PAUSE frame.

So the data that the switch cannot deliver goes into a bucket or buffer.  This buffer gets full and this is why you are seeing output drops.

Now, the question is WHERE is the fault.

One easy test is to bring down your interface speed to 10 Mbps and see if the output drops stops incrementing.  If it does, then I'd recommend you look at the remote end. 

Highlighted

thanks leo,

my host A is trying to talk to host B.... host A is connected to access layer switch which connects to core A ... host A DG points to hsrp ip for vlan configured on core A... core a send all traffic to coreB where host B is ... so if i see output drops on core A trunk ( trunk between host A access layer switch and cor A) this means core A sends but all traffic to Core B but due to congestion Core B cant take all frames and core B trunk puts it in buffern then drop... am i right? ... all the three core's have trunks configured two cores areusing hsrp third is datacenter vss ...lately more 48 port gig switches have been added to network n uplinks arenot increased they are only 10 gigs....

Content for Community-Ad