07-20-2015 04:22 PM - edited 03-05-2019 01:54 AM
Hi guys,
I have a few routers in a customers network that continuesly having output drops. The QoS policy applied shows no drops at all. I have cleared the counters and monitored it, it just keeps creeping up. Its not over utlization as I have netflow enabled and don't see it reaching full capacity, so QoS dont even come into the picture. Below in the output you can see the drop count after a week and no drops in QoS classes what so ever....
ORADMORT01#sh inter g0/0
GigabitEthernet0/0 is up, line protocol is up
Hardware is MV96340 Ethernet, address is 001b.5444.50f8 (bia 001b.5444.50f8)
Description: Telstra GWIP FNN:N6010599R VLAN:N6046175R
Internet address is 10.230.14.158/30
MTU 1500 bytes, BW 4096 Kbit/sec, DLY 100 usec,
reliability 255/255, txload 12/255, rxload 91/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, media type is T
output flow-control is XON, input flow-control is XON
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters 1w1d
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 77998
Queueing strategy: Class-based queueing
Output queue: 0/1000/0 (size/max total/drops)
30 second input rate 1463000 bits/sec, 292 packets/sec
30 second output rate 193000 bits/sec, 150 packets/sec
87137921 packets input, 1932383342 bytes, 0 no buffer
Received 196128 broadcasts, 2 runts, 0 giants, 0 throttles
4 input errors, 1 CRC, 0 frame, 0 overrun, 1 ignored
0 watchdog, 0 multicast, 0 pause input
0 input packets with dribble condition detected
37810913 packets output, 1780878043 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out
ORADMORT01#sh polic
ORADMORT01#sh policy-map inte
ORADMORT01#sh policy-map interface g0/0
GigabitEthernet0/0
Service-policy output: WAN_EGRESS
Class-map: class-default (match-any)
37657378 packets, 10198904105 bytes
30 second offered rate 239000 bps, drop rate 0 bps
Match: any
Queueing
queue limit 64 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 37855428/10411003452
shape (average) cir 4096000, bc 16384, be 16384
target shape rate 4096000
Service-policy : WAN_CONGESTION_MNGT
queue stats for all priority classes:
Queueing
queue limit 64 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 4257255/910981902
Class-map: VOICE_BEARER (match-any)
4257255 packets, 910981902 bytes
30 second offered rate 0 bps, drop rate 0 bps
Match: ip dscp ef (46)
4257255 packets, 910981902 bytes
30 second rate 0 bps
Priority: 900 kbps, burst bytes 22500, b/w exceed drops: 0
police:
rate 900000 bps, burst 28125 bytes, peak-burst 1500 bytes
conformed 4257255 packets, 910981902 bytes; actions:
transmit
exceeded 0 packets, 0 bytes; actions:
drop
violated 0 packets, 0 bytes; actions:
drop
conformed 0 bps, exceed 0 bps, violate 0 bps
Class-map: VOICE_SIGNALLING (match-any)
2003973 packets, 132821144 bytes
30 second offered rate 1000 bps, drop rate 0 bps
Match: ip dscp cs3 (24)
2003973 packets, 132821144 bytes
30 second rate 1000 bps
Queueing
queue limit 64 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 2003973/127387965
bandwidth 16 kbps
Class-map: DRIVES (match-any)
0 packets, 0 bytes
30 second offered rate 0 bps, drop rate 0 bps
Match: ip dscp af32 (28)
0 packets, 0 bytes
30 second rate 0 bps
Queueing
queue limit 64 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 0/0
bandwidth 100 kbps
Class-map: CITRIX (match-any)
0 packets, 0 bytes
30 second offered rate 0 bps, drop rate 0 bps
Match: ip dscp af31 (26)
0 packets, 0 bytes
30 second rate 0 bps
Queueing
queue limit 64 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 0/0
bandwidth 150 kbps
Class-map: NETWORK_MANAGEMENT (match-any)
363851 packets, 255045339 bytes
30 second offered rate 2000 bps, drop rate 0 bps
Match: access-group name NETWORK_MANAGEMENT_TRAFFIC
363849 packets, 255044626 bytes
30 second rate 2000 bps
Queueing
queue limit 64 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 285761/78583417
bandwidth 64 kbps
QoS Set
dscp cs2
Packets marked 363851
police:
rate 64000 bps, burst 2000 bytes, peak-burst 1500 bytes
conformed 285761 packets, 78583417 bytes; actions:
transmit
exceeded 21402 packets, 9652670 bytes; actions:
drop
violated 56688 packets, 30309197 bytes; actions:
drop
conformed 2000 bps, exceed 0 bps, violate 0 bps
Class-map: ROUTING_PROTOCOLS (match-any)
76848 packets, 18608605 bytes
30 second offered rate 0 bps, drop rate 0 bps
Match: ip dscp cs6 (48)
76848 packets, 18608605 bytes
30 second rate 0 bps
Queueing
queue limit 64 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 76848/4967660
bandwidth 24 kbps
Class-map: VIDEO_BEARER (match-any)
0 packets, 0 bytes
30 second offered rate 0 bps, drop rate 0 bps
Match: ip dscp af41 (34)
0 packets, 0 bytes
30 second rate 0 bps
police:
rate 1024000 bps, burst 102400 bytes, peak-burst 1500 bytes
conformed 0 packets, 0 bytes; actions:
transmit
exceeded 0 packets, 0 bytes; actions:
drop
violated 0 packets, 0 bytes; actions:
drop
conformed 0 bps, exceed 0 bps, violate 0 bps
Priority: 1024 kbps, burst bytes 25600, b/w exceed drops: 0
Class-map: class-default (match-any)
30955453 packets, 8881447608 bytes
30 second offered rate 237000 bps, drop rate 0 bps
Match: any
queue limit 1024 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 31231591/9289082508
ORADMORT01#
Software version running is 112.4(24)T3 and its a 2821 router. I thought it might be a bug, but cant find anything there either.
I reviewed a fair number of posts online, they all point to QOS, but this is not QoS related.
Thanx for your input, hopefully I'm not the only one with this problem....
H
Solved! Go to Solution.
07-21-2015 05:22 AM
Disclaimer
The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.
Liability Disclaimer
In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.
Posting
". . . no drops in QoS classes what so ever....":
Class-map: NETWORK_MANAGEMENT (match-any)
363851 packets, 255045339 bytes
30 second offered rate 2000 bps, drop rate 0 bps
Match: access-group name NETWORK_MANAGEMENT_TRAFFIC
363849 packets, 255044626 bytes
30 second rate 2000 bps
Queueing
queue limit 64 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 285761/78583417
bandwidth 64 kbps
QoS Set
dscp cs2
Packets marked 363851
police:
rate 64000 bps, burst 2000 bytes, peak-burst 1500 bytes
conformed 285761 packets, 78583417 bytes; actions:
transmit
exceeded 21402 packets, 9652670 bytes; actions:
drop
violated 56688 packets, 30309197 bytes; actions:
drop
conformed 2000 bps, exceed 0 bps, violate 0 bps
Compare the sum of this policer's drops to the total drops listed for the interface.
07-20-2015 08:25 PM
Hello,
One thing you can try is disabling flow control on the interface and see if that makes any difference. Disabling flow control will help us understand if there is any kind of congestion happening or not.
Would like to know if the drops are incrementing even after removing the QoS policy. If you haven't verified that, please do.
Another action plan i would suggest is, capturing the below output:
- show ip traffic | in fragment
- show interface gi 0/0 | in rate|drop
Capture the above output multiple times to see if the drops are incrementing at the rate at which the router is unable to fragment some packets. If a large packet is received on a different interface and tries to egress here the packet may have to be fragmented. If the DF bit is set on the same packet then we cannot fragment it and will have to drop the packet. This would also increment the couldn't fragment counter of the show ip traffic command as well.
Hope this helps.
Vinit
07-21-2015 08:14 PM
Thank you guys for your help, especially Joseph W. Doherty. I don't know how i missed that! my goodness, thank you!
I have been scratching my head and we have a few (around 80) routers with this same issue, I was thinking I would have had to upgrade the software on all of them.....
Thank you again!
07-21-2015 05:22 AM
Disclaimer
The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.
Liability Disclaimer
In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.
Posting
". . . no drops in QoS classes what so ever....":
Class-map: NETWORK_MANAGEMENT (match-any)
363851 packets, 255045339 bytes
30 second offered rate 2000 bps, drop rate 0 bps
Match: access-group name NETWORK_MANAGEMENT_TRAFFIC
363849 packets, 255044626 bytes
30 second rate 2000 bps
Queueing
queue limit 64 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 285761/78583417
bandwidth 64 kbps
QoS Set
dscp cs2
Packets marked 363851
police:
rate 64000 bps, burst 2000 bytes, peak-burst 1500 bytes
conformed 285761 packets, 78583417 bytes; actions:
transmit
exceeded 21402 packets, 9652670 bytes; actions:
drop
violated 56688 packets, 30309197 bytes; actions:
drop
conformed 2000 bps, exceed 0 bps, violate 0 bps
Compare the sum of this policer's drops to the total drops listed for the interface.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide