why packet still drop when there is less traffic over the circuit

[Herman2018]

Hi, we have one WAN p2p circuit , the bandwidth is 50Mb, the actual traffic is less than 30Mb, we turn on the traffic shaping at both ends and define a default class-map for all the trafffic. But I still see there is packet drop on the interface stats. Can anyone pls help advise? thanks. 

[balaji.bandi]

This may caused for many reasons - until we see your config and interface output drops (can not guessing what is wrong ?)

1. What Router or device (both the side)

2. what IOS code running ?

3. what is the QoS configuration

4. can you post show interface x/x both the sides ?

5. what P2P Link ( Leased Lines - Fiber - what delivery ?)

some troubleshoot steps to guide you :

https://community.cisco.com/t5/networking-knowledge-base/qos-troubleshooting-and-how-to-collect-useful-logs/ta-p/3147563

[Herman2018]

Thanks @balaji.bandi  for your reply. Following are the answers to your questions. 

1.What Router or device (both the side)

   A end is ASR 1001-x

  B end is 9300-48P

2. what IOS code running ?

 9300-48p OS is XE 17.03.06

 ASR 1001-x OS is XE 3.16.05.s 

3. what is the QoS configuration

Service-policy output: xxx-Traffic-Shaping

Class-map: class-default (match-any)
22807627 packets, 23946342633 bytes
5 minute offered rate 1360000 bps, drop rate 0000 bps
Match: any
Queueing
queue limit 400 packets
(queue depth/total drops/no-buffer drops) 0/38296/0
(pkts output/bytes output) 22217515/23854985771
shape (average) cir 100000000, bc 400000, be 400000
target shape rate 100000000

4. can you post show interface x/x both the sides ?

A end: ASR1001-X

interface GigabitEthernetx/x/x

ip address x.x.x.x,,,,,,
no ip redirects

ip ospf network point-to-point
ip ospf 1 area xxx
ip ospf cost xxx
no negotiation auto
mpls ip


service-policy output xxx-Traffic-Shaping

B-end : 9300

interface GigabitEthernetx/x/x
no switchport
ip address xxxxxx
no ip redirects

i
ip ospf network point-to-point

ip ospf 1 area xxx
ip ospf cost xxx
speed 100
duplex full
mpls ip

service-policy output xxOffice-100Mb

5. what P2P Link ( Leased Lines - Fiber - what delivery ?)

P2P ethernet lease line,but it should be shared with other clients 

[Herman2018]

I use another WAN for the example. the circuit bandwidth is 100Mb. the traffic also dropped at ASR router interface when there is a bit traffic. 

[balaji.bandi]

not the QOS output - show us the 3. what is the QoS configuration

not the interface config - show interface x/x (show you mentioned packet drops ?)

check on Cat 9300 Qos outputs :

https://www.cisco.com/c/en/us/support/docs/switches/catalyst-9300-switch/216236-troubleshoot-output-drops-on-catalyst-90.html

[Herman2018]

Thanks @balaji.bandi , the packet dropped at ASR interface, not 9300. 

the qos config on ASR router.  ASR is MPLS PE router, interface is mpls interface, connected to another PE router. 

policy-map xxx-traffic-Shaping
class class-default
shape average 100000000

[Herman2018]

sorry, @balaji.bandi , the output of sh int x/x/x is shown as below.

GigabitEthernetx/x/x  is up, line protocol is up
Hardware is BUILT-IN-2T+6X1GE, address is 

MTU 1xxx bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive not supported
Full Duplex, 1000Mbps, link type is force-up, media type is LX
output flow-control is on, input flow-control is on
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:01:27, output hang never
Last clearing of "show interface" counters 1d11h
Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 38296
Queueing strategy: Class-based queueing
Output queue: 0/40 (size/max)
5 minute input rate 1438000 bits/sec, 161 packets/sec
5 minute output rate 1354000 bits/sec, 167 packets/sec
27448101 packets input, 26843160247 bytes, 0 no buffer
Received 0 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 76484 multicast, 0 pause input
24126775 packets output, 25340783575 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
2350 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out

[balaji.bandi]

i would suggested to clear the counters and see how fast that drops increase (in terms of output drops)

2350 unknown protocol drops  - i also see some unknown drops ?

You said B end  -B end is 9300-48P ( so you do not see any DROPS that side ?)

[Herman2018]

Thanks @balaji.bandi  ,yes B-end no packet drop. only see the packet drop at A-end which is ASR router. The interface of ASR router is fiber link, the link speed is 1G, but the p2p circuit is 100M, so we use traffic shaping. The actual test traffic is very little only, around 4Mbps. Is it possible that the interface speed is 1G, but the circuit is only 100Mb, so the interface buffer was full at certain time when a burst traffics outgoing this interface? pls advise, thanks. 

[paul driver]

Hello
Do you have fair-queuing enabled, Can you share the qos policy please?.

[Herman2018]

Thanks @paul driver for your advice. The packet is dropped at ASR router interface. No packet dropped at 9300 interface. both devices are mpls PE routers. the qos config on 9300. is same as ASR, we just turn on traffic shaping without defining any other class-map for voice and other traffic, all traffics are under default class-map. please advise, thanks. 

[Joseph W. Doherty]

Your ". . . the actual traffic is less than 30Mb . . ." is usually some average utilization over some time period (in your case 5 minutes?) while drops happen in millisecond time intervals.  I.e. often this isn't unusual.

Also in your case (assuming my eyeball math is correct) your overall drop percentage is less than .2%.  Which may indicate it's not an issue.

If feel you need more help advice, please try to specify for what purpose or need.

[Herman2018]

@Joseph W. Doherty , thanks for your advice. Actually I want to know why the traffic will drop, because the peak traffic transmit rate should be much less than bandwidth, and the traffic shaping is enabled, so it should be better because there are some buffer for outgoing traffic queue. 

This is happening on our network, I am trying to figure out what is wrong or not. 

[Joseph W. Doherty]

It's unclear, to me, your topology and where QoS policies are implemented and where your drops are occurring.

To me, mention of a peak rate is a "red flag" unless it's controlled by a physical interface bandwidth rate.

For example, a 30 Mbps "rate" on 100 Mbps bandwidth really means for some measured time period only 30% of the bandwidth capacity was used but we don't know how it was used, and it's even possible to have a huge number of drops concurrently with such usage.

Consider an offered rate of gig for 30 seconds to 100 Mbps, over a 5 minute interval.  What happens if there's no egress queue?  How much egress queue buffer space do you need to guarantee no drops?  What happens if you only have half the prior question's buffer space?  What happens if gig is offered for 60 seconds?

If you can answer prior questions, we can proceed to the more complex that concerns shaping.

[Herman2018]

thanks @Joseph W. Doherty . it is just a point-to-point circuit. I think we do not need to consider the network topology. is it possible that the interface speed is 1G, but circuit is 100M with traffic shaping, so when burst was outgoing the interface, the buffer was full and cause some drop?