08-22-2018 04:26 AM - edited 03-05-2019 10:52 AM
I've had a new ASR9001 installed for a week and I've just noticed (via Solarwinds) that it's showing lots of input drops on our uplink.
We don't have any issues with customers experiencing packet loss or slow speeds so it doesn't 'seem' to be actually causing issues but I would obviously like to pin down what is causing the input drops.
The below shows the interface itself which counters were reset 7 minutes ago and you can see quite a few input drops already:
RP/0/RSP0/CPU0:Router#show interfaces tenGigE 0/0/2/3
Wed Aug 22 12:12:12.829 UTC
TenGigE0/0/2/3 is up, line protocol is up
Interface state transitions: 1
Hardware is TenGigE, address is b028.8055.ca50 (bia b028.8055.ca50)
Layer 1 Transport Mode is LAN
Description: "xxxxxxxxxxxxxxxxxxxxxx"
Internet address is xxx.xxx.xxx.xxx/xx
MTU 1514 bytes, BW 10000000 Kbit (Max: 10000000 Kbit)
reliability 255/255, txload 49/255, rxload 151/255
Encapsulation ARPA,
Full-duplex, 10000Mb/s, LR, link type is force-up
output flow control is off, input flow control is off
Carrier delay (up) is 10 msec
loopback not set,
Last link flapped 1w2d
ARP type ARPA, ARP timeout 04:00:00
Last input 00:00:00, output 00:00:00
Last clearing of "show interface" counters 00:07:00
30 second input rate 5953476000 bits/sec, 762688 packets/sec
30 second output rate 1952715000 bits/sec, 464959 packets/sec
320650333 packets input, 315760887793 bytes, 354718 total input drops
16 drops for unrecognized upper-level protocol
Received 78268 broadcast packets, 293636 multicast packets
0 runts, 0 giants, 0 throttles, 0 parity
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
198383805 packets output, 104002355955 bytes, 0 total output drops
Output 0 broadcast packets, 196 multicast packets
0 output errors, 0 underruns, 0 applique, 0 resets
0 output buffer failures, 0 output buffers swapped out
0 carrier transitions
The below shows the drops:
RP/0/RSP0/CPU0:Router#sh controllers np counters np1 location 0/0/CPU0
Wed Aug 22 12:15:22.584 UTC
Node: 0/0/CPU0:
----------------------------------------------------------------
Show global stats counters for NP1, revision v2
Last clearing of counters for this NP: NEVER
Read 76 non-zero NP counters:
Offset Counter FrameValue Rate (pps)
-------------------------------------------------------------------------------------
16 MDF_TX_LC_CPU 18308133632 201002
17 MDF_TX_WIRE 214134988509 479045
21 MDF_TX_FABRIC 323529555619 762199
33 PARSE_FAB_RECEIVE_CNT 214128071374 479036
37 PARSE_INTR_RECEIVE_CNT 534647264 521
41 PARSE_INJ_RECEIVE_CNT 7567660 10
45 PARSE_ENET_RECEIVE_CNT 325670279805 763986
49 PARSE_TM_LOOP_RECEIVE_CNT 4968835 5
53 PARSE_TOP_LOOP_RECEIVE_CNT 17535109549 200077
63 DBG_RSV_EP_L_RSV_ING_L3_IFIB 605830233 723
64 DBG_RSV_EP_L_RSV_ING_L3_IFIB_MATCH 605827620 723
65 DBG_RSV_EP_L_RSV_ING_L3_IFIB_PUNT_LOCAL 605827620 723
67 DBG_RSV_EP_L_RSV_ING_L3_RSLTS_MATCH 605827585 723
68 DBG_RSV_EP_L_RSV_ING_PUNT 42200673506 761333
124 RSV_DROP_IPV4_DROP_NULL_RTE 683369167 813
134 RSV_DROP_IPV4_RXADJ_DROP 3 0
157 RSV_DROP_ACL_DENY 10005965 13
250 RSV_DROP_IFIB_TTL_CHECK 35 0
258 RSV_DROP_IN_L3_NOT_MYMAC 683913374 48
419 RSV_SNF_SAMPLE_PKT 41204063229 759535
420 RSV_SNF_IPV4 41428268793 760409
421 RSV_SNF_ING_PUNT 41428268797 760409
514 MDF_PIPE_LPBK 17535110121 200077
532 MDF_PUNT_POLICE_DROP 23893174422 560332
539 MODIFY_PUNT_REASON_MISS_DROP 4 0
586 MODIFY_PUNT_NF_MSG 17534812948 200077
587 MODIFY_PUNT_NF_IPV4 17534812907 200077
599 MDF_LB_TM_LPBK0 4968834 5
774 PUNT_CDP 13792 0
776 PUNT_ARP 154096268 185
777 PUNT_ARP_EXCD 42 0
790 PUNT_DIAGS 17160 0
822 PUNT_IPv4_OPTIONS 26 0
832 PUNT_FOR_ICMP 399 0
833 PUNT_FOR_ICMP_EXCD 27 0
840 PUNT_TTL_EXCEEDED 414965 1
850 PUNT_ADJ 549378 1
852 PUNT_UNKNOWN_IFIB 2613 0
854 PUNT_ACL_DENY 401189 1
862 PUNT_IPV6_HOP_BY_HOP 56550 0
864 PUNT_IPV6_TTL_ERROR 132802 0
866 PUNT_IPV6_PLU_PUNT 11 0
876 PUNT_IPV6_SRC_LINK_LOCAL 2093 0
902 PUNT_STATISTICS 9060942 11
908 PUNT_NETIO_RP_TO_LC_CPU 130295 0
936 PUNT_NETFLOW 17535110624 200077
937 PUNT_NETFLOW_EXCD 23893160511 560332
946 PUNT_IPV6_ADJ_NULL_RTE 868494 0
947 PUNT_IPV6_ADJ_NULL_RTE_EXCD 10624 0
958 PUNT_LLDP 1454116 2
1100 DROP_FRM_CRC_ERR_XAUI9 305 0
1101 DROP_FRM_FRM_ERR_XAUI9 89 0
1229 PARSE_FRM_ERR_CNT 394 0
1233 PRS_HEALTH_MON 4968839 5
1242 INTR_FRAME_TYPE_7 9060944 11
1267 PARSE_ING_IPV6_LINK_LOCAL 4284558 5
1271 PARSE_RSP_INJ_FAB_CNT 18935806 23
1275 PARSE_EGR_INJ_PKT_TYP_UNKNOWN 130295 0
1276 PARSE_EGR_INJ_PKT_TYP_IPV4 13214834 16
1277 PARSE_EGR_INJ_PKT_TYP_IPV6 5450023 6
1280 PARSE_EGR_INJ_PKT_TYP_IPV4_PREROUTE 140654 0
1287 PARSE_LC_INJ_FAB_CNT 291 0
1288 PARSE_LC_INJ_PORT_CNT 7550209 9
1289 PARSE_LC_INJ_DIAGS_CNT 17160 0
1305 PARSE_DROP_IN_UIDB_TCAM_MISS 55 0
1307 PARSE_DROP_IN_UIDB_DOWN 9365 0
1314 PARSE_DROP_IPV4_CHECKSUM_ERROR 1 0
1316 PARSE_IPV4_SANITY_ADDR_CHK_FAILURES 33433 0
1351 PARSE_DROP_IPV6_MCAST_NOT_ENABLED 53187 0
1412 PARSE_LLDP 1454116 2
1413 UNKNOWN_L2_ON_L3_DISCARD 11563 0
1432 PARSE_UDLD 13792 0
1454 PARSE_FAST_DISCARD_LOW_PRIORITY_DROP_0 39741 1
1455 PARSE_FAST_DISCARD_LOW_PRIORITY_DROP_1 39591 1
1456 PARSE_FAST_DISCARD_LOW_PRIORITY_DROP_2 39711 1
1457 PARSE_FAST_DISCARD_LOW_PRIORITY_DROP_3 39579 1
If however I look at the controller for the drops I don't see them:
RP/0/RSP0/CPU0:Transit2#show controllers tenGigE0/0/2/3 stats
Wed Aug 22 12:22:47.490 UTC
Statistics for interface TenGigE0/0/2/3 (cached values):
Ingress:
Input total bytes = 794183557675
Input good bytes = 794183557675
Input total packets = 804072601
Input 802.1Q frames = 0
Input pause frames = 0
Input pkts 64 bytes = 82658170
Input pkts 65-127 bytes = 93208823
Input pkts 128-255 bytes = 64982073
Input pkts 256-511 bytes = 34177709
Input pkts 512-1023 bytes = 28984056
Input pkts 1024-1518 bytes = 500061777
Input pkts 1519-Max bytes = 0
Input good pkts = 804072601
Input unicast pkts = 803134804
Input multicast pkts = 738082
Input broadcast pkts = 199715
Input drop overrun = 0
Input drop abort = 0
Input drop invalid VLAN = 0
Input drop invalid DMAC = 0
Input drop invalid encap = 0
Input drop other = 0
Input error giant = 0
Input error runt = 0
Input error jabbers = 0
Input error fragments = 0
Input error CRC = 0
Input error collisions = 0
Input error symbol = 0
Input error other = 0
Input MIB giant = 0
Input MIB jabber = 0
Input MIB CRC = 0
Egress:
Output total bytes = 259526856170
Output good bytes = 259526856170
Output total packets = 495869390
Output 802.1Q frames = 0
Output pause frames = 0
Output pkts 64 bytes = 130622650
Output pkts 65-127 bytes = 141351332
Output pkts 128-255 bytes = 43833946
Output pkts 256-511 bytes = 14967400
Output pkts 512-1023 bytes = 17500366
Output pkts 1024-1518 bytes = 147593694
Output pkts 1519-Max bytes = 0
Output good pkts = 495869390
Output unicast pkts = 495868888
Output multicast pkts = 502
Output broadcast pkts = 0
Output drop underrun = 0
Output drop abort = 0
Output drop other = 0
Output error other = 0
I've removed the ACL from the interface to see if the input drops still increase and it doesn't make a difference. Could this be a bug in the reporting of input drops? I'm on 6.4.1
Thanks
08-22-2018 04:57 AM
Hello,
the bug below could apply:
CSCux88695 - Possible incorrect interface input drop
Symptom: Excessive input drop but there is no match in 'show controller interface' and 'show controller np'.
Conditions: n/a
Workaround: n/a
Further Problem Description: n/a
Known affected releases: 4.3.4/6.1.0
08-22-2018 05:17 AM
Thanks. That bug seems to affect bundled interfaces however and this is just a standalone interface. I've seen other similar bugs effecting HSRP/VRRP but again this shouldn't be affected as we are not running those.
Could be another bug of course :/
08-22-2018 05:44 AM
Did you google the bug id ? If so, you get a different result than that from the Cisco bug search tool. The tool result mentions onfividual interfaces...
08-22-2018 07:50 AM
Thanks again. I've google searched for the bug id's CSCul84303 and CSCux88695 but I can't see where it mentions it affecting individual interfaces? These also seem to be resolved by 6.4.1.
If it is a bug then great it means I can just schedule an upgrade but I'd just like to be sure first :)
08-22-2018 10:06 AM
When you click on the bug on Cisco.com, you get redirected to a different one that mentions the bundled interfaces. What I posted earlier is a cut and paste from the search tool...
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide