11-25-2018 02:44 PM - edited 03-05-2019 11:04 AM
Dear All,
Could anyone point me in the right direction to troubleshoot input drops on my ISR4331 Gi0 interface please? The drops are constantly incrementing and the interface stops responding to ping/ssh/snmp after around 1 or 2 hours. A shut/no shut makes it responsive for another while.
I have swapped the cable (it's a 2m cat6 to a nearby Nexus 2k FEX), tried auto neg, removed any access-policies on the line vty. The code version is 16.03.07, but I can't find any bugs. Packet capture is also not supported on the management interface.
4431#sho run inter gi0 ! interface GigabitEthernet0 description OOB-FEX Eth109/1/6 VL806 vrf forwarding Mgmt-intf ip address 10.0.0.1 255.255.255.192 no ip redirects no ip unreachables speed 1000 no negotiation auto end
Here are the errors:
4431#show inter gi0 GigabitEthernet0 is up, line protocol is up Hardware is RP management port, address is 4001.7a35.c85f (bia 4001.7a35.c85f) Description: GESOOBML03 Eth109/1/6 VL806 Internet address is 10.238.203.52/26 MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec, reliability 254/255, txload 1/255, rxload 1/255 Encapsulation ARPA, loopback not set Keepalive set (10 sec) Full Duplex, 1000Mbps, link type is force-up, media type is RJ45 output flow-control is unsupported, input flow-control is unsupported ARP type: ARPA, ARP Timeout 04:00:00 Last input 00:00:03, output 00:00:05, output hang never Last clearing of "show interface" counters never Input queue: 0/75/6156/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: fifo Output queue: 0/40 (size/max) 5 minute input rate 2000 bits/sec, 2 packets/sec 5 minute output rate 0 bits/sec, 0 packets/sec 442568 packets input, 51132110 bytes, 0 no buffer Received 1735 broadcasts (0 IP multicasts) 0 runts, 0 giants, 0 throttles 6156 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored 0 watchdog, 0 multicast, 0 pause input 19497 packets output, 4748654 bytes, 0 underruns 0 output errors, 0 collisions, 4 interface resets 0 unknown protocol drops 0 babbles, 0 late collision, 0 deferred 0 lost carrier, 0 no carrier, 0 pause output 0 output buffer failures, 0 output buffers swapped out
This also doesn't shed much light:
chges-d-svuvu-01#sho inter gi0 switching GigabitEthernet0 GESOOBML03 Eth109/1/6 VL806 Throttle count 0 Drops RP 6165 SP 0 SPD Flushes Fast 0 SSE 0 SPD Aggress Fast 0 SPD Priority Inputs 0 Drops 0 Protocol IP Switching path Pkts In Chars In Pkts Out Chars Out Process 71 3282 8487 866569 Cache misses 0 - - - Fast 0 0 0 0 Auton/SSE 0 0 0 0 Protocol ARP Switching path Pkts In Chars In Pkts Out Chars Out Process 2416 144960 1392 83520 Cache misses 0 - - - Fast 0 0 0 0 Auton/SSE 0 0 0 0 Protocol CDP Switching path Pkts In Chars In Pkts Out Chars Out Process 3073 875805 6900 3026964 Cache misses 0 - - - Fast 0 0 0 0 Auton/SSE 0 0 0 0 Protocol Other Switching path Pkts In Chars In Pkts Out Chars Out Process 0 0 12356 4606910 Cache misses 0 - - - Fast 0 0 0 0 Auton/SSE 0 0 0 0 NOTE: all counts are cumulative and reset only after a reload.
Any suggestions would be greatly appreciated.
James.
11-27-2018 05:45 AM
Hi James.
Did you try to set negotiation auto on both sides at the same time?
BR
Gaston
12-11-2018 03:25 AM
Yes, we tried disabling auto neg at both sides to no avail.
In fact, we recently resolved this issue. I should mention that the upstream device is a Nexus 2k FEX and the HSRP gateway is running on a VPC pair. We noticed this within a debug interface gi0 on the ISR4K:
316292: Dec 3 18:19:31.673 GMT: IOSXE-INJECT: set pak datagramstart(from 0x7sho F77CBFB6B1A to 0x7F77CBFB6B28), datagramsize(from 98 to 84); 316293: Dec 3 18:19:31.674 GMT: IOSXE-INJECT: add L3 inject hdr 316300: Dec 3 18:19:31.678 GMT: IOSXE-INJECT: ARP/ND trigger failed, id 0x1, linktype ip, type incomplete, nexthop defined, reason adj not found 316301: Dec 3 18:19:31.678 GMT: IOSXE-INJECT: simulate cpp forward drop failed packet,: triggering arp/nd, v4 10.238.203.62, adj-id 0x0, intf Gi0 Output intf Gi0 matches debug filer
It looks like the ISR was failing to make an ARP request for the default gateway.
So I added a static entry corresponding to the virtual mac on the HSRP GW:
arp vrf Mgmt-intf 10.0.0.1 0000.0c9f.f001 ARPA
This worked around the issue of the interface going unresponsive every few hours.
We then upgraded from Denali to Everest (without upgrading ROMMON or the CPLD) and removed the static arp entry. The issue has gone away, including all the input errors.
My conclusion from all this is that there's an undocumented bug in Denali which prevents the Gi0 interface from working with Nexus HSRP.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide