02-14-2017 09:33 AM - edited 03-08-2019 09:20 AM
I have 2 Nexus 5Ks that drop ping packets when pinging between them at 70-90% drop rate.
The ping drop rate also occurs for the following:
- Between access layer switches and 5Ks
- Between end user devices/servers and 5K
- Between PC and HSRP gateway address on 5K
No drops occur for following:
- From switch to switch passing through Nexus
- From PC to server passing though Nexus
- From PC to device on same network of HSRP gateway address
I am aware of the policing issue on the 5Ks but I don't think that is the case here. This just started happening recently and we've had the 5Ks for over 2 years now.
Any t-shooting or debug command ideas?
02-14-2017 12:02 PM
Hello,
what is the uptime of the switches ? Often slow response times are related to long uptimes...
In order to make sure that the issue is NOT related to the default CoPP policy, can you see any increases in the 'violated' counter for ICMP traffic ?
show policy-map interface control-plane
02-14-2017 12:26 PM
Hi,
Uptime is currently 240 days
I ran the sh command and found this:
class-map copp-system-class-icmp-echo (match-any)
match protocol icmp_echo
police cir 64 kbps , bc 3600000 bytes
conformed 150309163572 bytes; action: transmit
violated 4840309833 bytes;
02-14-2017 12:33 PM
Hello,
are the counters increasing when you ping (and have packet loss) ? 240 days is actually not that long, if you have a service window though, you might want to reboot...
02-14-2017 12:43 PM
They are definitely increasing even without me pinging but these 5Ks are also using snmp for a Solarwinds server
02-14-2017 12:52 PM
Hello,
you might want to change the default policer in the class map for ICMP traffic and check if that has any effect on your ping responses. If it does, you know it is the CoPP.
02-14-2017 01:02 PM
I did try to lower the ping packet size to something lower than 64 and still had the drops. Would that be a similar test?
02-14-2017 01:19 PM
You need to change the policing rate to effectively test...
02-15-2017 11:21 AM
Just an update. I did some ICMP debugging and found that the pings are being redirected to a bogus gateway IP and that is most likely causing the ping fails. We did some research and found this bug and this is exactly what we are experiencing. The gateway address showing up in the debug messages happens to be one digit off. So instead of it forwarding packets to something like 10.1.1.1, it's forwarding to 10.0.1.1. It's like it's nulling out the 2nd octet.
https://quickview.cloudapps.cisco.com/quickview/bug/CSCus28969
02-15-2017 12:25 PM
Hello,
very interesting indeed, and kind of a weird bug...
Either way, if that is the solution, good that you have found it...
11-21-2021 04:11 AM
Hello,
How did you finally solve this issue?
Best regards.
04-11-2024 05:37 AM
It's the same as my problem.
So I upgraded to a higher OS. But it didn't work out. I'm curious.
04-11-2024 05:40 AM
Make new post it better
MHM
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide