Different latency between traceroute and ping

[Alek5942]

Hello community,

Recently, I observed very strange situation. Users were complaining about general slowness of internet connection. I performed ping and traceroute from Core switch to 8.8.8.8 and ping latency was very good whereas traceroute showed high latency inside ISP's network. We opened ticket for ISP and after some time issue was resolved. But my question is, how is possible that latency of ping was fine? How is it possible that there is difference between in latency between ping and traceroute? Below is example of output with modified IP addresses:

CoreSwitch#ping 8.8.8.8
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 8.8.8.8, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 5/5/7 ms

CoreSwitch#traceroute 8.8.8.8 numeric
Type escape sequence to abort.
Tracing the route to 8.8.8.8
VRF info: (vrf in name/id, vrf out name/id)
1 192.21.17.225 0 msec 0 msec 0 msec
2 7.19.29.217 1 msec 1 msec 1 msec
3 3.72.12.251 1 msec 1 msec 1 msec
4 * * *
5 12.14.11.18 2993 msec
12.14.11.20 2411 msec
12.14.11.18 2680 msec
6 *
17.29.3.75 2986 msec
18.72.6.162 2560 msec
7 *
17.29.3.1 332 msec
17.29.3.4 980 msec
8 17.29.4.21 1483 msec * *
9 8.8.8.8 1633 msec 1794 msec *
CoreSwitch#

[Giuseppe Larosa]

Hello @Alek5942 ,

thanks for providing feedback on your issue and for the solution provided by ISP.

>> I think, maybe ISP core network has some load sharing / load balancing based on protocol and source IP 

Load balancing is generally based on  an EXOR of source IP less meaningful bits , destination IP  less meaningful bits and a seed value that remains the same until next reload. This is generally Cisco implementation other vendors do similar .

if all traffic downstream the core switch and the core switch itself are NATTed by the FW using PAT yes the source address is the same.

All these considerations usually apply to user transit traffic, the UDP probes of traceroute are process switched on the device where the TTL expires.

We don't know what the ISP has done to solve the issue. They may have found a faulty link or a link in saturation and they have taken corrective actions.

Hope to help

Giuseppe