Traceroute latency

[Jonathancert_2]

It takes 1 minute and 50 seconds to complete a "traceroute" from my branch to company headquarters.  Users have been complaining about slow connection, just thought they were whinning. "tracert" from desktop computer response was normal, so i pressed on with other matters but today i decided dig deeper.  Latency between hops (8 hops to headquarters) is less than 10ms.  This only happens on one access switch.  Tried the same destination (DHCP server) on a different access switch and results was done in less than a second.  All the access switches (4510 R-E) have uplink to two distribution switches (6509-E).  I am lost on how to find the problem.  I posted the configs of applicable access and distro swithes.

Jonathan,

[Richard Burts · Accepted solution]

Jonathan

One thing that frequently causes traceroute on Cisco devices to take longer is that the device trys to resolve the IP address at each hop to a name. And if the configured name server is not providing names for the addresses then there is a delay between hops as the Cisco waits for a name resolution. Could this be what you are observing?

HTH

Rick

[Jonathancert_2]

I don't think this is the cause.  Maybe, there is not problem and it's a software bug.  The servicies (email, web, FTP) are available to the customers but it appears there is a great latency.  Even when i traceroute from my access switch to external router it takes a few seconds to complete and all the devices are in the same building.

Jonathan,

[kapathak]

Hi Jonathan,

From the switch seeing the problem, can you give us the output of

sh mod

sh int | i line|drops

sh proc cpu sort | e 0.00

sh spann de ac | i exe|Number of top|from

show interface counters detail

Have you tried pinging hop-by-hop from a test PC to see where the latency lies? Is the problem affecting any particular VLAN (s)?

Regards,

Kapil

[Jonathancert_2]

Will try these commands tomorrow and post. No large latency doing ping command, only with traceroute.

Jonathan,

Sent from my Smartphone

[Jonathancert_2]

Thanks for your help.  I tried the commands and didn't find any drops, CRC errors or high CPU usage.   Although, i found bigger problems, that i have to work on.   Noticed the same latency on another 4510 in the same closet and the two switch have no network devices or ISP in common.  Was hoping it was a fiber problem but they do not sure the same bundle.  Have to love Monday.  May try re-loading the IOS for the heck of it.  I upgraded the IOS in June, not sure how long this problem has been occuring.

Jonathan,

[Jon Marshall]

Jonathan

Can you just confirm that when you run traceroute you are using the "numeric" option so it does try to resolve the IP as Rick mentioned ?

Jon

[Jonathancert_2]

I input the IP address of the DHCP and DNS server as the destination . But like I said before, I am having latency getting to the external router. Tested "traceroute" found three devices with latency to include one of my 6509's. Any suggestions are welcomed.

Jonathan,

Sent from my Smartphone

[Jonathancert_2]

After the fog in my head cleared, I realized i completely misunderstood your suggestion.  I did the expended "traceroute" option on the applicable devices and checked "yes" for numeric display (Numeric display [n]: y ) and everything worked fine.  I don't know why out of 24 devices only three are affected by this option.

Jonathan,

[Jonathancert_2]

Think I found the problem.  Had "ip domain-lookup" configured on all the affected devices.  Removed command and traceroute works without having to modify default settings.  I don't believe this ever had an affect on regular data traffic, so may look at QOS to priorities services.  Thanks for everyone help.

Jonathan,

[jimmysands73_2]

Thanks for the update.

A side quesiton...where would you use ip domain-lookup?  I ususally disable it (for labs).  Just curious in the real world why it would be used.  I guess that at the console level if you needed the names looked up, maybe I just answered my own question.

Glad you got it working