Core Network packet-loss

[steveo123]

I get 1% packet loss and latency between 200-1000ms when pinging from the switch(cisco 2970) to any directly connected interface to a router (mainly 7206VXR) My concern is i have never seen packet loss or latency in a core network before!! i know that its utilization related and our core traffic can get very bursty at times, but i realy don’t think that we should have any kind of latency or packet loss even though its 2 packets lost out of a thousand pings.!

My concern is the cabling as all the interfaces connected to our core switch are 1000FX and iam 100% sure that category 6 cables were not used and they are only cat5, thus might cause the latency and packet loss due to miss-negotiation on the ports. Have in mind that our over all utilization in the core does not even reach 50% at max peak?? And all devices experiencing packet loss are directly connected within the same rack.

anyone else experience this before ??

Regards,

[marikakis]

Do you have any kind of rate-limits (e.g. ICMP traffic rate-limit) in your core ?

Are your core routers running BGP ?

Note that although ping and traceroute are useful tools, sometimes results can be misleading.

You have to know well the network setup in order to reach safe conclusions

on what the ping/trace output means.

These tools can be useful to have a rough idea of the delay on a link.

However, these figures are not precise enough to be used for performance evaluation.

Note also that ping through a router is a better measure of the perfomance

of the router than pinging the router directly.

When a packet destination is the router itself, this packet has to be process-switched.

The processor has to handle the information from this packet, and send an answer back.

This is not the main goal of a router. By definition, a router is built to route packets.

Answering a ping is offered as a best-effort service.

You are interested in real traffic not having problems, right ?

Try to find 2 hosts that respond to ping requests and the network path between them

crosses the router whose performance you want to check.

Ping from one host to the other and see what happens.

M.

[steveo123]

Thank you for your replies, however this network is under no stress in terms of overall utilization, i mean i haven’t even put it into full service yet as far as upstream interconnects are concerned, and only 50% of expected Xdsl users are terminating at present so lets assume we do not have utilization issues. The problem is quite clear but hard to resolve, i basically get packet loss of 1% and latency between 100-1000ms on devices directly connected to our core switch via cat5 cables, when i send pings from switch to a routers directly connected interface i get packet loss, however when i ping from that same router to another router which is connected to that same core switch as well i get no packet loss? i understand that a layer 3 designed device might have problems keeping up to a layer 2 devices switching capabilities, but the packet loss in our core network gets amplified by the time it makes the edge and cpe devices.

also just a note i get lots of crc and output errors on the switches interfaces.

Regards,

[tltee]

how about runts? if you see runt counts continue to raise then possibility is that the cable needs to be replace.

regards,

[steveo123]

Yes, i saw 326 runts but that was over 5 weeks, so i guess it doesnt mean that much, still investigating iam about to to a TCP dump to see whats actually flowing from where to why

Thanks for your help.

[mheick]

Couple of questions?

What are the L3/L2 devices that you are working with?

We have multiple L2/L3 devices conencted on our network and we are not seeing the same issue. It does sound like cogestion but you mention the cabling as being the issue in the orginal posting and at this point I am not feeling that this is the issue.

Also, could you provide some configs for the L2/L3 devices, as well as the the show interface for the connected interface? You can forward them on to maheick@verizon.net

Thanks.

[mheick]

Troubleshooting overutilization can be one of the most aggravating issues to deal with. You mentioned that utilization doesn't reach 50% at max peak. Are you monitoring in real time or this an average? If it is an average, what is the polling interval? Quite often a large polling interval, even as small as 5 minutes will hide the peaks and valleys of the bursty traffic on your network. This makes it difficult in nailing the overutilization as the problem.