Long ping in network and outages

[PavelBischoffi70604]

Hello,

I have a strange problem in one segment of our network. We have a cisco 6509E router and some switches (3750, 2960 etc) connected into this 6509 (The 6509 is the centre of star and switches are edges).

Sometimes (once per day), occurs long pings at computers, there is no matter of VLAN where computers connected. But this is not a issue on all computers connected, only some. ( I mean that PC with IP adres at the end for example .3 works fine, but another with ip for example .54 has long ping and outages). Some of the problem computers are not connected directly to router, but to the switches on the edge. On this switches we have configured broadcast and multicast storm control.But without effect.

We tried to shutdown all access ports on 6509 (there are 3 pieces of 48 port metalic cards presented) and found the port which caused this behavior. We disconnected the PC connected to this port. After 2 days another port started to this situation again. Unfortunately, we can not configure broadcast storm control at main 6509, because access cards not supports this.

But i think this is not the broadcast problem, because there are not afected all computers. The ports when we have multicast storm control also are not blocked for it.

Have anybody some idea, what causes this?

Thanks Pavel.

[balaji.bandi]

Can you post the version of the device and configuration to look.

when this situation occurs what you see the Logs on the device?

[PavelBischoffi70604]

Hello,

I attached running configuration, show version and show logging. Router uptime is now around 14hours.

[balaji.bandi]

This IOS is too old and you need to atlease put to latest version stable code. Cat 6K end of Life.

can you post below output 

#show moduiles

# show process cpu sort |  ex 0.00

# show process cpu history

if you get maintenance window, i suggest to upgrade and check.

[PavelBischoffi70604]

Hello, the informations you asked for are in attached files.

BR,

[balaji.bandi]

is this single or VSS, is see the uptime 14 hours you regularly reboot  whole kit ?

if you get maintenance window upgrade to new Stable IOS.

[PavelBischoffi70604]

Hello,

this is a single router (no VSS). I confirm that we rebooted whole router 14 hours ago.

Unfortunately, we have not service contract, so we have not access to new firmwares.

[Meheretab Mengistu]

Hi Pavel,

Out of curiosity, I wanna see how long the 6509 was UP, and its version info as requested by Balaji.

[PavelBischoffi70604]

Hello,

the router is around 14 hours UP, in attachment there is show ver output.

[Georg Pauwen]

Hello,

some buffer tuning might help. Post the output of:

show buffers

[PavelBischoffi70604]

Hello,

show buffers is below:

CR-UL-RTD01#show buffers
Buffer elements:
499 in free list (500 max allowed)
62875727 hits, 0 misses, 0 created

Public buffer pools:
Small buffers, 104 bytes (total 1024, permanent 1024, peak 1606 @ 15:56:01):
1004 in free list (128 min, 2048 max allowed)
48462407 hits, 592 misses, 582 trims, 582 created
27 failures (0 no memory)
Medium buffers, 256 bytes (total 3000, permanent 3000):
1970 in free list (64 min, 3000 max allowed)
429999 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
Middle buffers, 600 bytes (total 1147, permanent 512, peak 1147 @ 01:29:48):
115 in free list (64 min, 1024 max allowed)
2225723 hits, 243 misses, 0 trims, 635 created
0 failures (0 no memory)
Big buffers, 1536 bytes (total 1000, permanent 1000):
997 in free list (64 min, 1000 max allowed)
570764 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
VeryBig buffers, 4520 bytes (total 10, permanent 10, peak 14 @ 15:57:24):
8 in free list (0 min, 100 max allowed)
41465 hits, 2 misses, 4 trims, 4 created
0 failures (0 no memory)
Large buffers, 9240 bytes (total 8, permanent 8):
8 in free list (0 min, 10 max allowed)
770 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
Huge buffers, 18024 bytes (total 4, permanent 2, peak 6 @ 00:24:14):
4 in free list (0 min, 4 max allowed)
1235 hits, 3 misses, 4 trims, 6 created
0 failures (0 no memory)

Interface buffer pools:
CF Small buffers, 104 bytes (total 100, permanent 100):
100 in free list (100 min, 200 max allowed)
542 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
CF Middle buffers, 600 bytes (total 100, permanent 100):
100 in free list (100 min, 200 max allowed)
176 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
Syslog ED Pool buffers, 600 bytes (total 288, permanent 288):
256 in free list (288 min, 288 max allowed)
411 hits, 0 misses
LI Middle buffers, 600 bytes (total 512, permanent 256, peak 512 @ 15:58:15):
256 in free list (256 min, 768 max allowed)
171 hits, 85 fallbacks, 0 trims, 256 created
0 failures (0 no memory)
256 max cache size, 256 in cache
0 hits in cache, 0 misses in cache
EOBC0/0 buffers, 1524 bytes (total 2400, permanent 2400):
1200 in free list (0 min, 2400 max allowed)
1200 hits, 0 fallbacks
1200 max cache size, 685 in cache
30756832 hits in cache, 0 misses in cache
CF Big buffers, 1536 bytes (total 25, permanent 25):
25 in free list (25 min, 50 max allowed)
276 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
LI Big buffers, 1536 bytes (total 512, permanent 256, peak 512 @ 15:58:15):
256 in free list (256 min, 768 max allowed)
171 hits, 85 fallbacks, 0 trims, 256 created
0 failures (0 no memory)
256 max cache size, 256 in cache
0 hits in cache, 0 misses in cache
IPC buffers, 4096 bytes (total 2352, permanent 2352):
2296 in free list (784 min, 7840 max allowed)
1529229 hits, 0 fallbacks, 0 trims, 0 created
0 failures (0 no memory)
CF VeryBig buffers, 4520 bytes (total 2, permanent 2):
2 in free list (2 min, 4 max allowed)
1 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
LI Very Big buffers, 4520 bytes (total 257, permanent 128, peak 257 @ 15:58:54):
129 in free list (128 min, 384 max allowed)
85 hits, 43 fallbacks, 10 trims, 139 created
0 failures (0 no memory)
128 max cache size, 128 in cache
0 hits in cache, 0 misses in cache
CF Large buffers, 5024 bytes (total 1, permanent 1):
1 in free list (1 min, 2 max allowed)
0 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
Private Huge IPC buffers, 18024 bytes (total 2, permanent 2):
2 in free list (1 min, 4 max allowed)
0 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
Private Huge buffers, 65280 bytes (total 2, permanent 2):
2 in free list (1 min, 4 max allowed)
203 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)

Header pools:

[Georg Pauwen]

Hello,

check if your device supports the command below (in global config mode):

buffers tune automatic

Otherwise, change the values as below, and monitor:

buffers small max-free 4096
buffers small min-free 256
buffers small permanent 2048
!
buffers middle max-free 2048
buffers middle min-free 128
buffers middle permanent 1024
!
buffers verybig max-free 200
buffers verybig min-free 10
buffers verybig permanent 20
!
buffers huge max-free 8
buffers huge min-free 2
buffers small permanent 8

[PavelBischoffi70604]

Hi,

I set the buffers manualy as below:

buffers small max-free 4096
buffers small min-free 256
buffers small permanent 2048
!
buffers middle max-free 2048
buffers middle min-free 128
buffers middle permanent 1024
!
buffers verybig max-free 200
buffers verybig min-free 10
buffers verybig permanent 20
!
buffers huge max-free 8
buffers huge min-free 2
buffers small permanent 8

Now i will monitoring if, the problem persist.

[PavelBischoffi70604]

Hello, 

after 6 days of monitoring, the problem persist. Any ohters ideas please?