cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Announcements

Ask the Expert- SD-WAN

1222
Views
5
Helpful
12
Replies
Beginner

Cisco 2950 loss connectivity management Vlan

Hi all,

Randomly when I try to access to 2950 from management tools, switch is unreachable, I have to  access from other switch and reload 2950.

Problem only is from managemt tool to managament vlan 1 2950.

The strange thing is that management interface is encountering a very fast increase of throttles, broadcast and ignored packets:

2950#show interfaces vlan 1

Vlan1 is up, line protocol is up

  Hardware is CPU Interface, address is 0023.3488.fd65 (bia 0023.3488.fd65)

  Internet address is xxxxxxxxxx

  MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,

     reliability 255/255, txload 1/255, rxload 1/255

  Encapsulation ARPA, loopback not set

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input 00:00:00, output never, output hang never

  Last clearing of "show interface" counters 18:01:29

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0   0 drops

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  30 second input rate 75000 bits/sec, 74 packets/sec

  30 second output rate 20000 bits/sec, 19 packets/sec

     3649696 packets input, 444766538 bytes, 0 no buffer

     Received 2146441 broadcasts (0 IP multicast)

     0 runts, 0 giants, 17226 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 1496048 ignored

     1276594 packets output, 211444551 bytes, 0 underruns

     0 output errors, 0 interface resets

     0 output buffer failures, 0 output buffers swapped out

show buffers

Buffer elements:

     499 in free list (500 max allowed)

     163971601 hits, 0 misses, 0 created

Public buffer pools:

Small buffers, 104 bytes (total 33, permanent 25, peak 181 @ 6w5d):

     33 in free list (20 min, 60 max allowed)

     191360417 hits, 4012 misses, 7336 trims, 7344 created

     26 failures (0 no memory)

Middle buffers, 600 bytes (total 30, permanent 15, peak 76 @ 6w5d):

     28 in free list (10 min, 30 max allowed)

     11560828 hits, 3320 misses, 3269 trims, 3284 created

     1562 failures (0 no memory)

Big buffers, 1524 bytes (total 9, permanent 5, peak 25 @ 5w6d):

     9 in free list (5 min, 10 max allowed)

     15740378 hits, 4396 misses, 3719 trims, 3723 created

     2344 failures (0 no memory)

VeryBig buffers, 4520 bytes (total 10, permanent 0, peak 24 @ 6w4d):

     10 in free list (0 min, 10 max allowed)

     147538 hits, 1870 misses, 1808 trims, 1818 created

     986 failures (0 no memory)

Large buffers, 5024 bytes (total 1, permanent 0, peak 6 @ 6w5d):

     1 in free list (0 min, 5 max allowed)

     123 hits, 863 misses, 600 trims, 601 created

     863 failures (0 no memory)

Huge buffers, 18024 bytes (total 1, permanent 0, peak 4 @ 5w6d):

     1 in free list (0 min, 2 max allowed)

     60 hits, 803 misses, 595 trims, 596 created

     803 failures (0 no memory)

Interface buffer pools:

Calhoun Packet Receive Pool buffers, 1560 bytes (total 512, permanent 512):

     447 in free list (0 min, 512 max allowed)

     310549933 hits, 0 misses

show buffers failures

Caller       Pool          Size      When

0x802D7634  Large            170    08:07:37

0x802D7634  Huge             170    08:07:37

0x802D7634  Middle           170    07:07:37

0x802D7634  Middle           170    05:33:54

0x802D7634  Middle           170    03:56:43

0x802D7634  Middle           170    03:56:43

0x802D7634  Middle           170    00:12:26

0x802D7634  Middle           178    00:12:26

0x802D7634  Middle           170    00:12:26

0x802D7634  Middle           170    00:12:26

2950#show region address 0x802D7634

Address 0x802D7634 is located physically in :

  Name  : text

  Class : IText

  Media : R/W

  Start : 0x80010000

  End   : 0x8056FA63

  Size  : 0x0055FA64

It could be  a hardware problem?, IOS has been upgraded.

Best Regards.

12 REPLIES 12
Advisor

Cisco 2950 loss connectivity management Vlan

  If it's been a long time since it was reloaded you might want to think about that .  If you need to know what the broadcasts are etc..  then  you will have to span a port and wireshark it ... to see what is going on .

Beginner

Cisco 2950 loss connectivity management Vlan

Hi,

    How many hosts are in VLAN 1? The last time I saw symptoms like this was on a very flat network with over 2500 hosts on VLAN 1. The big tipoff is the the volume of broadcast traffic. The management interface must check each broadcast to see if it is the intended recipient, and then discard the packet if it is not the intended recipient. This means the management interface is occupied with packet receipt/discard and not performing the intended response to management traffic.

   The network most likely has redundant links and a bit of spanningtree BPDU skew happening as well, check the spanning tree root for Topology changes and verify the spanning tree radius it's probably at the upper limit. Twenty bit mask or smaller in a switched network, never a good scene.

Cheers,

Brian

Beginner

Cisco 2950 loss connectivity management Vlan

Thanks for reply,

more or less are 1000 hosts in Vlan 1, but the big problem is if I ping from switch to Managament PC I can´t ping, but if I do clear arp I can,  in few minutes after the clear arp I can't ping again.

It could be a duplicated mac?

Regards. 

Advisor

Cisco 2950 loss connectivity management Vlan

  Could be a duplicate ip address .  Check at the L3 end for macs and see if it matches the 2950 or someone else.

Beginner

Cisco 2950 loss connectivity management Vlan

From switch to switch I can ping but from PC to switch or switch to PC I can`t, yes it's ok, I did  trace mac and matches the 2950.

Beginner

Cisco 2950 loss connectivity management Vlan

Hi,

    If the network is smaller than 1000 then you have either a subnet mask problem on the PC or the switch, alternately you could have a host doing a proxy arp response and the PC or switch is recieving this bogus proxy arp request.

I would not suspect a duplicate mac, I would lean more towards a duplicate IP address on the wire, which means the IP address of the switch is duplicate. Check the PC arp cache when ping is OK then again when the ping fails.

Cheers,

Beginner

Cisco 2950 loss connectivity management Vlan

Hi again,

I changed the managament Ip address of the Vlan 1 in the switch and the problem is still there. I don't know if there is hardware problem with the switch???

Thanks for your help.

Advisor

Cisco 2950 loss connectivity management Vlan

  Have you reloaded the switch seems I remember the 2950's having memory fragmentation issues in the earlier  IOS versions  and a reload would fix it at least for awhile.  Do the uplink ports look ok with no interface errors etc. . What version is the 2950 running ?    

Beginner

Cisco 2950 loss connectivity management Vlan

Hi,

Did you check the arp cache on the PC when ping was OK and when ping failed to determine if the MAC address was different for the switch?

Also just for a proof of concept, try the command "show spanning-tree blockedports". Then if you spot some ports that are in blocking mode, do a "shutdown" on each one of them, they are redundant paths so no risk of an outage. Watch to see if the switch remains responsive or goes non responsive again.

After the test of course do a "no shutdown" on the ports and again see if the switch behaves as badly as before.

Cheers,

Beginner

Cisco 2950 loss connectivity management Vlan

ok I will try. A possible solution could be put arp timeout 300 sec in VLan1?

Regards

Beginner

Cisco 2950 loss connectivity management Vlan

Hi,

I don't beleive adjusting the arp timeout on VLAN 1 will help much.

The first item is determine the root cause. This means we need to rule out the proxy-arp or duplicate IP suspicions.

This can be accomplished by examining the arp cache on the PC when ping is OK and when ping doesn't work. If this is ruled out, next take some workload off the switch to determine if this is load related, shutting down blocking ports will accomplish this.

Cheers,

Highlighted
Beginner

Re: Cisco 2950 loss connectivity management Vlan

I was with the problem on my network with 2950s, but after I tried a lot of things I figured out what was happens.

NO stp recalculation, no crc, process and memory issue.

I was not using vlan 1, but my management vlan was showing packet loss and high latency.

I put a machine on the same vlan as the management vlan and the machine not show the same problem as management of switch 2950.

The other switches on the same network seguiment was not showing packet loss or high latency.

 

Problem solved:

I allowed on the trunk just the vlans used on that switch.

switchport trunk allowed vlan x,y

CreatePlease to create content
Content for Community-Ad
July's Community Spotlight Awards