cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
5207
Views
50
Helpful
23
Replies

Cisco Switch 2950/2960 Latency Issue

Hamidsattarrana
Level 1
Level 1

Hi All

We have 2950 and 2960 switches installed in LAN network.

I want to know how can I monitor those switches health etc.

Issue is that the ping RTT should be 1 ms in LAN network but sometimes the latency got high like 1500ms or 1200ms. 

How can I find the issue? 

There are almost 8 Access switches 2950 and 2960 and they are connected to Cisco 3750-G Core switch.

 

Need suggestion. 

23 Replies 23

Hello,

 

a couple of things you can you can check:

 

CPU

 

show processes cpu sorted 1min | ex 0.00

 

Check if the CPU utilization for the top listed processes is extraordinarily high

 

Interface errors

 

show interfaces x

 

Check if there are output errors, output drops, or other errors on any of the interfaces the PING traverses

 

Uptime

 

sh ver

 

Sometimes, long device uptime can cause this. If the device uptime is e.g. several years, a reboot of the device might help

Please check the show processes output.

 

 

CPU utilization for five seconds: 27%/3%; one minute: 23%; five minutes: 23%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
10 310472 328181 946 0.57% 0.59% 0.63% 0 ARP Input
21 17573300 2160978 8132 16.79% 16.10% 16.06% 0 Calhoun LED Proc
22 1654876 205803 8041 1.39% 1.45% 1.43% 0 Calhoun Statisti
25 795128 515050 1543 0.32% 0.54% 0.65% 0 Link Status Proc
30 59776 84053 711 0.24% 0.17% 0.18% 0 IP Input
36 543820 197009 2760 0.32% 0.24% 0.24% 0 Spanning Tree
50 52012 137100 379 0.08% 0.07% 0.07% 0 IP Multicast Dat
55 1192 407 2928 4.17% 0.87% 0.20% 1 Virtual Exec

Hello,

 

CPU looks normal (low, actually). What about the output for the other command (sh ver/sh interfaces x) ?

balaji.bandi
Hall of Fame
Hall of Fame

This could be due to port-utilisation, can you look any uplnk ports are utilising high  ?

 

or show us some example and config to understand the issue to advise better.

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

Trunk Link Configuration on 1 side:

interface FastEthernet0/1
switchport trunk allowed vlan 300,400
switchport mode trunk
logging event spanning-tree status
logging event trunk-status
spanning-tree portfast disable
ip dhcp snooping trust
end

 

 

Access Ports Configurations:

interface FastEthernet0/2
switchport access vlan 300
switchport mode access
switchport voice vlan 400
switchport port-security
switchport port-security maximum 5
switchport port-security aging time 5
switchport port-security aging type inactivity
logging event trunk-status
no keepalive
storm-control broadcast level 20.00 10.00
storm-control multicast level 20.00 10.00
storm-control unicast level 20.00 10.00
storm-control action trap
spanning-tree portfast
spanning-tree bpduguard enable
ip dhcp snooping limit rate 100
end

 

Show Processes cpu Output:

show processes cpu | exclude 0.00
CPU utilization for five seconds: 30%/5%; one minute: 29%; five minutes: 26%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
10 380632 409171 930 0.81% 0.67% 0.60% 0 ARP Input
21 19413196 2382788 8147 16.03% 16.18% 16.11% 0 Calhoun LED Proc
22 1826948 226943 8050 1.47% 1.44% 1.43% 0 Calhoun Statisti
25 866036 568036 1524 0.32% 0.55% 0.60% 0 Link Status Proc
27 814336 231069 3524 0.32% 0.06% 0.05% 0 Calhoun DHCP Sno
30 85988 117189 733 0.49% 0.63% 0.43% 0 IP Input
38 53944 21867 2466 0.65% 0.06% 0.01% 0 Port-Security
50 64572 179603 359 0.08% 0.10% 0.08% 0 IP Multicast Dat
55 4968 815 6095 2.70% 1.09% 0.80% 1 Virtual Exec
70 2340 2899 807 0.16% 0.65% 0.38% 0 IP SNMP
71 2776 1453 1910 0.16% 1.17% 0.68% 0 PDU DISPATCHER
72 4320 1453 2973 0.24% 1.82% 1.06% 0 SNMP ENGINE

 

show interface fa 0/1 Output: Trunk Link One side:

FastEthernet0/1 is up, line protocol is up (connected)
Hardware is Fast Ethernet, address is 0011.5c02.e001 (bia 0011.5c02.e001)
MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, media type is 100BaseTX
input flow-control is unsupported output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:01, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/1/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 68000 bits/sec, 32 packets/sec
5 minute output rate 6000 bits/sec, 3 packets/sec
1872878 packets input, 704834278 bytes, 21 no buffer
Received 1388925 broadcasts (0 multicast)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 21 ignored
0 watchdog, 555647 multicast, 0 pause input
0 input packets with dribble condition detected
398354 packets output, 122315890 bytes, 0 underruns
0 output errors, 0 collisions, 2 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped out

 

 

Trunk Link Interface Output on the other side:

403-SW2#show int gigabitEthernet 0/5
GigabitEthernet0/5 is up, line protocol is up (connected)
Hardware is Gigabit Ethernet, address is d0c2.821b.c305 (bia d0c2.821b.c305)
Description: 403-SW4
MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, media type is 10/100/1000BaseTX
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:01, output 00:00:01, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 154
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 11000 bits/sec, 8 packets/sec
5 minute output rate 70000 bits/sec, 30 packets/sec
402549 packets input, 123163993 bytes, 0 no buffer
Received 65234 broadcasts (8670 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 8670 multicast, 0 pause input
0 input packets with dribble condition detected
1886648 packets output, 708387124 bytes, 0 underruns
0 output errors, 0 collisions, 1 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped out

 

 

Uptime:

403-SW4#show version | include uptime
403-SW4 uptime is 1 day, 8 hours, 5 minutes

 

 

show interfaces | include line protocol | errors

Vlan300 is up, line protocol is up
0 input errors, 0 CRC, 0 frame, 0 overrun, 48 ignored
0 output errors, 2 interface resets
FastEthernet0/1 is up, line protocol is up (connected)
0 input errors, 0 CRC, 0 frame, 0 overrun, 21 ignored
0 output errors, 0 collisions, 2 interface resets

403-SW4#show spanning-tree sum
403-SW4#show spanning-tree summary
Switch is in pvst mode
Root bridge for: none
EtherChannel misconfig guard is enabled
Extended system ID is enabled
Portfast Default is disabled
PortFast BPDU Guard Default is disabled
Portfast BPDU Filter Default is disabled
Loopguard Default is disabled
UplinkFast is disabled
BackboneFast is disabled
Pathcost method used is short

403-SW4#show version
Cisco Internetwork Operating System Software
IOS (tm) C2950 Software (C2950-I6Q4L2-M), Version 12.1(22)EA7, RELEASE SOFTWARE (fc1)
Copyright (c) 1986-2006 by cisco Systems, Inc.
Compiled Fri 27-Jan-06 21:19 by antonino
Image text-base: 0x80010000, data-base: 0x8056A000

ROM: Bootstrap program is C2950 boot loader

403-SW4 uptime is 1 day, 8 hours, 15 minutes
System returned to ROM by power-on
System image file is "flash:/c2950-i6q4l2-mz.121-22.EA7.bin"

cisco WS-C2950SX-48-SI (RC32300) processor (revision H0) with 21004K bytes of memory.
Processor board ID FHK0823Z0H3
Last reset from system-reset
Running Standard Image
48 FastEthernet/IEEE 802.3 interface(s)
2 Gigabit Ethernet/IEEE 802.3 interface(s)

32K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address: 00:11:5C:02:E0:00
Motherboard assembly number: 73-9103-03
Power supply part number: 34-0965-01
Motherboard serial number: FOC082210BC
Power supply serial number: DAB08188ZSJ
Model revision number: H0
Motherboard revision number: A0
Model number: WS-C2950SX-48-SI
System serial number: FHK0823Z0H3
CLEI Code Number: CNMRV002RA
Top Assembly Part Number: 800-24115-01
Top Assembly Revision Number: E0
Version ID: N/A
Configuration register is 0xF

Ping statistics from my Local Laptop to Access Switch:

Ping statistics for 192.168.32.3:
Packets: Sent = 1466, Received = 1465, Lost = 1 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 1ms, Maximum = 34ms, Average = 2ms

Control-C

 

 

 

Actually we have configured Nagios for network monitoring.And nagios send alerts when latency reach at specific threshold.

See the nagios output: And latency

***** Nagios *****

 

Notification Type: PROBLEM

 

Service: PING

Host: 403-SW4

Address: 192.168.32.8

State: CRITICAL

 

Date/Time: Wed Jan 6 07:53:40 EST 2021

 

Additional Info:

 

PING CRITICAL - Packet loss = 0%, RTA = 2300.92 ms

Hello,

 

interface GigabitEthernet0/5 has some output drops, which could indicate an overloaded interface (possibly due to microbursts).

 

Check if your switches supports the command:

 

qos queue-softmax-multiplier 1200

 

Chances are they don't. If not, configure the below and check if the output drops disappear (and the PING results improve):

 

403-SW2(config)#mls qos queue-set output 2 threshold 3 3100 3100 100 3200
!
interface GigabitEthernet0/5
queue-set 2

Thanks for your support. But these commands are not supported on 2950/2950. They are layer 02 switches. 

One more thing can you tell me more about microbursts and Micro-Burst Monitoring?

 

Thanks.

Hello,

 

what exact IOS version are these switches running ? Post the output of 'sh ver'.

 

Microbursts typically are small spikes in network traffic that can cause e.g. output drops.

 

That said, are your users actually experiencing delays, or is it just the ICMP (ping) that is slow sometimes ?

Hello,

 

based on the diagram, the 'delay' can happen anywhere from the ESXi server to the SW4 switch. Test a ping from SW2 to SW4, then from the Core switch to SW4, and check if there is a difference in response times.

 

Actually, post the running configs of the Core, SW2, and SW4 switches, maybe we can spot something...

balaji.bandi
Hall of Fame
Hall of Fame

where is the NMS nagios connected ? same switch ?  i see these NMS some time alerts due to responce, we need to look what was the load that time and is there any routing change for the NMS to reach that IP.

 

what is the NMS IP address ? how far from this IP ? what devices are in the path ?

 

Service: PING

Host: 403-SW4

Address: 192.168.32.8

State: CRITICAL

 

can you do traceroute from nagios to switch and show us the results. is this latency all time or only certain time ?

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

Review Cisco Networking for a $25 gift card