Solved: Re: Moderate CPU, slow ping

jasonmeyer · ‎01-07-2020

Have an odd problem that seems to get worse over time. Ping times across vlans gets progressively slower until packets start dropping on workstations/servers. Seems like a monthly occurrence.

Include a putty log of output from various show cpu/platform commands.

Leo Laohoo · ‎01-07-2020

The supervisor card is running version 3.4.0. The last "digit" ends with a "0". I'll say that'll be the culprit right there.
My recommendation is upgrade the firmware to something more recent.
No, I won't "recommend" the version. Pick a version that has a very high last digit (more than "5") and read the Release Notes.

View solution in original post

Georg Pauwen · ‎01-07-2020

Hello,

hard to say. You might want to upgrade to CAT4500E Universal Image cat4500e-universal.SPA.03.11.01.E.152-7.E1.bin., which is the latest release...

Can you post the running configuration of the switch ?

View solution in original post

Leo Laohoo · ‎01-07-2020

The supervisor card is running version 3.4.0. The last "digit" ends with a "0". I'll say that'll be the culprit right there.
My recommendation is upgrade the firmware to something more recent.
No, I won't "recommend" the version. Pick a version that has a very high last digit (more than "5") and read the Release Notes.

Georg Pauwen · ‎01-07-2020

Hello,

hard to say. You might want to upgrade to CAT4500E Universal Image cat4500e-universal.SPA.03.11.01.E.152-7.E1.bin., which is the latest release...

Can you post the running configuration of the switch ?

jasonmeyer · ‎01-07-2020

I know I need to do a firmware and rommon upgrade, but this started happening a month or two ago. A reload clears things up.

here is the running config

jasonmeyer · ‎01-07-2020

=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2020.01.07 15:21:08 =~=~=~=~=~=~=~=~=~=~=~=
show run
Building configuration...

Current configuration : 18227 bytes
!
! Last configuration change at 14:59:10 CST Tue Jan 7 2020
!
version 15.1
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
service compress-config
!
hostname ISD623Core-4503
!
boot-start-marker
boot system flash bootflash:cat4500e-universal.SPA.03.04.00.SG.151-2.SG.bin
boot-end-marker
!
!
vrf definition mgmtVrf
!
address-family ipv4
exit-address-family
--More-- !
address-family ipv6
exit-address-family
!
logging console warnings
enable password #######
!
no aaa new-model
clock timezone CST -6 0
!
flow record r1
match ipv4 tos
match ipv4 protocol
match ipv4 source address
match ipv4 destination address
match transport source-port
match transport destination-port
match interface input
collect ipv4 dscp
collect transport tcp source-port
collect transport tcp destination-port
collect transport tcp flags
collect transport udp source-port
--More-- collect transport udp destination-port
collect interface output
collect counter bytes
collect counter packets
collect timestamp sys-uptime first
collect timestamp sys-uptime last
!
!
flow record netmap
!
!
flow exporter netmap
destination 10.249.254.23
transport udp 2055
!
!
flow monitor m1
record r1
exporter netmap
cache timeout inactive 30
cache timeout active 60
cache entries 1000
!
--More-- !
ip vrf Liin-vrf
!
ip domain-name ad.roseville.k12.mn.us
ip name-server 10.249.254.35
ip name-server 10.249.254.34
!
!
ipv6 multicast rpf use-bgp
vtp mode transparent
!
power redundancy-mode redundant
!
!
!
!
!
spanning-tree mode pvst
spanning-tree extend system-id
!
vlan internal allocation policy ascending
!
vlan 2
--More-- name admin
!
vlan 5
name GuestWLAN
!
vlan 6
name RoseAirNet
!
vlan 7
name EMPhones
!
vlan 20
name internet
!
vlan 21
name BH
!
vlan 22
name EDW
!
vlan 23
name FV
!
--More-- vlan 24
name iPrisim-Bridge
!
vlan 25
name inet
!
vlan 26
name public
!
vlan 99
name CTV
!
vlan 171
name NSCC_vBrick
!
vlan 198
name RoseAirNetM
!
vlan 253
name RVDATA
!
vlan 313
name iSCSI
--More-- !
vlan 353
name RVVOICE
!
vlan 357
name city-voice
!
vlan 1701
name vbrick
!
vlan 2102
name DC_LAN
!
vlan 2103
name HS_LAN
!
vlan 2104
name HAR-Data
!
vlan 2105
name MS_LAN
!
vlan 2106
--More-- name BH_LAN
!
vlan 2107
name CP_LAN
!
vlan 2108
name EG_LAN
!
vlan 2109
name EDW_LAN
!
vlan 2110
name FH_LAN
!
vlan 2111
name LC_LAN
!
vlan 2112
name PCS_LAN
!
vlan 2113
name FV_LAN
!
--More-- vlan 2115
name HS_StLAN
!
vlan 2116
name HSBells
!
vlan 2148
name Switch-MGT
!
vlan 3102
name DC_Voice
!
vlan 3103
name HS_Voice
!
vlan 3104
name HB_Voice
!
vlan 3105
name MS_Voice
!
vlan 3106
name BH_Voice
--More-- !
vlan 3107
name CP_Voice
!
vlan 3108
name EG_Voice
!
vlan 3109
name EDW_Voice
!
vlan 3110
name FH_Voice
!
vlan 3111
name LC_Voice
!
vlan 3112
name PCS_Voice
!
vlan 3113
name FV_Voice
!
!
--More-- !
!
!
!
!
interface Port-channel1
description COR-RASCore
switchport
switchport trunk allowed vlan 2,5-7,20-26,99,171,198,253,313,353,357,1701
switchport trunk allowed vlan add 2102-2116,2148,3102-3113
switchport mode trunk
ip flow monitor m1 input
!
interface Port-channel2
description RASCore-ServerSwitch
switchport
switchport trunk allowed vlan 2,5-7,20-26,99,171,198,253,313,353,357,1701
switchport trunk allowed vlan add 2102-2116,2148,3102-3113
switchport mode trunk
ip flow monitor m1 input
!
interface Port-channel3
description RASCore-HSCore
--More-- switchport
switchport trunk allowed vlan 2,5-7,20-26,99,171,198,253,313,353,357,1701
switchport trunk allowed vlan add 2102-2115,2148,3102-3113
switchport mode trunk
ip flow monitor m1 input
!
interface Port-channel5
description Wifi portchannel5
switchport
switchport trunk allowed vlan 2,5-7,21-23,198,2102-2115
switchport mode trunk
ip flow monitor m1 input
!
interface FastEthernet1
vrf forwarding mgmtVrf
no ip address
shutdown
speed auto
duplex auto
!
interface TenGigabitEthernet1/1
description 8510 Uplink
switchport trunk allowed vlan 2,5-7,21-23,198,2102-2115
--More-- switchport mode trunk
ip flow monitor m1 input
!
interface TenGigabitEthernet1/2
description COR-RASCore
switchport trunk allowed vlan 2,5-7,20-26,99,171,253,313,353,357,1701
switchport trunk allowed vlan add 2102-2116,2148,3102-3113
switchport mode trunk
ip flow monitor m1 input
!
interface TenGigabitEthernet1/3
description iBossLAN
switchport access vlan 2
switchport mode access
ip flow monitor m1 input
spanning-tree portfast
!
interface TenGigabitEthernet1/4
description iBoss Management
switchport access vlan 2103
switchport mode access
!
interface GigabitEthernet2/1
--More-- switchport access vlan 20
switchport mode access
!
interface GigabitEthernet2/2
switchport access vlan 3111
switchport mode access
!
interface GigabitEthernet2/3
!
interface GigabitEthernet2/4
switchport access vlan 2
switchport mode access
!
interface GigabitEthernet2/5
!
interface GigabitEthernet2/6
!
interface GigabitEthernet2/7
!
interface GigabitEthernet2/8
!
interface GigabitEthernet2/9
!
--More-- interface GigabitEthernet2/10
!
interface GigabitEthernet2/11
switchport trunk allowed vlan 2148
switchport mode trunk
!
interface GigabitEthernet2/12
!
interface GigabitEthernet2/13
!
interface GigabitEthernet2/14
!
interface GigabitEthernet2/15
!
interface GigabitEthernet2/16
!
interface GigabitEthernet2/17
!
interface GigabitEthernet2/18
!
interface GigabitEthernet2/19
!
interface GigabitEthernet2/20
--More-- !
interface GigabitEthernet2/21
!
interface GigabitEthernet2/22
!
interface GigabitEthernet2/23
!
interface GigabitEthernet2/24
!
interface GigabitEthernet2/25
!
interface GigabitEthernet2/26
!
interface GigabitEthernet2/27
!
interface GigabitEthernet2/28
!
interface GigabitEthernet2/29
!
interface GigabitEthernet2/30
!
interface GigabitEthernet2/31
switchport trunk allowed vlan 2,5-7,21-23,198,2102-2115
--More-- switchport mode trunk
channel-group 5 mode on
!
interface GigabitEthernet2/32
description UpLink to Harambee
switchport trunk allowed vlan 2104,3104
switchport mode trunk
ip flow monitor m1 input
!
interface GigabitEthernet2/33
description BigTripp connection
switchport access vlan 2
switchport mode access
spanning-tree portfast
!
interface GigabitEthernet2/34
description iSCSI trunk
switchport trunk allowed vlan 313
switchport mode trunk
!
interface GigabitEthernet2/35
description 5508Controller
switchport trunk allowed vlan 2,5-7,21-23,198,2102-2115
--More-- switchport mode trunk
ip flow monitor m1 input
channel-group 5 mode on
!
interface GigabitEthernet2/36
description iPrismInternal
switchport access vlan 2
switchport mode access
ip flow monitor m1 input
shutdown
spanning-tree portfast
!
interface GigabitEthernet2/37
description Sonicwall X1
switchport access vlan 20
switchport mode access
ip flow monitor m1 input
spanning-tree portfast
!
interface GigabitEthernet2/38
description HS-Mitel
switchport access vlan 3103
switchport mode access
--More-- spanning-tree portfast
!
interface GigabitEthernet2/39
switchport trunk allowed vlan 2,5-7,20-26,99,171,198,253,313,353,357,1701
switchport trunk allowed vlan add 2102-2115,2148,3102-3113
switchport mode trunk
channel-group 3 mode active
!
interface GigabitEthernet2/40
switchport trunk allowed vlan 2,5-7,20-26,99,171,198,253,313,353,357,1701
switchport trunk allowed vlan add 2102-2115,2148,3102-3113
switchport mode trunk
channel-group 3 mode active
!
interface GigabitEthernet2/41
switchport trunk allowed vlan 2,5-7,20-26,99,171,198,253,313,353,357,1701
switchport trunk allowed vlan add 2102-2115,2148,3102-3113
switchport mode trunk
channel-group 3 mode active
!
interface GigabitEthernet2/42
switchport trunk allowed vlan 2,5-7,20-26,99,171,198,253,313,353,357,1701
switchport trunk allowed vlan add 2102-2115,2148,3102-3113
--More-- switchport mode trunk
channel-group 3 mode active
!
interface GigabitEthernet2/43
switchport trunk allowed vlan 2,5-7,20-26,99,171,198,253,313,353,357,1701
switchport trunk allowed vlan add 2102-2116,2148,3102-3113
switchport mode trunk
channel-group 2 mode active
!
interface GigabitEthernet2/44
switchport trunk allowed vlan 2,5-7,20-26,99,171,198,253,313,353,357,1701
switchport trunk allowed vlan add 2102-2116,2148,3102-3113
switchport mode trunk
channel-group 2 mode active
!
interface GigabitEthernet2/45
switchport trunk allowed vlan 2,5-7,20-26,99,171,198,253,313,353,357,1701
switchport trunk allowed vlan add 2102-2116,2148,3102-3113
switchport mode trunk
channel-group 2 mode active
!
interface GigabitEthernet2/46
switchport trunk allowed vlan 2,5-7,20-26,99,171,198,253,313,353,357,1701
--More-- switchport trunk allowed vlan add 2102-2116,2148,3102-3113
switchport mode trunk
channel-group 2 mode active
!
interface GigabitEthernet2/47
switchport trunk allowed vlan 2,5-7,20-26,99,171,198,253,313,353,357,1701
switchport trunk allowed vlan add 2102-2116,2148,3102-3113
switchport mode trunk
shutdown
channel-group 1 mode active
!
interface GigabitEthernet2/48
switchport trunk allowed vlan 2,5-7,20-26,99,171,198,253,313,353,357,1701
switchport trunk allowed vlan add 2102-2116,2148,3102-3113
switchport mode trunk
shutdown
channel-group 1 mode active
!
interface TenGigabitEthernet3/1
!
interface TenGigabitEthernet3/2
!
interface TenGigabitEthernet3/3
--More-- !
interface TenGigabitEthernet3/4
!
interface TenGigabitEthernet3/5
!
interface TenGigabitEthernet3/6
!
interface TenGigabitEthernet3/7
!
interface TenGigabitEthernet3/8
!
interface TenGigabitEthernet3/9
!
interface TenGigabitEthernet3/10
!
interface TenGigabitEthernet3/11
!
interface TenGigabitEthernet3/12
description DC Uplink
switchport trunk allowed vlan 2,26,99,171,253,353,357,2102,2103,2105,2112,2115
switchport trunk allowed vlan add 2148,3102,3103
switchport mode trunk
!
--More-- interface Vlan1
no ip address
!
interface Vlan2
ip address 10.249.254.254 255.255.0.0
ip helper-address 10.249.254.105
ip helper-address 10.103.254.6
ip helper-address 10.103.254.9
ip helper-address 10.113.254.6
ip helper-address 10.112.254.9
ip helper-address 10.105.255.255
ip directed-broadcast 101
!
interface Vlan5
ip address 10.30.1.254 255.255.0.0
!
interface Vlan6
ip address 10.99.1.254 255.255.0.0
ip helper-address 10.249.255.255
ip directed-broadcast 101
!
interface Vlan7
ip address 10.97.254.254 255.255.0.0
--More-- !
interface Vlan20
description internet
no ip address
!
interface Vlan21
ip address 10.106.254.254 255.255.0.0
ip helper-address 10.249.254.40
ip helper-address 10.249.254.120
ip helper-address 10.249.254.121
ip helper-address 10.249.254.122
ip helper-address 10.249.254.123
ip helper-address 10.249.254.124
ip helper-address 10.249.254.90
ip helper-address 10.249.254.91
ip helper-address 10.249.254.92
ip directed-broadcast 101
!
interface Vlan22
ip address 10.109.254.254 255.255.0.0
ip helper-address 10.249.254.40
ip helper-address 10.249.254.120
ip helper-address 10.249.254.121
--More-- ip helper-address 10.249.254.122
ip helper-address 10.249.254.123
ip helper-address 10.249.254.124
ip helper-address 10.249.254.90
ip helper-address 10.249.254.91
ip helper-address 10.249.254.92
ip directed-broadcast 101
!
interface Vlan23
ip address 10.113.254.254 255.255.0.0
ip helper-address 10.249.254.40
ip helper-address 10.249.254.120
ip helper-address 10.249.254.121
ip helper-address 10.249.254.122
ip helper-address 10.249.254.123
ip helper-address 10.249.254.124
ip helper-address 10.249.254.90
ip helper-address 10.249.254.91
ip helper-address 10.249.254.92
ip directed-broadcast 101
!
interface Vlan24
no ip address
--More-- !
interface Vlan25
no ip address
!
interface Vlan26
no ip address
!
interface Vlan198
ip address 10.98.1.254 255.255.0.0
!
interface Vlan313
ip address 172.16.1.254 255.255.255.0
!
interface Vlan2102
ip address 10.102.254.254 255.255.0.0
ip helper-address 10.249.254.40
ip helper-address 10.249.254.105
ip helper-address 10.249.254.120
ip helper-address 10.249.254.121
ip helper-address 10.249.254.122
ip helper-address 10.249.254.123
ip helper-address 10.249.254.124
ip helper-address 10.249.254.90
--More-- ip helper-address 10.249.254.91
ip helper-address 10.249.254.92
ip directed-broadcast 101
!
interface Vlan2103
ip address 10.103.254.254 255.255.0.0
ip helper-address 10.249.254.40
ip helper-address 10.249.254.105
ip helper-address 10.249.254.120
ip helper-address 10.249.254.121
ip helper-address 10.249.254.122
ip helper-address 10.249.254.123
ip helper-address 10.249.254.124
ip helper-address 10.249.254.90
ip helper-address 10.249.254.91
ip helper-address 10.249.254.92
ip directed-broadcast 101
!
interface Vlan2104
ip address 172.25.254.254 255.255.0.0
ip helper-address 10.249.254.40
ip helper-address 10.112.254.8
ip helper-address 10.249.254.86
--More-- ip helper-address 10.249.254.87
ip helper-address 10.249.254.88
ip helper-address 10.249.254.120
ip helper-address 10.249.254.121
ip helper-address 10.249.254.122
ip helper-address 10.249.254.123
ip helper-address 10.249.254.124
ip helper-address 10.249.254.90
ip helper-address 10.249.254.91
ip helper-address 10.249.254.92
ip directed-broadcast 101
!
interface Vlan2105
ip address 10.105.254.254 255.255.0.0
ip helper-address 10.249.254.40
ip helper-address 10.249.254.120
ip helper-address 10.249.254.121
ip helper-address 10.249.254.123
ip helper-address 10.249.254.122
ip helper-address 10.249.254.124
ip helper-address 10.249.254.90
ip helper-address 10.249.254.91
ip helper-address 10.249.254.92
--More-- ip directed-broadcast 101
!
interface Vlan2106
no ip address
ip helper-address 10.249.254.40
ip helper-address 10.249.254.120
ip helper-address 10.249.254.121
ip helper-address 10.249.254.122
ip helper-address 10.249.254.123
ip helper-address 10.249.254.124
ip helper-address 10.249.254.90
ip helper-address 10.249.254.91
ip helper-address 10.249.254.92
ip directed-broadcast 101
!
interface Vlan2107
ip address 10.107.254.254 255.255.0.0
ip helper-address 10.249.254.40
ip helper-address 10.249.254.120
ip helper-address 10.249.254.121
ip helper-address 10.249.254.122
ip helper-address 10.249.254.123
ip helper-address 10.249.254.124
--More-- ip helper-address 10.249.254.90
ip helper-address 10.249.254.91
ip helper-address 10.249.254.92
ip directed-broadcast 101
!
interface Vlan2108
ip address 10.108.254.254 255.255.0.0
ip helper-address 10.249.254.40
ip helper-address 10.249.254.120
ip helper-address 10.249.254.122
ip helper-address 10.249.254.121
ip helper-address 10.249.254.123
ip helper-address 10.249.254.124
ip helper-address 10.249.254.90
ip helper-address 10.249.254.91
ip helper-address 10.249.254.92
ip directed-broadcast 101
!
interface Vlan2109
no ip address
ip helper-address 10.249.254.40
ip helper-address 10.249.254.120
ip helper-address 10.249.254.121
--More-- ip helper-address 10.249.254.122
ip helper-address 10.249.254.123
ip helper-address 10.249.254.124
ip helper-address 10.249.254.90
ip helper-address 10.249.254.91
ip helper-address 10.249.254.92
ip directed-broadcast 101
!
interface Vlan2110
ip address 10.110.254.254 255.255.0.0
ip helper-address 10.249.254.40
ip helper-address 10.249.254.120
ip helper-address 10.249.254.121
ip helper-address 10.249.254.122
ip helper-address 10.249.254.123
ip helper-address 10.249.254.124
ip helper-address 10.249.254.90
ip helper-address 10.249.254.91
ip helper-address 10.249.254.92
ip directed-broadcast 101
!
interface Vlan2111
ip address 10.111.254.254 255.255.0.0
--More-- ip helper-address 10.249.254.40
ip helper-address 10.249.254.120
ip helper-address 10.249.254.121
ip helper-address 10.249.254.122
ip helper-address 10.249.254.123
ip helper-address 10.249.254.124
ip helper-address 10.249.254.90
ip helper-address 10.249.254.91
ip helper-address 10.249.254.92
ip directed-broadcast 101
!
interface Vlan2112
ip address 10.112.254.254 255.255.0.0
ip helper-address 10.249.254.40
ip helper-address 10.249.254.120
ip helper-address 10.249.254.121
ip helper-address 10.249.254.122
ip helper-address 10.249.254.123
ip helper-address 10.249.254.124
ip helper-address 10.249.254.90
ip helper-address 10.249.254.91
ip helper-address 10.249.254.92
ip directed-broadcast 101
--More-- !
interface Vlan2113
no ip address
ip helper-address 10.249.254.40
ip helper-address 10.249.254.120
ip helper-address 10.249.254.121
ip helper-address 10.249.254.122
ip helper-address 10.249.254.123
ip helper-address 10.249.254.124
ip helper-address 10.249.254.90
ip helper-address 10.249.254.91
ip helper-address 10.249.254.92
ip directed-broadcast 101
!
interface Vlan2114
no ip address
!
interface Vlan2115
ip address 10.115.254.254 255.255.0.0
ip helper-address 10.249.254.40
ip helper-address 10.249.254.120
ip helper-address 10.249.254.121
ip helper-address 10.249.254.122
--More-- ip helper-address 10.249.254.123
ip helper-address 10.249.254.124
ip helper-address 10.249.254.90
ip helper-address 10.249.254.91
ip helper-address 10.249.254.92
ip directed-broadcast 101
!
interface Vlan2116
ip address 10.116.1.254 255.255.255.0
!
interface Vlan2148
ip address 10.248.254.254 255.255.255.0
!
interface Vlan3102
ip address 10.202.1.254 255.255.255.0
!
interface Vlan3103
ip address 10.203.1.254 255.255.254.0
!
interface Vlan3104
ip address 10.204.1.254 255.255.255.0
!
interface Vlan3105
--More-- ip address 10.205.1.254 255.255.255.0
!
interface Vlan3106
ip address 10.206.1.254 255.255.255.0
!
interface Vlan3107
ip address 10.207.1.254 255.255.255.0
!
interface Vlan3108
ip address 10.208.1.254 255.255.255.0
!
interface Vlan3109
ip address 10.209.1.254 255.255.255.0
!
interface Vlan3110
ip address 10.210.1.254 255.255.255.0
!
interface Vlan3111
ip address 10.211.1.254 255.255.255.0
!
interface Vlan3112
ip address 10.212.1.254 255.255.255.0
!
--More-- interface Vlan3113
ip address 10.213.1.254 255.255.255.0
!
ip forward-protocol udp echo
ip http server
ip route profile
ip route 0.0.0.0 0.0.0.0 10.249.254.230
!
!
logging trap warnings
access-list 101 permit udp host 10.249.254.105 any eq echo
!
snmp-server community public RO
snmp-server location RAHS MDF Telco Racks
snmp-server contact helpdesk@isd623.org
!
!
line con 0
password #####
login
stopbits 1
line vty 0
password #####
--More-- login
length 0
line vty 1 4
login
length 0
line vty 5 15
login
!
ntp server 10.249.254.34
ntp server 10.249.254.35
end

ISD623Core-4503#

jasonmeyer · ‎01-08-2020

I am assuming i still have perform the ROMMON upgrade, then the firmware?

Reza Sharifi · ‎01-08-2020

The latest RMMON version is 15.0(1r)SG16. Al thought it is a good idea but depending on the IOS you want to upgrade to you may not need to upgrade ROMMON. In any case, you would upgrade the ROMMON first and than the main IOS for the chassis.

Please make sure to read the release notes very carefully.

HTH

jasonmeyer · ‎01-08-2020

Well I was thinking 3.11.1E as its the latest.

Reza Sharifi · ‎01-08-2020

That should work. I would just upgrade ROMMON first and than read the release notes for 11.01.E.1 to make sure you are not hitting a major bug.