- Cisco Community
- Technology and Support
- Networking
- Switching
- 4500 switch very high CPU
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
4500 switch very high CPU
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-05-2017 12:57 PM - edited 03-08-2019 10:51 AM
I am seeing very high CPU on a WS-C4503 (cat4500-ipbasek9-mz.150-2.SG8.bin)
Looking for some ideas on how I can troubleshoot this please
sh processes cpu sorted
CPU utilization for five seconds: 98%/2%; one minute: 98%; five minutes: 98%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
56 9583135 1088691 8802 57.95% 50.36% 55.04% 0 Cat4k Mgmt LoPri
116 3060591 581096 5266 17.82% 18.22% 18.26% 0 Spanning Tree
55 1508484 3557450 424 9.03% 10.16% 9.29% 0 Cat4k Mgmt HiPri
109 1397911 2044930 683 7.75% 6.81% 7.34% 0 IP Input
STV-4503-01#show platform health
%CPU %CPU RunTimeMax Priority Average %CPU Total
Target Actual Target Actual Fg Bg 5Sec Min Hour CPU
GalChassisVp-review 3.00 0.36 10 48 100 500 0 0 0 0:55
S2w-JobEventSchedule 10.00 0.37 10 10 100 500 0 0 0 0:54
Stub-JobEventSchedul 10.00 2.99 10 6 100 500 3 2 2 5:57
Lj-poll 1.00 0.02 2 1 100 500 0 0 0 0:04
StatValueMan Update 1.00 0.01 1 0 100 500 0 0 0 0:16
Pim-review 0.10 0.00 1 0 100 500 0 0 0 0:01
Ebm-host-review 1.00 0.00 8 4 100 500 0 0 0 0:13
Ebm-host-util-review 1.00 0.00 10 0 100 500 0 0 0 0:00
Ebm-port-review 0.10 0.00 1 0 100 500 0 0 0 0:00
Protocol-aging-revie 0.20 0.00 2 0 100 500 0 0 0 0:00
EbmHostRedundancyMan 2.00 0.00 20 0 100 500 0 0 0 0:00
Acl-Flattener 1.00 0.00 10 0 100 500 0 0 0 0:00
GalChassisVp Ondeman 2.00 0.00 2 0 100 500 0 0 0 0:00
KxAclPathMan create/ 1.00 0.00 10 5 100 500 0 0 0 0:08
KxAclPathMan update 2.00 0.00 10 0 100 500 0 0 0 0:00
KxAclPathMan reprogr 1.00 0.00 2 0 100 500 0 0 0 0:00
TagMan-RecreateMtegR 1.00 0.00 10 0 100 500 0 0 0 0:00
K2CpuMan Review 30.00 57.92 30 108 100 500 71 63 46 135:04
K2AccelPacketMan: Tx 10.00 8.03 20 0 100 500 9 8 6 19:10
K2AccelPacketMan: Au 0.10 0.00 0 0 100 500 0 0 0 0:00
K2AclMan-labeledFlat 1.00 0.00 10 0 100 500 0 0 0 0:00
K2AclCamMan stale en 1.00 0.00 10 0 100 500 0 0 0 0:00
K2AclCamMan consolid 1.00 0.00 10 0 100 500 0 0 0 0:00
K2AclCamMan hw stats 3.00 0.13 10 5 100 500 0 0 0 1:25
K2AclCamMan kx stats 1.00 0.00 10 5 100 500 0 0 0 0:49
K2AclCamMan Audit re 1.00 0.00 10 5 100 500 0 0 0 0:48
K2AclPolicerTableMan 1.00 0.00 10 1 100 500 0 0 0 0:01
K2L2 Address Table R 2.00 0.05 12 5 100 500 1 1 1 3:34
K2L2 New Static Addr 2.00 0.00 10 5 100 500 0 0 0 0:00
K2L2 New Multicast A 2.00 0.00 10 5 100 500 0 0 0 0:01
K2L2 Dynamic Address 2.00 0.00 10 5 100 500 0 0 0 0:00
K2L2 PPE Vlan Table 2.00 0.02 10 5 100 500 0 0 0 1:26
K2L2 Vlan Table Revi 2.00 0.00 12 8 100 500 0 0 0 0:04
K2PortMan Review 3.00 3.99 15 11 100 500 4 4 3 8:53
Gigaport0 Review 0.40 0.01 4 2 100 500 0 0 0 0:02
Gigaport1 Review 0.40 0.01 4 2 100 500 0 0 0 0:02
Gigaport2 Review 0.40 0.01 4 2 100 500 0 0 0 0:02
Gigaport3 Review 0.40 0.00 4 2 100 500 0 0 0 0:02
Gigaport4 Review 0.40 0.01 4 2 100 500 0 0 0 0:02
Gigaport5 Review 0.40 0.01 4 2 100 500 0 0 0 0:02
Gigaport6 Review 0.40 0.01 4 2 100 500 0 0 0 0:02
Gigaport7 Review 0.40 0.01 4 2 100 500 0 0 0 0:02
Gigaport8 Review 0.40 0.07 4 2 100 500 0 0 0 0:13
Gigaport9 Review 0.40 0.06 4 3 100 500 0 0 0 0:13
Gigaport10 Review 0.40 0.08 4 3 100 500 0 0 0 0:13
Gigaport11 Review 0.40 0.10 4 2 100 500 0 0 0 0:13
Gigaport12 Review 0.40 0.07 4 2 100 500 0 0 0 0:13
Gigaport13 Review 0.40 0.06 4 2 100 500 0 0 0 0:13
Gigaport14 Review 0.40 0.06 4 1 100 500 0 0 0 0:13
Gigaport15 Review 0.40 0.08 4 3 100 500 0 0 0 0:13
Gigaport16 Review 0.40 0.00 4 2 100 500 0 0 0 0:02
Gigaport17 Review 0.40 0.00 4 2 100 500 0 0 0 0:02
Gigaport18 Review 0.40 0.00 4 2 100 500 0 0 0 0:02
Gigaport19 Review 0.40 0.00 4 2 100 500 0 0 0 0:02
Gigaport20 Review 0.40 0.00 4 2 100 500 0 0 0 0:02
Gigaport21 Review 0.40 0.01 4 2 100 500 0 0 0 0:02
Gigaport22 Review 0.40 0.01 4 2 100 500 0 0 0 0:02
Gigaport23 Review 0.40 0.01 4 2 100 500 0 0 0 0:02
Gigaport24 Review 0.40 0.00 4 2 100 500 0 0 0 0:02
Gigaport25 Review 0.40 0.00 4 2 100 500 0 0 0 0:02
Gigaport26 Review 0.40 0.02 4 2 100 500 0 0 0 0:02
Gigaport27 Review 0.40 0.02 4 2 100 500 0 0 0 0:02
Gigaport28 Review 0.40 0.02 4 3 100 500 0 0 0 0:01
Gigaport29 Review 0.40 0.02 4 2 100 500 0 0 0 0:02
Gigaport30 Review 0.40 0.02 4 2 100 500 0 0 0 0:02
Gigaport31 Review 0.40 0.01 4 3 100 500 0 0 0 0:02
K2Fib cam usage revi 2.00 0.00 15 0 100 500 0 0 0 0:00
K2Fib IrmFib Review 2.00 0.00 15 0 100 500 0 0 0 0:00
K2Fib AdjRepop Revie 2.00 0.00 15 0 100 500 0 0 0 0:00
K2Fib Vrf Unpunt Rev 2.00 0.00 15 0 100 500 0 0 0 0:01
K2Fib Consistency Ch 1.00 0.17 5 2 100 500 0 0 0 0:34
K2FibAdjMan Stats Re 2.00 0.22 10 7 100 500 0 0 0 0:37
K2FibAdjMan Host Mov 2.00 0.00 10 7 100 500 0 0 0 0:10
K2FibAdjMan Adj Chan 2.00 0.00 10 0 100 500 0 0 0 0:00
K2FibMulticast Signa 2.00 0.00 10 1 100 500 0 0 0 0:05
K2FibMulticast Entry 2.00 0.00 10 7 100 500 0 0 0 0:00
K2FibMulticast Irm M 2.00 0.00 10 0 100 500 0 0 0 0:00
K2FibFastDropMan Rev 2.00 0.00 7 0 100 500 0 0 0 0:00
K2FibPbr route map r 2.00 0.34 20 5 100 500 0 0 0 1:15
K2FibPbr flat acl pr 2.00 0.01 20 2 100 500 0 0 0 0:11
K2FibPbr consolidati 2.00 0.00 10 0 100 500 0 0 0 0:01
K2FibPerVlanPuntMan 2.00 0.00 15 3 100 500 0 0 0 0:00
K2FibWccpMan Service 2.00 0.00 20 0 100 500 0 0 0 0:00
K2FibWccpMan Cache E 2.00 0.00 20 0 100 500 0 0 0 0:00
K2FibWccpMan Redirec 2.00 0.00 20 0 100 500 0 0 0 0:00
K2FibWccpMan AdjRepo 2.00 0.00 20 0 100 500 0 0 0 0:00
K2FibFC Aging review 2.00 0.00 10 0 100 500 0 0 0 0:01
K2FibFC DelFlow revi 2.00 0.00 10 0 100 500 0 0 0 0:00
K2FibFC Adj review 2.00 0.00 10 0 100 500 0 0 0 0:01
K2FibFC overload/rel 2.00 0.00 10 0 100 500 0 0 0 0:00
K2MetStatsMan Review 2.00 0.04 5 2 100 500 0 0 0 1:10
K2FibMulticast MET S 2.00 0.00 10 0 100 500 0 0 0 0:00
K2QosDblMan Rate DBL 2.00 0.10 7 0 100 500 0 0 0 0:17
K2 VlanStatsMan Revi 2.00 0.48 15 2 100 500 0 0 0 1:12
K2 Packet Memory Dia 2.00 0.00 15 9 100 500 0 1 0 2:30
K2 L2 Aging Table Re 2.00 0.12 20 3 100 500 0 0 0 0:25
K2PacketBufMonitor-P 3.00 2.89 10 1 100 500 3 2 1 5:17
RkiosPortMan Port Re 2.00 0.44 12 7 100 500 0 0 0 1:32
Rkios Module State R 4.00 0.01 40 2 100 500 0 0 0 0:05
Rkios Online Diag Re 4.00 0.01 40 0 100 500 0 0 0 0:04
MatMan Review 0.50 0.00 4 0 100 500 0 0 0 0:00
RkiosIpPbr IrmPort R 2.00 0.00 10 2 100 500 0 0 0 0:09
RkiosIpWccp IrmWccpW 2.00 0.00 10 0 100 500 0 0 0 0:00
RkiosAclMan Review 3.00 0.05 30 0 100 500 0 0 0 0:09
EthHoleLinecardMan(1 5.00 0.37 10 8 100 500 0 0 0 0:56
Grandprix 1-1 Stub R 2.00 0.17 15 3 100 500 0 0 0 0:26
Grandprix 1-2 Stub R 2.00 0.23 15 48 100 500 0 0 0 0:35
Grandprix 1-1 Stats 2.00 0.38 4 1 100 500 0 0 0 0:52
Grandprix 1-2 Stats 2.00 0.18 4 1 100 500 0 0 0 0:34
GalLinecardDiag Mana 1.00 0.00 10 0 100 500 0 0 0 0:00
GalLinecardDiag Mana 1.00 0.00 10 0 100 500 0 0 0 0:00
GalLinecardDiag Mana 1.00 0.00 10 0 100 500 0 0 0 0:00
-------------
%CPU Totals 209.80 81.20
Allocation ceiling Current allocation
------------------ ------------------
kbytes % in use kbytes % in use
Linecard 1's Store 258.00 32% 84.99 100%
Linecard 2's Store 258.00 84% 217.13 100%
Linecard 3's Store 258.00 84% 217.14 100%
TSM objects ------------------ ------------------
PacketInfoItem 781.25 0% 1.28 0%
VbufNodes2400 80.50 0% 0.00 0%
VbufNodes1600 55.50 0% 8.67 0%
VbufNodes400 288.00 0% 7.87 21%
VbufNodes64 60.00 0% 3.28 0%
VbufNodes4200 68.37 0% 0.00 0%
PacketBufRaw 25219.50 100% 25219.50 100%
PacketBufRawJumbo 184.29 100% 184.29 100%
RkiosSysPacketBuf 250.00 0% 4.12 0%
Packet 1925.39 2% 50.19 99%
MatEntrys 9600.00 0% 4.98 100%
MatEntryTableIterato 1.00 0% 0.03 0%
PimPhyports 1093.75 12% 131.25 100%
PimPorts 921.87 18% 169.62 100%
PimModules 262.00 1% 3.07 100%
PimSlots 18.00 1% 0.21 100%
PimChassis 40.12 6% 2.50 100%
PimQuack 1.64 0% 0.00 0%
EbmVlans 5408.00 2% 110.90 100%
EbmVlanGroupEntrys 3968.00 0% 9.32 100%
EbmPorts 392.00 17% 70.05 100%
EbmPortHostEntrys 448.00 0% 0.00 0%
EbmIeNodes 540.00 0% 5.27 100%
EbmPortVlanAclFeatur 3072.00 0% 0.00 0%
EbmPortVlanMap Alloc 64.00 0% 0.00 0%
EbmSortedHostTableIt 1.75 0% 0.05 0%
EbmSortedGroupTableI 1.75 0% 0.05 0%
EbmHostRedundancyMan 1082.81 0% 0.00 0%
IrmVrfs 318.00 0% 2.48 100%
IrmFibLoadBalances 1280.00 0% 0.00 0%
IrmFibAdjs 1920.00 1% 26.95 91%
IrmPortEtherAddrEntr 500.00 0% 0.00 0%
IrmFibEntries 12288.00 0% 37.64 92%
IrmMfibEntryMemMan 6656.00 0% 0.10 100%
IrmWccpMemMan 34.25 0% 0.00 0%
IrmWccpServiceGroupL 0.06 0% 0.00 0%
AclOp 2176.00 0% 0.00 0%
AclOpAceSet 3264.00 0% 0.00 0%
AclClassifier 1280.00 0% 1.52 100%
AclFeature 2574.79 0% 1.78 100%
Acl 1536.00 0% 1.87 100%
Ace24 10880.00 0% 5.46 100%
Ace48 17408.00 0% 0.37 100%
AclFlowLabelListNode 7616.00 0% 0.00 0%
AceActionDescStorage 1088.00 0% 0.00 0%
AclListNode 512.00 0% 0.30 100%
AceListNode 25600.00 0% 0.03 0%
AclClassifierActionL 2560.00 0% 1.36 100%
AclLayerFeatureListN 512.00 0% 0.00 0%
AclClassifierListNod 512.00 0% 0.00 0%
TableMapMan NameToTa 27.00 0% 0.00 0%
TableMapAllocator 128.00 0% 0.00 0%
FlatAcl 512.00 0% 0.00 0%
FlatAce24 11264.00 0% 0.00 0%
FlatAce48 17408.00 0% 0.00 0%
FlatAceActionListNod 307200.00 0% 0.09 0%
FlatAclOpSetStorage 3072.00 0% 0.00 0%
FlatAclCacheNode 4224.00 0% 0.00 0%
FlatAclListNode 256.00 0% 0.00 0%
QosFeatureClassifier 343.51 0% 0.00 0%
QosFeatureClassifier 687.03 0% 0.00 0%
QosClassifierActionL 9618.43 0% 0.00 0%
QosNestedClassifierA 15114.68 0% 0.00 0%
QosPortVlanAclFeatur 1224.00 0% 0.00 0%
QoS Policers 37000.00 0% 0.00 0%
Qos FlowFnf 7.81 0% 0.00 0%
KxAclPath 1024.00 0% 0.00 0%
KxAclPathListNode 1280.00 0% 0.00 0%
KxAclConstPathListNo 1280.00 0% 0.00 0%
IndexCache 800.78 0% 0.00 0%
ConfigToHwAfMap 203.27 0% 0.00 0%
HwToCfgAceMap 816.00 0% 0.00 0%
CommandTables 48.00 22% 10.59 100%
K2FibWccpServiceGrou 162.59 0% 0.00 0%
K2FibPbrFlatRouteMap 467.79 0% 0.00 0%
K2FibPbrExpandedFlat 2304.00 0% 0.00 0%
K2FibPbrFlatRouteMap 320.00 0% 0.00 0%
K2FibVrfs 152.00 0% 1.18 100%
K2 Netflow FPT Entri 11.97 0% 0.00 0%
K2TxPacket 384.00 0% 0.11 0%
K2TxPacketInfo 256.00 0% 0.53 0%
KxAclLabeledFlatAcl 3584.00 0% 0.00 0%
KxAclLabeledFlatAclE 3072.00 0% 0.00 0%
EbmVlanHostEntrys 2048.00 3% 76.37 80%
GalGbicEntrys 5.48 5% 0.32 100%
Rkios QoS PolicyMaps 357.57 0% 0.00 0%
AclClassifierIdToCla 48.00 0% 0.00 0%
Rkios QoS ClassMaps 1024.00 0% 0.03 100%
AclToIosFilterMapLis 384.00 0% 0.00 0%
Rkios QoS Policers 3500.00 0% 0.00 0%
RkiosAclMan NamedGal 43.75 0% 0.00 0%
EpmPolicyListNode 120.00 0% 0.00 0%
EpmAceListNode 192.00 0% 0.00 0%
RkiosAclSecurityEpmP 4080.00 0% 0.00 0%
Rkios Acl VlanMaps 128.00 0% 0.00 0%
Rkios Acl VlanMapEnt 937.50 0% 0.00 0%
RkiosTableMap Galios 3.00 0% 0.00 0%
IrmMfibIntrfs 3072.00 0% 0.00 0%
Event Nodes 160.00 0% 1.15 2%
Event Nodes 160.00 1% 15.25 10%
IrmFlows 384.00 0% 0.00 0%
K2AclMappingTableEnt 100.26 0% 0.00 0%
K2AclMappingTableEnt 66.84 0% 0.00 0%
K2AclTagPairSignatur 1792.00 0% 0.00 0%
RkisoIpPbrRouteMaps 97.65 0% 0.00 0%
------------------ ------------------
TSM totals 599301.60 4% 26172.15 99%
I am also seeing show spanning-tree detail | include ieee|from|occur|is exec
VLAN0001 is executing the ieee compatible Spanning Tree protocol
Number of topology changes 33 last change occurred 03:19:30 ago
from Port-channel33
VLAN0002 is executing the ieee compatible Spanning Tree protocol
Number of topology changes 201 last change occurred 00:47:26 ago
from Port-channel33
VLAN0010 is executing the ieee compatible Spanning Tree protocol
Number of topology changes 17 last change occurred 03:41:16 ago
from GigabitEthernet2/43
VLAN0033 is executing the ieee compatible Spanning Tree protocol
Number of topology changes 35 last change occurred 03:19:30 ago
from Port-channel33
VLAN0034 is executing the ieee compatible Spanning Tree protoco
- Labels:
-
Other Switching
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-05-2017 01:24 PM
It appears that Cat4k Mgmt LoPri and spanning tree processes are taking up 75% of the processes.
Have a look at this link for more info:
http://www.cisco.com/c/en/us/support/docs/switches/catalyst-4000-series-switches/65591-cat4500-high-cpu.html
Also, anything in the logs?
HTH
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-05-2017 01:27 PM
Hi
Thanks for the prompt response, the log is small and these are the entries. I have just inherited this switch and was very concerned by the CPU
000161: 00:48:05: STP: VLAN0002 Topology Change rcvd on Po12
000162: 00:48:05: STP: VLAN0002 sent Topology Change Notice on Gi1/16
000163: 00:52:55: STP: VLAN0002 Topology Change rcvd on Po19
000164: 00:52:55: STP: VLAN0002 sent Topology Change Notice on Gi1/16
000165: 00:52:57: STP: VLAN0002 Topology Change rcvd on Po19
000166: 00:52:57: STP: VLAN0002 sent Topology Change Notice on Gi1/16
000167: 00:52:57: STP: VLAN0002 Topology Change rcvd on Po19
000168: 00:52:59: STP: VLAN0002 Topology Change rcvd on Po19
000169: 00:52:59: STP: VLAN0002 sent Topology Change Notice on Gi1/16
000170: 00:52:59: STP: VLAN0002 Topology Change rcvd on Po19
000171: 00:54:14: STP: VLAN0002 Topology Change rcvd on Po5
000172: 00:54:14: STP: VLAN0002 sent Topology Change Notice on Gi1/16
000173: 00:54:40: STP: VLAN0002 Topology Change rcvd on Po19
000174: 00:54:40: STP: VLAN0002 sent Topology Change Notice on Gi1/16
000175: 00:54:42: STP: VLAN0002 Topology Change rcvd on Po19
000176: 00:54:42: STP: VLAN0002 sent Topology Change Notice on Gi1/16
000177: 00:54:47: STP: VLAN0002 Topology Change rcvd on Po5
000178: 00:54:47: STP: VLAN0002 sent Topology Change Notice on Gi1/16
000179: 00:55:52: STP: VLAN0002 Topology Change rcvd on Po21
000180: 00:55:52: STP: VLAN0002 sent Topology Change Notice on Gi1/16
000181: 00:56:44: STP: VLAN0002 Topology Change rcvd on Po10
000182: 00:56:44: STP: VLAN0002 sent Topology Change Notice on Gi1/16
000183: 00:56:45: STP: VLAN0002 Topology Change rcvd on Po10
000184: 00:56:45: STP: VLAN0002 sent Topology Change Notice on Gi1/16
000185: 00:56:46: STP: VLAN0002 Topology Change rcvd on Po10
000186: 00:56:56: STP: VLAN0002 Topology Change rcvd on Po13
000187: 00:56:56: STP: VLAN0002 sent Topology Change Notice on Gi1/16
000188: 00:56:58: STP: VLAN0002 Topology Change rcvd on Po17
000189: 00:56:58: STP: VLAN0002 sent Topology Change Notice on Gi1/16
000190: .Jun 5 08:01:51: %C4K_L2MAN-6-INVALIDSOURCEADDRESSPACKET: Packet received with invalid source MAC address (00:00:00:00:00:00) on port Po2 in vlan 34
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-05-2017 01:46 PM
Hi,
It seems like you may be dealing with some sort of stp issue, as the topology change is very frequent which also correspond to high cpu for spanning tree. What is the physical/logical topology of the network?
HTH
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide
