06-01-2010 08:42 AM - edited 03-06-2019 11:21 AM
Hi All,
Greetings,
Someone could help me with the following issue, I have a devices 4506 with a supervisor 6 E, the equipment has been working properly but
from one moment to another CPU usage went up 80%
SW2_4506#show process cpu | exc 0.00
CPU utilization for five seconds: 79%/9%; one minute: 79%; five minutes: 79%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
38 1430548 6238658 229 0.07% 0.05% 0.07% 0 IDB Work
53 2983375401428662390 208 3.83% 3.79% 3.77% 0 Cat4k Mgmt HiPri
54 4158989881851588418 224 64.47% 64.42% 64.03% 0 Cat4k Mgmt LoPri
103 21781572 981500164 22 0.15% 0.20% 0.23% 0 IP Input
109 1234024 65311050 18 0.23% 0.26% 0.25% 0 Spanning Tree
214 521240 384154925 1 0.07% 0.08% 0.07% 0 HSRP Common
217 851449481842211393 46 0.39% 0.35% 0.42% 0 IP SNMP
218 14951900 920455180 16 0.07% 0.07% 0.09% 0 PDU DISPATCHER
219 78767604 922349560 85 1.03% 0.58% 0.62% 0 SNMP ENGINE
SW2_4506#
The process by which this high intake is Cat4k Mgmt LoPri, verifying the output of the show processes sorted cpu I found following:
SW2_4506#show process cpu sorted | exc 0.00
CPU utilization for five seconds: 79%/9%; one minute: 79%; five minutes: 79%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
54 4161304801852801555 224 64.71% 64.33% 64.22% 0 Cat4k Mgmt LoPri
53 2983434121428674551 208 3.59% 3.76% 3.76% 0 Cat4k Mgmt HiPri
219 78769972 922369535 85 0.47% 0.73% 0.67% 0 SNMP ENGINE
217 851465201842251326 46 0.39% 0.48% 0.46% 0 IP SNMP
109 1234196 65312628 18 0.23% 0.27% 0.26% 0 Spanning Tree
103 21782384 981527554 22 0.23% 0.25% 0.24% 0 IP Input
218 14952416 920475154 16 0.15% 0.12% 0.10% 0 PDU DISPATCHER
214 521280 384173951 1 0.07% 0.08% 0.07% 0 HSRP Common
SW2_4506#
Verified the output show platform health I found that K5CpuMan Review and K5ForerunnerPacketMa are the process with longer CPU use how you see in the output bellow
SW2_4506#show platform health
%CPU %CPU RunTimeMax Priority Average %CPU Total
Target Actual Target Actual Fg Bg 5Sec Min Hour CPU
RkiosObflMan 0.50 0.00 4 0 100 500 0 0 0 4:20
GalChassisVp-review 3.00 0.10 10 30 100 500 0 0 0 97:58
S2w-JobEventSchedule 10.00 0.25 10 7 100 500 0 0 0 250:00
Stub-JobEventSchedul 10.00 1.28 10 15 100 500 1 1 0 1571:59
Lj-poll 1.00 0.02 2 0 100 500 0 0 0 21:33
StatValueMan Update 1.00 0.03 1 0 100 500 0 0 0 36:57
Pim-review 0.10 0.00 1 0 100 500 0 0 0 5:19
Ebm-host-review 1.00 0.00 8 2 100 500 0 0 0 14:29
Ebm-host-util-review 1.00 0.00 10 0 100 500 0 0 0 0:10
Ebm-port-review 0.10 0.00 1 0 100 500 0 0 0 0:26
Protocol-aging-revie 0.20 0.00 2 0 100 500 0 0 0 0:01
EbmHostRedundancyMan 2.00 0.00 20 0 100 500 0 0 0 0:00
Acl-Flattener 1.00 0.00 10 2 100 500 0 0 0 0:00
IrmFibThrottler Thro 2.00 0.01 7 0 100 500 0 0 0 7:14
GalChassisVp Ondeman 2.00 0.00 2 0 100 500 0 0 0 0:00
KxAclPathMan create/ 1.00 0.00 10 5 100 500 0 0 0 0:03
KxAclPathMan update 2.00 0.00 10 26 100 500 0 0 0 15:20
clPathMan reprogr 1.00 0.00 2 0 100 500 0 0 0 0:00
GalK5TatooineStatsMa 0.70 0.01 4 0 100 500 0 0 0 22:30
MOL FastDropReview 2.00 0.00 15 0 100 500 0 0 0 0:00
RkiosL3PortMan AclFe 2.00 0.00 15 2 100 500 0 0 0 0:00
K5L3FlcMan FwdEntry 2.00 0.00 15 0 100 500 0 0 0 0:00
K5L3FlcMan Consisten 2.00 0.56 15 7 100 500 1 0 0 805:49
FwdEntry Zombie Revi 2.00 0.00 15 0 100 500 0 0 0 0:00
K5L3FlcMan Cam Shuff 4.00 0.00 25 0 100 500 0 0 0 0:00
K5L3Unciast IFE Revi 2.00 0.00 15 0 100 500 0 0 0 0:00
K5L3UnicastRpf IFE R 2.00 0.00 15 0 100 500 0 0 0 0:00
K5L3Unicast Fwd Entr 2.00 0.00 15 0 100 500 0 0 0 0:00
K5L3McastMan IrmMfib 2.00 0.00 15 0 100 500 0 0 0 0:00
K5L3McastMan ImeSync 2.00 0.00 15 0 100 500 0 0 0 0:02
K5L3Unicast Fwd Entr 2.00 0.00 15 0 100 500 0 0 0 0:00
K5L3Unicast Adj Grou 2.00 0.00 15 0 100 500 0 0 0 0:00
K5L3Unicast Adj Chan 2.00 0.00 15 0 100 500 0 0 0 0:28
K5L3Unicast Adj Tabl 2.00 0.00 15 7 100 500 0 0 0 105:41
K5L3AdjStatsMan Revi 2.00 0.05 10 8 100 500 0 0 0 322:10
K5L3McastMan RetSync 1.00 0.00 8 0 100 500 0 0 0 0:00
K5FlcHitMan review 2.00 0.01 5 2 100 500 0 0 0 149:05
K5PortMan Regular Re 2.00 0.25 15 10 100 500 0 0 0 352:02
K5PortMan Ondemand L 3.00 0.53 30 0 100 500 0 0 0 423:30
K5PortMan Stats Revi 2.00 0.05 15 1 100 500 0 0 0 24:19
K5PortMan Tx Queue R 3.00 0.00 15 1 100 500 0 0 0 0:00
K5L2 Vlan Table Revi 2.00 0.00 12 8 100 500 0 0 0 4:18
K5 L2 Aging Table Re 2.00 0.07 20 4 100 500 0 0 0 127:40
K5 L2 Unicast Addres 2.00 0.00 20 2 100 500 0 0 0 0:00
K5 L2 Multicast Addr 2.00 0.00 20 0 100 500 0 0 0 0:03
K5 L2 Hardware Addre 2.00 0.00 20 10 100 500 0 0 0 234:10
K5 L2 Hardware Mac L 1.00 0.00 2 0 100 500 0 0 0 0:00
K5RetStatsMan Review 2.00 0.00 5 0 100 500 0 0 0 0:00
K5CpuMan Review 30.00 83.63 30 41 100 500 189 170 132 11419:43
K5ForerunnerPacketMa 1.50 54.45 4 0 100 500 111 100 76 5434:56
K5ForerunnerPacketMa 0.70 0.24 4 0 100 500 0 0 0 229:50
K5QosDhmMan Rate DBL 2.00 4.23 7 5 100 500 5 4 3 3818:16
K5QosPolicerStatsMan 1.00 0.00 10 0 100 500 0 0 0 4:21
K5VlanStatsReview 2.00 0.78 10 3 100 500 1 0 0 768:28
K5AclMan-labeledFlat 1.00 0.00 10 1 100 500 0 0 0 0:00
K5AclLabelMan-punted 1.00 0.00 10 0 100 500 0 0 0 0:00
K5AclCamMan stale en 1.00 0.00 10 5 100 500 0 0 0 0:04
K5AclCamMan Audit re 1.00 2.50 10 5 100 500 2 2 1 1997:08
K5AclCamStatsMan hw 3.00 0.20 10 7 100 500 0 0 0 193:40
K5Acl Action Update 2.00 0.00 15 7 100 500 0 0 0 0:00
VSI DMA Local Jawa R 1.00 0.09 6 0 100 500 0 0 0 73:29
VSI DMA Darkside Rev 1.00 0.00 6 0 100 500 0 0 0 0:00
00 0.00 100 5ble Review 1:260 6 10SI DMA Slot-030 0 0 0:00lot-0 0 100 0:00
G00 0.00 0 0 bmHostTabl0 8 0 0 2Man Port0 0 odule Sta.02 0 0 iag Re 0 100 50 0 22 2.00 0.0 500sIpWccp IrmWc :00
LocalJawa 0 0 0 0:5 4 100 500 :02
SW2_4506#show ip traffic
IP statistics:
Rcvd: 991804425 total, 975895167 local destination
0 format errors, 0 checksum errors, 2402921 bad hop count
0 unknown protocol, 12284439 not a gateway
0 security failures, 0 bad options, 1083246 with options
Opts: 98552 end, 38 nop, 0 basic security, 0 loose source route
0 timestamp, 0 extended security, 0 record route
0 stream ID, 0 strict source route, 984656 alert, 0 cipso, 0 ump
0 other
Frags: 0 reassembled, 0 timeouts, 0 couldn't reassemble
176 fragmented, 0 couldn't fragment
Bcast: 19259087 received, 115872 sent
Mcast: 36587362 received, 35908604 sent
Sent: 957941998 generated, 3166031479 forwarded
Drop: 18407 encapsulation failed, 0 unresolved, 0 no adjacency
3655 no route, 0 unicast RPF, 0 forced drop
0 options denied, 0 source IP address zero
ICMP statistics:
Rcvd: 0 format errors, 0 checksum errors, 0 redirects, 2140 unreachable
72781 echo, 33460 echo reply, 0 mask requests, 0 mask replies, 0 quench
0 parameter, 2 timestamp, 0 info request, 0 other
0 irdp solicitations, 0 irdp advertisements
Sent: 6432 redirects, 4438 unreachable, 33571 echo, 72781 echo reply
0 mask requests, 0 mask replies, 0 quench, 2 timestamp
0 info reply, 1797935 time exceeded, 0 parameter problem
0 irdp solicitations, 0 irdp advertisements
TCP statistics:
Rcvd: 424811 total, 4 checksum errors, 71 no port
Sent: 518597 total
Probe statistics:
Rcvd: 0 address requests, 0 address replies
0 proxy name requests, 0 where-is requests, 0 other
Sent: 0 address requests, 0 address replies (0 proxy)
0 proxy name replies, 0 where-is replies
UDP statistics:
Rcvd: 975212598 total, 1 checksum errors, 19186986 no port
Sent: 955303199 total, 0 forwarded broadcasts
BGP statistics:
Rcvd: 0 total, 0 opens, 0 notifications, 0 updates
0 keepalives, 0 route-refresh, 0 unrecognized
Sent: 0 total, 0 opens, 0 notifications, 0 updates
0 keepalives, 0 route-refresh
EIGRP-IPv4 statistics:
Rcvd: 0 total
Sent: 0 total
PIMv2 statistics: Sent/Received
Total: 0/0, 0 checksum errors, 0 format errors
Registers: 0/0 (0 non-rp, 0 non-sm-group), Register Stops: 0/0, Hellos: 0/0
Join/Prunes: 0/0, Asserts: 0/0, grafts: 0/0
Bootstraps: 0/0, Candidate_RP_Advertisements: 0/0
Queue drops: 0
State-Refresh: 0/0
IGMP statistics: Sent/Received
Total: 0/0, Format errors: 0/0, Checksum errors: 0/0
Host Queries: 0/0, Host Reports: 0/0, Host Leaves: 0/0
DVMRP: 0/0, PIM: 0/0
Queue drops: 0
OSPF statistics:
Rcvd: 149305 total, 0 checksum errors
140388 hello, 137 database desc, 14 link state req
5389 link state updates, 3377 link state acks
Sent: 205182 total
196996 hello, 70 database desc, 10 link state req
6559 link state updates, 1558 link state acks
ARP statistics:
Rcvd: 15671151 requests, 345047 replies, 0 reverse, 0 other
Sent: 22108 requests, 511910 replies (90924 proxy), 0 reverse
Drop due to input queue full: 0
SW2_4506#
SW2_4506#show processes cpu history table
%CPU utilization history table is not enabled
Tigo_SW2_4506_Bquilla#show processes cpu history
7777777777777777777777777777777777777777777777888887777777
9999999999999999999999999999999999999999999999000009999999
100
90
80 **********************************************************
70 **********************************************************
60 **********************************************************
50 **********************************************************
40 **********************************************************
30 **********************************************************
20 **********************************************************
10 **********************************************************
0....5....1....1....2....2....3....3....4....4....5....5....
0 5 0 5 0 5 0 5 0 5
CPU% per second (last 60 seconds)
8887888888887778877778887888888888888888888888888888888888
0009000001009990079990019000001110011010111111110001111010
100
90
80 ##########################################################
70 ##########################################################
60 ##########################################################
50 ##########################################################
40 ##########################################################
30 ##########################################################
20 ##########################################################
10 ##########################################################
0....5....1....1....2....2....3....3....4....4....5....5....
0 5 0 5 0 5 0 5 0 5
CPU% per minute (last 60 minutes)
* = maximum CPU% # = average CPU%
8776555777888888898877788886577854223333333343333535433334342242233333
1797269178010201025088903409908200869899898808788197494639977898389489
100
90 **
80 #** *#######*#########* **
70 ###* *###################* *#*
60 ###* **####################***#* *
50 ####**######################**#** * * * * *
40 ############################**#*** ****************** * *** * ** **
30 ###############################********************************* *****
20 ##################################**#########################*#**#####
10 ######################################################################
0....5....1....1....2....2....3....3....4....4....5....5....6....6....7.
0 5 0 5 0 5 0 5 0 5 0 5 0
CPU% per hour (last 72 hours)
* = maximum CPU% # = average CPU%
SW2_4506#
Can your help me giving me some clue to what may be the cause of higher CPU usage and how I can correct IOS version of the devices is 122 (53) SG1, I upload attach file with the show tech-support of the equipmente
Thanks,
Sincerely,
PEDRO PABLO PACHECO
Support & Operation Engineer Colombia |e-mail: ppacheco@desca.com
Teléfono: +57 1 3766900 Ext. 67266| Teléfono directo: +57 1 3767266 | fax: +57 1 3766900| MSN IM: libreton3@hotmail.com | Celular: +57 3212008694
Carrera 7 No 71-52 Torre A Piso 11 Bogota D.C., Colombia
06-01-2010 10:03 AM
Observation:
54 4158989881851588418 224 64.47% 64.42% 64.03% 0 Cat4k Mgmt LoPri
When the same MAC addresses are learned and aged out on different VLANs, the Cat4k Mgmt LoPri process will cause CPU utilization to increase. This does not impact local data switching performance because the LoPri process is of low priority with limited access to the CPU.
Workaround: None. (CSCsg76868)
Bug details:
High CPU due to Mgmt LoPri process, MAC addresses being added/deleted | |
Symptom: A Catalyst 4500 switch might experience high CPU utilization due to the Cat4k Mgmt LoPri process and the K2CpuMan and K2L2 Address Table reviews (show platform health. High CPU utilization does not impact the traffic switched in hardware. Conditions: The problem is seen when a large MAC address table exists and when the switch is frequently relearning MAC addresses on multiple VLANs. Enabling service internal followed by debug platform log feature k2l2addresstable will show output similar to the following. Do not enable these commands on a production switch unless instructed by Cisco TAC. *Nov 13 12:56:32.066 CLT-1: K2L2AddressTableMan::newEntry index 61956 vlan 1020 address 00:D0:02:2D:38:1A *Nov 13 12:56:34.030 CLT-1: K2L2AddressTableMan::deleteEntry index 55620 vlan 1010 address 00:D0:02:2D:38:1A *Nov 13 12:56:34.046 CLT-1: K2L2AddressTableMan::newEntry index 55620 vlan 1010 address 00:D0:02:2D:38:1A *Nov 13 12:56:34.062 CLT-1: K2L2AddressTableMan::deleteEntry index 61956 vlan 1020 address 00:D0:02:2D:38:1A Workaround: None |
Documentation:
http://www.cisco.com/en/US/partner/docs/switches/lan/catalyst4500/release/note/OL_11511.html
Please let me know if this was helpful.
Regards,
podhillo
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide