hig processor CPU WS-C3560X-48P

networkadnen · ‎01-21-2016

Hi all,

My switch is a hig processor CPU, could you please advice me what to do.

Switch Ports Model SW Version SW Image
------ - ---- ----- ---------- ----------
* 1 54 WS-C3560X-48P 15.0(2)SE4 C3560E-UNIVERSALK9NPE-M

CPU utilization for five seconds: 99%/28%; one minute: 89%; five minutes: 90%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
153 2557429536 1689359934 1513 34.30% 36.49% 36.66% 0 Hulc LED Process
79 1502873480 895466063 1678 5.93% 4.98% 5.18% 0 RedEarth Tx Mana
78 2335980822 1535804589 1521 4.96% 3.16% 3.29% 0 RedEarth I2C dri
363 76824581 4791953 16032 2.72% 3.02% 2.94% 0 SNMP ENGINE
120 850452394 75595328 11250 1.44% 1.75% 1.78% 0 hpm counter proc
5 1297046015 57925014 22391 11.54% 1.92% 1.58% 0 Check heaps

thanks

Mark Malone · ‎01-21-2016

Hey HULC is a bug so its running at 63% really as its cosmetic but you have interrupt traffic over the recommended 5% too

Can you post the full show proc cpu sorted while its occurring

If you cant capture it live use this script below it will send it too flash when it reaches over 80% , do you have cef enabled ?

"event manager applet High_CPU

event snmp oid 1.3.6.1.4.1.9.9.109.1.1.1.1.4.1 get-type exact entry-op ge entry-val "80" exit-time 10 poll-interval 5

action 0.1 syslog msg "CPU Utilization is high"

action 0.2 cli command "enable"

action 0.4 cli command "show log | append flash:CPU_Profile.txt"

action 0.5 cli command "show process cpu sorted | append flash:CPU_Profile.txt"

action 0.6 cli command "show interfaces | append flash:CPU_Profile.txt"

action 0.7 cli command " show ip cef switching stat | append flash:CPU_Profile.txt"

action 0.8 cli command " show ip traffic | append flash:CPU_Profile.txt"

action 0.9 cli command " show int switching | append flash:CPU_Profile.txt"

action 1.0 cli command "end" "

Good doc on cpu issues

http://www.cisco.com/c/en/us/support/docs/routers/7500-series-routers/41120-highcpu-interrupts.html

networkadnen · ‎01-22-2016

hi,

this is the output, and about the cef is not enabled i juste use the switch as L2, the switch connected to this one has the same problem except th core switch.

thx.

sh processes cpu sorted
CPU utilization for five seconds: 80%/27%; one minute: 90%; five minutes: 90%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
153 2605086008 1690536536 1540 37.11% 35.98% 36.19% 0 Hulc LED Process
79 1509311667 896266812 1683 4.63% 4.56% 4.52% 0 RedEarth Tx Mana
78 2339763404 1537221370 1522 2.07% 2.89% 2.99% 0 RedEarth I2C dri
120 852083839 75679496 11259 1.75% 1.82% 1.81% 0 hpm counter proc
160 1209 85 14223 0.79% 1.18% 0.27% 1 SSH Process
279 111040904 163040315 681 0.47% 0.29% 0.28% 0 Marvell wk-a Pow
214 116496216 253887720 458 0.47% 0.28% 0.27% 0 Spanning Tree
166 335272890 15026034 22312 0.31% 0.36% 0.38% 0 HQM Stack Proces
231 90002902 764742072 117 0.31% 0.08% 0.10% 0 UDLD
54 83494482 75704341 1102 0.15% 0.15% 0.16% 0 Per-Second Jobs
52 10762190 15139291 710 0.15% 0.04% 0.00% 0 Compute load avg
63 63016744 1306775182 48 0.15% 0.05% 0.00% 0 DownWhenLooped
156 15251901 54932712 277 0.15% 0.03% 0.00% 0 HL3U bkgrd proce
198 117382341 1688005580 69 0.15% 0.05% 0.05% 0 IP ARP Retry Age
14 6550872 77652644 84 0.00% 0.00% 0.00% 0 ARP Background
15 0 1 0 0.00% 0.00% 0.00% 0 AAA_SERVER_DEADT
16 0 1 0 0.00% 0.00% 0.00% 0 Policy Manager
18 0 1 0 0.00% 0.00% 0.00% 0 IFS Agent Manage
13 5996207 18155719 330 0.00% 0.00% 0.00% 0 ARP Input
20 4183370 72804601 57 0.00% 0.00% 0.00% 0 IPC Mcast Pendin
12 102 6 17000 0.00% 0.00% 0.00% 0 Image License br
11 161169541 1258375 128078 0.00% 0.13% 0.15% 0 Licensing Auto U
17 16 12 1333 0.00% 0.00% 0.00% 0 Entity MIB API
19 824100 15027989 54 0.00% 0.00% 0.00% 0 IPC Event Notifi
25 4007965 72804595 55 0.00% 0.00% 0.00% 0 IPC Deferred Por
26 0 1 0 0.00% 0.00% 0.00% 0 IPC Process leve
27 518141 5033483 102 0.00% 0.00% 0.00% 0 IPC Seat Manager
21 51066 1258468 40 0.00% 0.00% 0.00% 0 IPC Dynamic Cach
29 0 1 0 0.00% 0.00% 0.00% 0 IPC Seat RX Cont
30 0 1 0 0.00% 0.00% 0.00% 0 IPC Seat TX Cont
22 0 1 0 0.00% 0.00% 0.00% 0 IPC Session Serv
32 904835 15100414 59 0.00% 0.00% 0.00% 0 IPC Loadometer
33 84 6 14000 0.00% 0.00% 0.00% 0 PrstVbl
23 0 1 0 0.00% 0.00% 0.00% 0 IPC Zone Manager
24 4457179 72804601 61 0.00% 0.00% 0.00% 0 IPC Periodic Tim
36 3455033 75679586 45 0.00% 0.00% 0.00% 0 GraphIt
28 195350 4320147 45 0.00% 0.00% 0.00% 0 IPC Check Queue
38 0 2 0 0.00% 0.00% 0.00% 0 SMART
39 0 2 0 0.00% 0.00% 0.00% 0 XML Proxy Client
40 0 1 0 0.00% 0.00% 0.00% 0 ARP Snoop
41 2718960 74446329 36 0.00% 0.01% 0.00% 0 Dynamic ARP Insp
42 0 1 0 0.00% 0.00% 0.00% 0 Critical Bkgnd
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
43 41437608 92830301 446 0.00% 0.06% 0.08% 0 Net Background
44 9 75 120 0.00% 0.00% 0.00% 0 IDB Work
31 367983 7567976 48 0.00% 0.00% 0.00% 0 IPC Keep Alive M
46 2741239 74446279 36 0.00% 0.00% 0.00% 0 TTY Background
47 418 941 444 0.00% 0.00% 0.00% 0 SXP CORE
48 9 6 1500 0.00% 0.00% 0.00% 0 IF-MGR control p
49 8090 78271 103 0.00% 0.00% 0.00% 0 IF-MGR event pro
50 0 1 0 0.00% 0.00% 0.00% 0 Inode Table Dest
51 17 35 485 0.00% 0.00% 0.00% 0 Net Input
34 0 1 0 0.00% 0.00% 0.00% 0 Crash writer
53 48657900 1305142 37281 0.00% 0.07% 0.05% 0 Per-minute Jobs
10 0 3 0 0.00% 0.00% 0.00% 0 License Client N
55 0 1 0 0.00% 0.00% 0.00% 0 AggMgr Process
56 16811 161864 103 0.00% 0.00% 0.00% 0 Transport Port A
57 4689104 22145082 211 0.00% 0.01% 0.00% 0 HC Counter Timer
35 0 1 0 0.00% 0.00% 0.00% 0 Exception contro
37 8468 6400 1323 0.00% 0.00% 0.00% 0 crypto sw pk pro
9 4760 138717 34 0.00% 0.00% 0.00% 0 WATCH_AFS
61 388296 2817114 137 0.00% 0.00% 0.00% 0 EEM ED ND
62 41 77 532 0.00% 0.00% 0.00% 0 USB Startup
45 24800 184190 134 0.00% 0.00% 0.00% 0 Logger
64 0 1 0 0.00% 0.00% 0.00% 0 HULC PBR Tcam Me
65 0 1 0 0.00% 0.00% 0.00% 0 HULC ACL Tcam Me
66 0 1 0 0.00% 0.00% 0.00% 0 Hulc Warm Upgrad
67 0 1 0 0.00% 0.00% 0.00% 0 HRPC lpip reques
68 0 2 0 0.00% 0.00% 0.00% 0 HLPIP Sync Proce
69 0 1 0 0.00% 0.00% 0.00% 0 HRPC hnetwpol re
8 0 2 0 0.00% 0.00% 0.00% 0 Timers
71 0 1 0 0.00% 0.00% 0.00% 0 HRPC actual powe
72 16909724 302576697 55 0.00% 0.00% 0.00% 0 HUSB Console
73 181358 2516949 72 0.00% 0.00% 0.00% 0 PSP Timer
74 0 1 0 0.00% 0.00% 0.00% 0 HULC QM Tcam Mem
75 3616 11 328727 0.00% 0.00% 0.00% 0 X2 HAMM detect p
76 0 1 0 0.00% 0.00% 0.00% 0 Madmax Process
7 0 1 0 0.00% 0.00% 0.00% 0 DiscardQ Backgro
6 13297 24376 545 0.00% 0.00% 0.00% 0 Pool Manager
5 1298929339 58005781 22393 0.00% 1.69% 1.56% 0 Check heaps
59 8 54 148 0.00% 0.00% 0.00% 0 EEM ED Identity
81 1899116 15135985 125 0.00% 0.00% 0.00% 0 HULC Thermal Pro
82 0 1 0 0.00% 0.00% 0.00% 0 HRPC hsm request
4 0 1 0 0.00% 0.00% 0.00% 0 Connection Mgr
84 285 12 23750 0.00% 0.00% 0.00% 0 Stack Mgr Notifi
85 83871 3570705 23 0.00% 0.00% 0.00% 0 Adjust Regions
86 4507759 37544599 120 0.00% 0.00% 0.00% 0 hrpc -> response
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
87 0 47 0 0.00% 0.00% 0.00% 0 hrpc -> request
88 3590896 12574790 285 0.00% 0.00% 0.00% 0 hrpc <- response
89 0 1 0 0.00% 0.00% 0.00% 0 HRPC hcomp reque
90 19601709 378001627 51 0.00% 0.02% 0.00% 0 yeti2_emac_proce
91 0 3 0 0.00% 0.00% 0.00% 0 HULC Device Mana
3 5253 27240 192 0.00% 0.00% 0.00% 0 hulc_entropy_thr
93 0 2 0 0.00% 0.00% 0.00% 0 HRPC hdm blockin
94 1249186 15027971 83 0.00% 0.00% 0.00% 0 HIPC bkgrd proce
95 0 1 0 0.00% 0.00% 0.00% 0 RTTYS Process
96 22884 183389 124 0.00% 0.00% 0.00% 0 Hulc Port-Securi
97 0 1 0 0.00% 0.00% 0.00% 0 HRPC hpsecure re
98 0 1 0 0.00% 0.00% 0.00% 0 HRPC hlfm reques
2 4650032 15135928 307 0.00% 0.00% 0.00% 0 Load Meter
100 3530924 74446273 47 0.00% 0.00% 0.00% 0 HLFM aging proce
101 120115011 1689143954 71 0.00% 0.02% 0.02% 0 HLFM address ret
102 0 1 0 0.00% 0.00% 0.00% 0 HRPC hrcmd reque
103 24 18 1333 0.00% 0.00% 0.00% 0 HRPC emac reques
104 0 1 0 0.00% 0.00% 0.00% 0 HRPC hulc misc r
105 1591045 25112535 63 0.00% 0.00% 0.00% 0 HVLAN main bkgrd
106 0 2 0 0.00% 0.00% 0.00% 0 HVLAN Mapped Vla
58 33779672 252125 133982 0.00% 0.11% 0.05% 0 SFF8472
108 9 27 333 0.00% 0.00% 0.00% 0 HRPC vlan reques
109 0 1 0 0.00% 0.00% 0.00% 0 HULC VLAN REF Ba
110 55344 1258469 43 0.00% 0.00% 0.00% 0 hulc_xps_process
111 0 1 0 0.00% 0.00% 0.00% 0 HRPC ilp request
112 4226077 74446976 56 0.00% 0.00% 0.00% 0 Hulc ILP Alchemy
113 2852124 5033482 566 0.00% 0.00% 0.00% 0 Strider Tcam Mem
114 0 1 0 0.00% 0.00% 0.00% 0 HULC PM Vector P
115 0 1 0 0.00% 0.00% 0.00% 0 HPM Msg Retry Pr
60 0 107 0 0.00% 0.00% 0.00% 0 EEM ED MAT
117 299959049 1306620743 229 0.00% 0.58% 0.62% 0 hpm main process
118 14547 174864 83 0.00% 0.00% 0.00% 0 HPM Stack Sync P
119 0 1 0 0.00% 0.00% 0.00% 0 HRPC pm request
70 0 1 0 0.00% 0.00% 0.00% 0 HRPC EnergyWise
121 0 1 0 0.00% 0.00% 0.00% 0 HRPC pm-counters
122 0 1 0 0.00% 0.00% 0.00% 0 hpm vp events ca
123 0 1 0 0.00% 0.00% 0.00% 0 HRPC hcmp reques
124 2057 36036 57 0.00% 0.00% 0.00% 0 HCEF ADJ Refresh
125 0 1 0 0.00% 0.00% 0.00% 0 hl3md_rpfq_thrl_
126 1389358 37544579 37 0.00% 0.00% 0.00% 0 hl3mm_rp
127 0 1 0 0.00% 0.00% 0.00% 0 HACL Queue Proce
128 0 1 0 0.00% 0.00% 0.00% 0 HRPC acl request
129 42 365 115 0.00% 0.00% 0.00% 0 HACL Acl Manager
130 0 1 0 0.00% 0.00% 0.00% 0 HRPC aim request
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
131 0 3 0 0.00% 0.00% 0.00% 0 HRPC axiom-mgmt
132 1458778 37544560 38 0.00% 0.00% 0.00% 0 Hulc Axiom Mgmt
133 0 1 0 0.00% 0.00% 0.00% 0 HRPC backup inte
134 74815 46088 1623 0.00% 0.00% 0.00% 0 SpanTree Helper
135 0 1 0 0.00% 0.00% 0.00% 0 HRPC cdp request
136 0 1 0 0.00% 0.00% 0.00% 0 HULC CISP Proces
137 0 1 0 0.00% 0.00% 0.00% 0 HRPC dot1x reque
138 0 3 0 0.00% 0.00% 0.00% 0 HULC DOT1X Proce
139 0 1 0 0.00% 0.00% 0.00% 0 HRPC lldp reques
77 0 1 0 0.00% 0.00% 0.00% 0 HRPC asic-stats
141 0 1 0 0.00% 0.00% 0.00% 0 HRPC sdm request
142 0 1 0 0.00% 0.00% 0.00% 0 SMI MSG Retry Pr
143 0 1 0 0.00% 0.00% 0.00% 0 HRPC Smart Insta
144 49913585 360375154 138 0.00% 0.07% 0.08% 0 Hulc Storm Contr
145 0 2 0 0.00% 0.00% 0.00% 0 HSTP Sync Proces
146 0 1 0 0.00% 0.00% 0.00% 0 HRPC stp_cli req
147 0 1 0 0.00% 0.00% 0.00% 0 HRPC stp_state_s
148 0 2 0 0.00% 0.00% 0.00% 0 S/W Bridge Proce
149 0 1 0 0.00% 0.00% 0.00% 0 HRPC hudld reque
150 0 1 0 0.00% 0.00% 0.00% 0 HRPC vqpc reques
151 0 1 0 0.00% 0.00% 0.00% 0 HRPC l2pt qnq rp
152 0 1 0 0.00% 0.00% 0.00% 0 HRPC hled reques
1 387 2460 157 0.00% 0.00% 0.00% 0 Chunk Manager
154 0 1 0 0.00% 0.00% 0.00% 0 HRPC hl3mm reque
155 3700309 51886105 71 0.00% 0.00% 0.00% 0 hl3mm
80 155327501 751062275 206 0.00% 0.21% 0.17% 0 RedEarth Rx Mana
157 0 1 0 0.00% 0.00% 0.00% 0 HRPC hl3u reques
158 1229227 17239203 71 0.00% 0.00% 0.00% 0 HL3U PBR bkgrd p
159 34034 630302 53 0.00% 0.00% 0.00% 0 HL3U PBR n-h res
83 0 8 0 0.00% 0.00% 0.00% 0 Stack Mgr
161 0 1 0 0.00% 0.00% 0.00% 0 HRPC dtp request
162 0 1 0 0.00% 0.00% 0.00% 0 HRPC hmedianet p
163 0 11 0 0.00% 0.00% 0.00% 0 HRPC show_forwar
164 0 1 0 0.00% 0.00% 0.00% 0 HRPC snmp reques
165 0 1 0 0.00% 0.00% 0.00% 0 HULC SNMP Proces
92 0 3 0 0.00% 0.00% 0.00% 0 HRPC hdm non blo
167 78490967 30052040 2611 0.00% 0.07% 0.08% 0 HRPC qos request
168 0 1 0 0.00% 0.00% 0.00% 0 HRPC span reques
169 0 1 0 0.00% 0.00% 0.00% 0 HRPC system post
170 0 1 0 0.00% 0.00% 0.00% 0 Hulc Reload Mana
171 0 1 0 0.00% 0.00% 0.00% 0 HRPC hrcli-event
172 12737 420443 30 0.00% 0.00% 0.00% 0 HCMP sync proces
173 0 2 0 0.00% 0.00% 0.00% 0 image mgr
174 0 1 0 0.00% 0.00% 0.00% 0 HRFS OIR Proc
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
175 8 7 1142 0.00% 0.00% 0.00% 0 HL2MCM
176 0 7 0 0.00% 0.00% 0.00% 0 HL2MCM
177 0 6 0 0.00% 0.00% 0.00% 0 Hulc SM10G Infra
178 2572 17906 143 0.00% 0.00% 0.00% 0 AAA Server
179 6746 45836 147 0.00% 0.00% 0.00% 0 AAA ACCT Proc
180 0 1 0 0.00% 0.00% 0.00% 0 ACCT Periodic Pr
181 0 1 0 0.00% 0.00% 0.00% 0 AAA System Acct
182 0 1 0 0.00% 0.00% 0.00% 0 AUTH POLICY Fram
183 5219616 74193963 70 0.00% 0.02% 0.00% 0 Auth Manager
99 952291515 1687921542 564 0.00% 0.11% 0.10% 0 HLFM address lea
185 14653 252255 58 0.00% 0.00% 0.00% 0 IP Admin SM Proc
186 5988061 10227297 585 0.00% 0.00% 0.00% 0 CDP Protocol
187 913197 5033481 181 0.00% 0.00% 0.00% 0 MDFS LC Download
188 11787357 113799354 103 0.00% 0.00% 0.00% 0 CEF: IPv4 proces
189 0 2 0 0.00% 0.00% 0.00% 0 CMD HANDLER
190 0 2 0 0.00% 0.00% 0.00% 0 AAA Dictionary R
191 57512 630664 91 0.00% 0.00% 0.00% 0 DHCP Snooping
192 0 1 0 0.00% 0.00% 0.00% 0 DHCP Snooping db
193 0 2 0 0.00% 0.00% 0.00% 0 Dot1x Mgr Proces
194 0 1 0 0.00% 0.00% 0.00% 0 EAP Framework
195 0 1 0 0.00% 0.00% 0.00% 0 EAP Test
196 0 2 0 0.00% 0.00% 0.00% 0 CEF switching ba
197 1729 937 1845 0.00% 0.00% 0.00% 0 IP ARP Adjacency
107 0 2 0 0.00% 0.00% 0.00% 0 Vlan shutdown Pr
199 19483625 113981617 170 0.00% 0.05% 0.01% 0 IP Input
200 0 1 0 0.00% 0.00% 0.00% 0 ICMP event handl
201 7105446 143559551 49 0.00% 0.00% 0.00% 0 IP ARP Track
202 1513110 74897446 20 0.00% 0.00% 0.00% 0 fnvRootHubPortSt
203 0 1 0 0.00% 0.00% 0.00% 0 tHUB
204 44710765 718112122 62 0.00% 0.04% 0.00% 0 MDFS MFIB Proces
205 0 1 0 0.00% 0.00% 0.00% 0 ADJ NSF process
206 0 1 0 0.00% 0.00% 0.00% 0 IPv6 ping proces
207 18838 252301 74 0.00% 0.00% 0.00% 0 OBFL TEMP obfl0
208 8 10 800 0.00% 0.00% 0.00% 0 SMI Director DB
209 265847 2659206 99 0.00% 0.00% 0.00% 0 SMI CDP Update H
210 0 1 0 0.00% 0.00% 0.00% 0 SMI Backup Proce
211 0 2 0 0.00% 0.00% 0.00% 0 SMI IBC server p
212 9 1 9000 0.00% 0.00% 0.00% 0 SMI IBC client p
213 0 2 0 0.00% 0.00% 0.00% 0 SMI IBC Download
116 0 1 0 0.00% 0.00% 0.00% 0 LICENSE AGENT
215 48885 1261013 38 0.00% 0.00% 0.00% 0 Spanning Tree St
216 0 2 0 0.00% 0.00% 0.00% 0 CDP Forward Proc
217 652214 137618 4739 0.00% 0.00% 0.00% 0 802.1x switch
218 0 1 0 0.00% 0.00% 0.00% 0 802.1x Webauth F
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
219 2041420 7559201 270 0.00% 0.00% 0.00% 0 DTP Protocol
220 0 1 0 0.00% 0.00% 0.00% 0 axiom-module cra
221 0 1 0 0.00% 0.00% 0.00% 0 HRPC dai request
222 0 1 0 0.00% 0.00% 0.00% 0 HULC DAI Process
223 0 1 0 0.00% 0.00% 0.00% 0 HRPC power down
224 0 1 0 0.00% 0.00% 0.00% 0 HRPC ip device t
225 0 1 0 0.00% 0.00% 0.00% 0 HRPC ip source g
226 0 1 0 0.00% 0.00% 0.00% 0 HULC IP Source g
227 0 1 0 0.00% 0.00% 0.00% 0 HRPC sisf reques
228 0 2 0 0.00% 0.00% 0.00% 0 HULC SISF Proces
229 0 1 0 0.00% 0.00% 0.00% 0 HULC SISF Source
230 17228252 74446057 231 0.00% 0.02% 0.00% 0 PI MATM Aging Pr
140 0 1 0 0.00% 0.00% 0.00% 0 HRPC system mtu
232 115367 2522656 45 0.00% 0.00% 0.00% 0 Port-Security
233 0 2 0 0.00% 0.00% 0.00% 0 Switch Backup In
234 0 2 0 0.00% 0.00% 0.00% 0 IP Host Track Pr
235 0 1 0 0.00% 0.00% 0.00% 0 Link State Group
236 40581 1258468 32 0.00% 0.00% 0.00% 0 MMN bkgrd proces
237 250777 7550047 33 0.00% 0.00% 0.00% 0 Ethchnl
238 14938 46325 322 0.00% 0.00% 0.00% 0 VMATM Callback
239 0 1 0 0.00% 0.00% 0.00% 0 IPC UDP Input
240 268194 1509060 177 0.00% 0.00% 0.00% 0 CEF background p
241 0 1 0 0.00% 0.00% 0.00% 0 fib_fib_bfd_sb e
242 0 1 0 0.00% 0.00% 0.00% 0 IP IRDP
243 0 1 0 0.00% 0.00% 0.00% 0 CEF RF HULC Conv
244 0 1 0 0.00% 0.00% 0.00% 0 XDR background p
245 0 1 0 0.00% 0.00% 0.00% 0 XDR mcast
246 356069 7550463 47 0.00% 0.00% 0.00% 0 IP ACL XDR LC Ba
247 0 1 0 0.00% 0.00% 0.00% 0 XDR RP Ping Back
248 0 1 0 0.00% 0.00% 0.00% 0 XDR mcast rcv
249 67015 1258468 53 0.00% 0.00% 0.00% 0 XDR RP backgroun
250 0 1 0 0.00% 0.00% 0.00% 0 IPC LC Message H
251 0 1 0 0.00% 0.00% 0.00% 0 XDR RP Test Back
252 0 3 0 0.00% 0.00% 0.00% 0 ADJ background
253 0 3 0 0.00% 0.00% 0.00% 0 MDFS LC Process
254 0 1 0 0.00% 0.00% 0.00% 0 Critical Auth
255 0 2 0 0.00% 0.00% 0.00% 0 Routing Topology
256 0 2 0 0.00% 0.00% 0.00% 0 IP RIB Update
257 0 2 0 0.00% 0.00% 0.00% 0 ADJ resolve proc
258 0 1 0 0.00% 0.00% 0.00% 0 IPv6 RIB Event H
259 4296 24790 173 0.00% 0.00% 0.00% 0 TCP Timer
260 33800 77335 437 0.00% 0.00% 0.00% 0 TCP Protocols
261 0 1 0 0.00% 0.00% 0.00% 0 Socket Timers
262 16220 252944 64 0.00% 0.00% 0.00% 0 HTTP CORE
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
263 40085 945799 42 0.00% 0.00% 0.00% 0 Cluster L2
264 319736 7550049 42 0.00% 0.01% 0.00% 0 Cluster RARP
265 1017657 10700749 95 0.00% 0.00% 0.00% 0 Cluster Base
266 0 2 0 0.00% 0.00% 0.00% 0 Dot1x Supplicant
267 0 2 0 0.00% 0.00% 0.00% 0 Dot1x Supplicant
268 0 2 0 0.00% 0.00% 0.00% 0 Dot1x Supplicant
269 0 3 0 0.00% 0.00% 0.00% 0 Flow Exporter Ti
270 0 2 0 0.00% 0.00% 0.00% 0 Multicast Offloa
271 0 1 0 0.00% 0.00% 0.00% 0 RARP Input
272 0 2 0 0.00% 0.00% 0.00% 0 static
273 0 1 0 0.00% 0.00% 0.00% 0 MAB Framework
274 922628 7550056 122 0.00% 0.00% 0.00% 0 QoS stats proces
275 0 27 0 0.00% 0.00% 0.00% 0 SNMP Timers
276 0 1 0 0.00% 0.00% 0.00% 0 HRPC dhcp snoopi
277 8 4 2000 0.00% 0.00% 0.00% 0 HULC DHCP Snoopi
278 156102037 205417073 759 0.00% 0.18% 0.18% 0 Inline Power
184 0 1 0 0.00% 0.00% 0.00% 0 Auth-proxy AAA B
280 479556 3775225 127 0.00% 0.00% 0.00% 0 HULC stack power
281 0 1 0 0.00% 0.00% 0.00% 0 HRPC stack power
282 94096 92229 1020 0.00% 0.00% 0.00% 0 IGMPSN L2MCM
283 0 1 0 0.00% 0.00% 0.00% 0 IGMPSN MRD
284 181724 368847 492 0.00% 0.00% 0.00% 0 IGMPSN
285 0 1 0 0.00% 0.00% 0.00% 0 IGMPQR
286 0 3 0 0.00% 0.00% 0.00% 0 L2TRACE SERVER
287 85702 92243 929 0.00% 0.00% 0.00% 0 MLDSN L2MCM
288 0 1 0 0.00% 0.00% 0.00% 0 MRD
289 0 1 0 0.00% 0.00% 0.00% 0 MLD_SNOOP
290 0 1 0 0.00% 0.00% 0.00% 0 HRPC hl2mcm igmp
291 37852558 696528393 54 0.00% 0.03% 0.01% 0 MDFS RP process
292 0 2 0 0.00% 0.00% 0.00% 0 AAA Cached Serve
293 0 1 0 0.00% 0.00% 0.00% 0 HRPC hl2mcm mlds
294 0 3 0 0.00% 0.00% 0.00% 0 Collection proce
295 17090817 42036 406575 0.00% 0.00% 0.00% 0 NIST rng proc
296 0 2 0 0.00% 0.00% 0.00% 0 ENABLE AAA
297 0 3 0 0.00% 0.00% 0.00% 0 LDAP process
298 0 2 0 0.00% 0.00% 0.00% 0 LINE AAA
299 0 2 0 0.00% 0.00% 0.00% 0 LOCAL AAA
300 0 1 0 0.00% 0.00% 0.00% 0 EPM MAIN PROCESS
301 1565340 47841 32719 0.00% 0.00% 0.00% 0 OBFL UPTIME obfl
302 310442 325270 954 0.00% 0.03% 0.00% 0 TPLUS
303 12205364 3775125 3233 0.00% 0.01% 0.00% 0 HULC ISM FNF
304 620 20 31000 0.00% 0.00% 0.00% 0 OBFL MSG obfl0
305 0 2 0 0.00% 0.00% 0.00% 0 crypto engine pr
306 0 1 0 0.00% 0.00% 0.00% 0 encrypt proc
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
307 277 5 55400 0.00% 0.00% 0.00% 0 Crypto CA
308 0 1 0 0.00% 0.00% 0.00% 0 Crypto PKI-CRL
309 0 1 0 0.00% 0.00% 0.00% 0 HRPC x_setup req
310 0 1 0 0.00% 0.00% 0.00% 0 Licensing MIB pr
311 0 21 0 0.00% 0.00% 0.00% 0 VTP Trap Process
312 11017 207506 53 0.00% 0.00% 0.00% 0 ASP Process Crea
313 58238 220462 264 0.00% 0.00% 0.00% 0 AAA SEND STOP EV
314 0 1 0 0.00% 0.00% 0.00% 0 Test AAA Client
315 0 2 0 0.00% 0.00% 0.00% 0 DHCP Security He
316 0 2 0 0.00% 0.00% 0.00% 0 EEM ED Routing
317 9844 152548 64 0.00% 0.00% 0.00% 0 EEM ED Syslog
318 0 1 0 0.00% 0.00% 0.00% 0 Syslog Traps
319 56668 1258468 45 0.00% 0.00% 0.00% 0 vlmap logger
320 0 1 0 0.00% 0.00% 0.00% 0 HCD Process
321 0 1 0 0.00% 0.00% 0.00% 0 HRPC cable diagn
322 1032 56 18428 0.00% 0.00% 0.00% 0 OBFL ENV obfl0
323 0 2 0 0.00% 0.00% 0.00% 0 DiagCard1/-1
324 0 1 0 0.00% 0.00% 0.00% 0 Online Diag EEM
325 17175342 206360176 83 0.00% 0.01% 0.00% 0 PM Callback
326 41791184 1264739 33043 0.00% 0.02% 0.00% 0 OBFL VOLT obfl0
327 445043 6046307 73 0.00% 0.00% 0.00% 0 dhcp snooping sw
329 182 1075 169 0.00% 0.00% 0.00% 0 EEM Server
330 8 2 4000 0.00% 0.00% 0.00% 0 Call Home proces
331 0 2 0 0.00% 0.00% 0.00% 0 EEM Policy Direc
332 0 1 0 0.00% 0.00% 0.00% 0 CEF RP IPC Backg
333 0 2 0 0.00% 0.00% 0.00% 0 EEM ED CLI
334 0 3 0 0.00% 0.00% 0.00% 0 EEM ED Config
335 0 3 0 0.00% 0.00% 0.00% 0 EEM ED Counter
336 0 3 0 0.00% 0.00% 0.00% 0 EEM ED Env
337 0 3 0 0.00% 0.00% 0.00% 0 EM ED GOLD
338 0 3 0 0.00% 0.00% 0.00% 0 EEM ED Interface
339 9 3 3000 0.00% 0.00% 0.00% 0 EEM ED IOSWD
340 0 3 0 0.00% 0.00% 0.00% 0 EEM ED Ipsla
341 0 3 0 0.00% 0.00% 0.00% 0 EEM ED None
342 0 3 0 0.00% 0.00% 0.00% 0 EEM ED OIR
343 93152 1258479 74 0.00% 0.00% 0.00% 0 Call Home Timer
344 0 3 0 0.00% 0.00% 0.00% 0 EEM ED SNMP
345 0 3 0 0.00% 0.00% 0.00% 0 EEM ED SNMP Obje
346 0 2 0 0.00% 0.00% 0.00% 0 EEM ED SNMP Noti
347 0 3 0 0.00% 0.00% 0.00% 0 EEM ED Test
348 159251 2372188 67 0.00% 0.00% 0.00% 0 EEM ED Timer
349 207476 103631 2002 0.00% 0.00% 0.00% 0 Syslog
350 0 6 0 0.00% 0.00% 0.00% 0 RBM CORE
351 0 1 0 0.00% 0.00% 0.00% 0 HRPC eee request
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
352 0 2 0 0.00% 0.00% 0.00% 0 STP FAST TRANSIT
353 0 2 0 0.00% 0.00% 0.00% 0 CSRT RAPID TRANS
354 16 13 1230 0.00% 0.00% 0.00% 0 VLAN Manager
355 409989 5032006 81 0.00% 0.00% 0.00% 0 OBFL I/O Buffer
356 2588269 74446230 34 0.00% 0.00% 0.00% 0 DHCP Client
357 41 376 109 0.00% 0.00% 0.00% 0 SpanTree Flush
358 0 1 0 0.00% 0.00% 0.00% 0 hulc cfg mgr mas
359 33 470 70 0.00% 0.00% 0.00% 0 hulc running con
360 1330 20855 63 0.00% 0.00% 0.00% 0 SSH Event handle
361 2630049 7517831 349 0.00% 0.07% 0.01% 0 IP SNMP
362 840940 3698021 227 0.00% 0.06% 0.01% 0 PDU DISPATCHER
363 77430340 4884112 15853 0.00% 0.45% 0.24% 0 SNMP ENGINE
364 0 1 0 0.00% 0.00% 0.00% 0 IP SNMPV6
365 0 1 0 0.00% 0.00% 0.00% 0 SNMP ConfCopyPro
366 808824 175133 4618 0.00% 0.00% 0.00% 0 SNMP Traps
367 10508641 75370037 139 0.00% 0.00% 0.00% 0 NTP
368 2813 12232 229 0.00% 0.00% 0.00% 0 TACACS+

InayathUlla Sharieff · ‎01-27-2016

If you're looking at a switch and you see the HULC LED Process running high. The HULC LED process does some of the following tasks:

1. Check link status on every port
2. If the switch supports POE, it checks to see if there is a power device
3. Transceiver checks
4. Fan status updates
5. Set LED ports
6. Check on temperature status

Overall the HULC LED Process is a monitoring process running.

Ganesh Hariharan · ‎01-23-2016

Hi all,

My switch is a hig processor CPU, could you please advice me what to do.

Switch     Ports       Model                        SW Version                            SW Image
------ -      ----           -----                           ----------                                    ----------
* 1           54          WS-C3560X-48P       15.0(2)SE4                              C3560E-UNIVERSALK9NPE-M
CPU utilization for five seconds: 99%/28%; one minute: 89%; five minutes: 90%PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
153 2557429536 1689359934 1513 34.30% 36.49% 36.66% 0 Hulc LED Process
79 1502873480 895466063 1678 5.93% 4.98% 5.18% 0 RedEarth Tx Mana
78 2335980822 1535804589 1521 4.96% 3.16% 3.29% 0 RedEarth I2C dri
363 76824581 4791953 16032 2.72% 3.02% 2.94% 0 SNMP ENGINE
120 850452394 75595328 11250 1.44% 1.75% 1.78% 0 hpm counter proc
5 1297046015 57925014 22391 11.54% 1.92% 1.58% 0 Check heapsthanks

Hello , Based on the output provided it seems to HULC process is playing major role in High CPU issue.

Check out the Bug details CSCtc60988 for more details.

Hope it Helpss.

-GI

Leo Laohoo · ‎01-30-2016

Try using a different IOS, 12.2(55)SE10.

Joseph W. Doherty · ‎02-01-2016

https://supportforums.cisco.com/discussion/12753641/ask-expert-troubleshooting-high-cpu-catalyst-switches