- Cisco Community
- Technology and Support
- Networking
- Switching
- Cat 3750 Switch Crashed
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-10-2013 09:26 AM - edited 03-07-2019 02:51 PM
Hi Team
We have Stack of catalyst 3750 switches ( WS-C3750X-48T-S) running software
15.0(2)SE1.One of the switch in stack crashed and the users stopped getting connectivity.
Reloading this switch,retained the network connectivity.
The switch crashed with follwoing information
Aug 10 08:19:08.855: %SYS-2-NOBLOCK: may_suspend with blocking disabled. -Process= "hpm main process", ipl= 0, pid= 131
-Traceback= 4FC7E8z 26B9D54z 1C856E8z 2300D10z 2191200z 23E995Cz 242188Cz 2421994z 19C1794z 241786Cz 23E776Cz 21D9440z 21D9644z 22FDCB4z 1DEF740z 22F861Cz
Aug 10 08:19:08.855: %SYS-3-MGDTIMER: Timer has parent, timer link, timer = 491AE60. -Process= "hpm main process", ipl= 6, pid= 131
-Traceback= 4FC7E8z 1C3B5C0z 1C83328z 21D715Cz 26BF144z 26B9898z
Apr 4 08:19:14.845: %SYS-3-LOGGER_FLUSHED: System was paused for 00:00:00 to ensure console debugging output.
i have seen few logs causing switch crashed related to SYS MGDTIMER but those are not matching the release i am running.Is thhere any knon bug related to this in the current release ( i couldnt found the same in release notes as well in bug toolkit related to this symptoms).
Please suggest.
Thanks
Sameer
Solved! Go to Solution.
- Labels:
-
Other Switching
Accepted Solutions
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-10-2013 09:12 PM
Hi Sameer,
I have verified throughly and I can confirm that the 15.0(2)SE1 version is also affected beacuse of the BUG CSCue83689.
And not neccessary that all the conditions mentioned in the BUG should be an exact match. If one of the conditions is matching then also it should be fine. The more important stuff is that when analyzing from the code perspective after decoding your crashinfo the BUG should be an exact match. And in this case it is an exact match.
You can be assured that the root cause for the switch crash is - Cisco BUG CSCue83689.
As I mentioned earlier this BUG is fixed from 15.0(2)SE3 version onwards.
***********Plz do rate this post if you found it helpful*************************
Thanks & Regards,
Vignesh R P
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-10-2013 09:44 AM
Hi,
Is the CPU also running high?
Is the switch running SSH?
What is the output of
sh process cpu sor
HTH
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-10-2013 09:53 AM
Hi
Everything is working normal currently after the switch reload.During the issue as specific users were not getting connectivity,concentrated on the switch and after initial troubleshooting reloaded the switch.Attching crashinfo file for reference (please ignore date mismatch)...
REgards
Sameer
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-10-2013 09:55 AM
Hi
Yes the switch is running SSH..(Misse in last post)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-10-2013 10:18 AM
Hi reza
Looks like its matching the Bug CSCue83689 though its observed in 15.0.2(SE2) though not observed in 15.0.2 SE1 which is i am riunning currently..(we are running dot1x authentication )
Please suggest
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-10-2013 10:30 AM
Hi Sameer,
I would require the output of show version from the device to have the crashinfo file decoded and further assist you in understanding the root cause of the crash. Kindly do the needful at the earliest.
Thanks & Regards,
Vignesh R P
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-10-2013 10:35 AM
Hi Vignesh
i am putting down the show version before we upgraded to 15.0.2SE1 which is current release..( i do not have device access currently)
------------------ show version ------------------
Cisco IOS Software, C3750E Software (C3750E-UNIVERSALK9-M), Version 15.0(1)SE3, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2012 by Cisco Systems, Inc.
Compiled Wed 30-May-12 13:41 by prod_rel_team
ROM: Bootstrap program is C3750E boot loader
BOOTLDR: C3750E Boot Loader (C3750X-HBOOT-M) Version 12.2(53r)SE2, RELEASE SOFTWARE (fc1)
Ozone-5A-115.97 uptime is 1 week, 3 days, 21 hours, 26 minutes
System returned to ROM by power-on
System image file is "flash:/c3750e-universalk9-mz.150-1.SE3/c3750e-universalk9-mz.150-1.SE3.bin"
This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.
A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html
If you require further assistance please contact us by sending email to
License Level: ipbase
License Type: Permanent
Next reload license Level: ipbase
cisco WS-C3750X-48 (PowerPC405) processor (revision A0) with 262144K bytes of memory.
Processor board ID FDO1627V09D
Last reset from power-on
3 Virtual Ethernet interfaces
1 FastEthernet interface
312 Gigabit Ethernet interfaces
12 Ten Gigabit Ethernet interfaces
The password-recovery mechanism is enabled.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-10-2013 11:36 AM
Any Luck Vignesh...
please let me know if you get any more details on this crash..
Thanks in advance..
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-10-2013 05:53 PM
Hi Sameer,
The noblock message means that macsecdev_set_port_option called process_may_suspend but blocking was disabled. This is what caused the crash. And yes your finding is 100% right. The crash was caused due to the Cisco BUG CSCue83689.
This BUG is fixed from 15.0(2)SE3 version onwards.
***********Plz do rate this post if you found it helpful*************************
Thanks & Regards,
Vignesh R P
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-10-2013 07:44 PM
Hi Vignesh
Regarding the bug you mentioned,the conditions mentioed in the bug are not matching (Thats why i mentioned n my previous mail as looks like)
- its seen in 15.0(1)SE1 and 15.0(2)SE2 and we are running 15.0(2)SE1
- The other condition says when vlan is assigned via ACS and switch is configured for 802.1x authentication...Though we do are not using ACS,we have dot1x configured and radius server is doing the aunthentication.
can it be the case that the bug went unidentified in release 15.0.(2) SE1 ?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-10-2013 09:12 PM
Hi Sameer,
I have verified throughly and I can confirm that the 15.0(2)SE1 version is also affected beacuse of the BUG CSCue83689.
And not neccessary that all the conditions mentioned in the BUG should be an exact match. If one of the conditions is matching then also it should be fine. The more important stuff is that when analyzing from the code perspective after decoding your crashinfo the BUG should be an exact match. And in this case it is an exact match.
You can be assured that the root cause for the switch crash is - Cisco BUG CSCue83689.
As I mentioned earlier this BUG is fixed from 15.0(2)SE3 version onwards.
***********Plz do rate this post if you found it helpful*************************
Thanks & Regards,
Vignesh R P
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-10-2013 09:45 PM
Hi Vignesh
Thanks for your detailed response.
so as i mentioned in my prevoius post,this bug must have gone unidentified in 15.0.2SE1 and probably will reflect in coming days.
Even the bug condition says AT LEASE SEEN IN so we are on right track and probably cisco will add 15.0.2 se1 also as affected release in coming days.
Regards
Sameer
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-10-2013 01:36 PM
Hi Sam,
Is the CPU running high?
can you post
sh procce cpu sort
Reza
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-10-2013 07:45 PM
Hi Reza
The cpu is running normal ..
i can post the output of show proc cpu sort by tomorrow as switch access is not available currently .
Thanks
Sameer
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-12-2013 12:08 AM
Hi reza
Please find required details
------------------ show process cpu ------------------
CPU utilization for five seconds: 59%/2%; one minute: 61%; five minutes: 43%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
1 748 11647 64 0.00% 0.00% 0.00% 0 Chunk Manager
2 1447 17500 82 0.00% 0.00% 0.00% 0 Load Meter
3 67 34 1970 0.00% 0.00% 0.00% 0 hulc_entropy_thr
4 21379 34958 611 0.16% 0.02% 0.00% 0 Hulc Axiom Therm
5 1703952 74294 22935 0.00% 2.39% 1.90% 0 Check heaps
6 51 64 796 0.00% 0.00% 0.00% 0 Pool Manager
7 0 1 0 0.00% 0.00% 0.00% 0 DiscardQ Backgro
8 0 2 0 0.00% 0.00% 0.00% 0 Timers
9 109 12708 8 0.00% 0.00% 0.00% 0 WATCH_AFS
10 8 5 1600 0.00% 0.00% 0.00% 0 License Client N
11 214729 1456 147478 0.00% 0.21% 0.18% 0 Licensing Auto U
12 185 12 15416 0.00% 0.00% 0.00% 0 Image License br
13 92888 135475 685 0.00% 0.10% 0.14% 0 ARP Input
14 1182 91089 12 0.00% 0.00% 0.00% 0 ARP Background
15 0 1 0 0.00% 0.00% 0.00% 0 CEF MIB API
16 0 1 0 0.00% 0.00% 0.00% 0 AAA_SERVER_DEADT
17 0 1 0 0.00% 0.00% 0.00% 0 Policy Manager
18 278 66 4212 0.00% 0.00% 0.00% 0 Entity MIB API
19 0 1 0 0.00% 0.00% 0.00% 0 IFS Agent Manage
20 661 17450 37 0.00% 0.00% 0.00% 0 IPC Event Notifi
21 633 84830 7 0.00% 0.00% 0.00% 0 IPC Mcast Pendin
22 36 1459 24 0.00% 0.00% 0.00% 0 IPC Dynamic Cach
23 0 1 0 0.00% 0.00% 0.00% 0 IPC Session Serv
24 0 1 0 0.00% 0.00% 0.00% 0 IPC Zone Manager
25 532 84830 6 0.00% 0.00% 0.00% 0 IPC Periodic Tim
26 539 84829 6 0.00% 0.00% 0.00% 0 IPC Deferred Por
27 0 1 0 0.00% 0.00% 0.00% 0 IPC Process leve
28 4999 54073 92 0.00% 0.00% 0.00% 0 IPC Seat Manager
29 42 5009 8 0.00% 0.00% 0.00% 0 IPC Check Queue
30 24 125 192 0.00% 0.00% 0.00% 0 IPC Seat RX Cont
31 0 1 0 0.00% 0.00% 0.00% 0 IPC Seat TX Cont
32 42 8772 4 0.00% 0.00% 0.00% 0 IPC Keep Alive M
33 800 17497 45 0.00% 0.00% 0.00% 0 IPC Loadometer
34 284 9 31555 0.00% 0.00% 0.00% 0 PrstVbl
35 0 1 0 0.00% 0.00% 0.00% 0 Crash writer
36 0 1 0 0.00% 0.00% 0.00% 0 Exception contro
37 8 11 727 0.00% 0.00% 0.00% 0 License IPC stat
38 0 1 0 0.00% 0.00% 0.00% 0 License IPC serv
39 347 87491 3 0.00% 0.00% 0.00% 0 GraphIt
40 2297 350 6562 0.00% 0.00% 0.00% 0 crypto sw pk pro
41 0 2 0 0.00% 0.00% 0.00% 0 SMART
42 0 2 0 0.00% 0.00% 0.00% 0 XML Proxy Client
43 0 1 0 0.00% 0.00% 0.00% 0 ARP Snoop
44 1047 86765 12 0.00% 0.00% 0.00% 0 Dynamic ARP Insp
45 0 1 0 0.00% 0.00% 0.00% 0 Critical Bkgnd
46 151005 29436 5129 0.00% 0.18% 0.17% 0 Net Background
47 0 1 0 0.00% 0.00% 0.00% 0 IDB Work
48 555 3229 171 0.00% 0.00% 0.00% 0 Logger
49 1190 86565 13 0.00% 0.00% 0.00% 0 TTY Background
50 0 4 0 0.00% 0.00% 0.00% 0 CTS CORE
51 0 28 0 0.00% 0.00% 0.00% 0 SXP CORE
52 51 30 1700 0.00% 0.00% 0.00% 0 IF-MGR control p
53 9 470 19 0.00% 0.00% 0.00% 0 IF-MGR event pro
54 0 1 0 0.00% 0.00% 0.00% 0 Inode Table Dest
55 8 8 1000 0.00% 0.00% 0.00% 0 IP Admission HA
56 229 5902 38 0.00% 0.00% 0.00% 0 Net Input
57 92440 17505 5280 0.16% 0.10% 0.10% 0 Compute load avg
58 52666 1511 34855 0.00% 0.03% 0.04% 0 Per-minute Jobs
59 1582374 87829 18016 1.28% 1.34% 1.28% 0 Per-Second Jobs
60 0 1 0 0.00% 0.00% 0.00% 0 AggMgr Process
61 0 135 0 0.00% 0.00% 0.00% 0 Transport Port A
62 9730 25456 382 0.00% 0.00% 0.00% 0 HC Counter Timer
63 15555 30091 516 0.00% 0.02% 0.00% 0 SFF8472
64 0 179 0 0.00% 0.00% 0.00% 0 EEM ED Identity
65 0 357 0 0.00% 0.00% 0.00% 0 EEM ED MAT
66 691 5245 131 0.00% 0.00% 0.00% 0 EEM ED ND
67 34 80 425 0.00% 0.00% 0.00% 0 USB Startup
68 5187 1660729 3 0.00% 0.01% 0.00% 0 DownWhenLooped
69 0 1 0 0.00% 0.00% 0.00% 0 HULC PBR Tcam Me
70 0 1 0 0.00% 0.00% 0.00% 0 HULC ACL Tcam Me
71 0 1 0 0.00% 0.00% 0.00% 0 Hulc Warm Upgrad
72 0 1 0 0.00% 0.00% 0.00% 0 HRPC hdwl reques
73 0 1 0 0.00% 0.00% 0.00% 0 Hulc EEM Process
74 0 1 0 0.00% 0.00% 0.00% 0 HRPC lpip reques
75 0 2 0 0.00% 0.00% 0.00% 0 HLPIP Sync Proce
76 0 1 0 0.00% 0.00% 0.00% 0 HRPC Multi-FS Sy
77 17 18 944 0.00% 0.00% 0.00% 0 HULC multifs pro
78 0 1 0 0.00% 0.00% 0.00% 0 HRPC hnetwpol re
79 0 1 0 0.00% 0.00% 0.00% 0 HRPC EnergyWise
80 0 1 0 0.00% 0.00% 0.00% 0 HRPC actual powe
81 3971 349814 11 0.00% 0.01% 0.00% 0 HUSB Console
82 0 1 0 0.00% 0.00% 0.00% 0 HRPC ipadm reque
83 16 2 8000 0.00% 0.00% 0.00% 0 SM RBCP Monitor
84 106 2917 36 0.00% 0.00% 0.00% 0 PSP Timer
85 0 1 0 0.00% 0.00% 0.00% 0 HULC QM Tcam Mem
86 24043 141 170517 0.00% 0.00% 0.00% 0 X2 HAMM detect p
87 0 1 0 0.00% 0.00% 0.00% 0 Madmax Process
88 0 1 0 0.00% 0.00% 0.00% 0 HRPC asic-stats
89 807025 679970 1186 1.12% 1.04% 0.82% 0 RedEarth I2C dri
90 2479857 439297 5645 2.24% 2.45% 2.47% 0 RedEarth Tx Mana
91 83156 339982 244 0.00% 0.06% 0.07% 0 RedEarth Rx Mana
92 220 17501 12 0.00% 0.00% 0.00% 0 HULC Thermal Pro
93 0 3 0 0.00% 0.00% 0.00% 0 HRPC hsm request
94 59 210 280 0.00% 0.00% 0.00% 0 Stack Mgr
95 2671 1185 2254 0.00% 0.00% 0.00% 0 Stack Mgr Notifi
96 269 31987 8 0.00% 0.00% 0.00% 0 Adjust Regions
97 9050 43633 207 0.00% 0.00% 0.00% 0 hrpc -> response
98 7757 202700 38 0.00% 0.00% 0.00% 0 hrpc -> request
99 99518 312677 318 0.32% 0.33% 0.16% 0 hrpc <- response
100 0 11 0 0.00% 0.00% 0.00% 0 HRPC hcomp reque
101 4109 437174 9 0.00% 0.05% 0.01% 0 yeti2_emac_proce
102 0 43 0 0.00% 0.00% 0.00% 0 HULC Device Mana
103 2620 7475 350 0.00% 0.00% 0.00% 0 HRPC hdm non blo
104 0 12 0 0.00% 0.00% 0.00% 0 HRPC hdm blockin
105 1215 17413 69 0.00% 0.00% 0.00% 0 HIPC bkgrd proce
106 0 1 0 0.00% 0.00% 0.00% 0 RTTYS Process
107 95 153 620 0.00% 0.00% 0.00% 0 Hulc Port-Securi
108 126 1136 110 0.00% 0.00% 0.00% 0 HRPC hpsecure re
109 0 13 0 0.00% 0.00% 0.00% 0 HRPC hfbm reques
110 167 28 5964 0.00% 0.00% 0.00% 0 hulc cfg mgr mas
111 6072 18021 336 0.00% 0.00% 0.00% 0 HRPC hlfm reques
112 79109 2161380 36 0.16% 0.16% 0.11% 0 HLFM address lea
113 3551 86570 41 0.00% 0.00% 0.00% 0 HLFM aging proce
114 25879 2182337 11 0.00% 0.02% 0.00% 0 HLFM address ret
115 0 1 0 0.00% 0.00% 0.00% 0 HRPC hrcmd reque
116 24 120 200 0.00% 0.00% 0.00% 0 HRPC emac reques
117 0 28 0 0.00% 0.00% 0.00% 0 HRPC hulc misc r
118 489 29097 16 0.00% 0.00% 0.00% 0 HVLAN main bkgrd
119 8 2 4000 0.00% 0.00% 0.00% 0 HVLAN Mapped Vla
120 8 2 4000 0.00% 0.00% 0.00% 0 Vlan shutdown Pr
121 41 262 156 0.00% 0.00% 0.00% 0 HRPC vlan reques
122 0 1 0 0.00% 0.00% 0.00% 0 HULC VLAN REF Ba
123 67 1457 45 0.00% 0.00% 0.00% 0 hulc_xps_process
124 0 11 0 0.00% 0.00% 0.00% 0 HRPC ilp request
125 6802 5819 1168 0.00% 0.00% 0.00% 0 Strider Tcam Mem
126 131 142 922 0.00% 0.00% 0.00% 0 HULC PM Vector P
127 0 1 0 0.00% 0.00% 0.00% 0 HPM Msg Retry Pr
128 605 5817 104 0.00% 0.00% 0.00% 0 MDFS LC Download
129 83360 1364667 61 0.00% 0.08% 0.08% 0 hpm main process
130 307 602 509 0.00% 0.00% 0.00% 0 HPM Stack Sync P
131 2923 6771 431 0.00% 0.00% 0.00% 0 HRPC pm request
132 0 9 0 0.00% 0.00% 0.00% 0 HPM if_num mappi
133 1071101 87493 12242 1.28% 1.30% 1.29% 0 hpm counter proc
134 162962 349241 466 0.00% 0.15% 0.18% 0 HRPC pm-counters
135 0 1 0 0.00% 0.00% 0.00% 0 hpm vp events ca
136 1437 86716 16 0.00% 0.01% 0.00% 0 Hulc CTS Process
137 0 1 0 0.00% 0.00% 0.00% 0 HRPC cts request
138 0 1 0 0.00% 0.00% 0.00% 0 HULC Msg Retry P
139 0 1 0 0.00% 0.00% 0.00% 0 HRPC hcmp reques
140 159 43 3697 0.00% 0.00% 0.00% 0 HCEF ADJ Refresh
141 0 1 0 0.00% 0.00% 0.00% 0 hl3md_rpfq_thrl_
142 339 43580 7 0.00% 0.00% 0.00% 0 hl3mm_rp
143 0 1 0 0.00% 0.00% 0.00% 0 HACL Queue Proce
144 0 1 0 0.00% 0.00% 0.00% 0 HRPC acl request
145 16 54 296 0.00% 0.00% 0.00% 0 HACL Acl Manager
146 0 1 0 0.00% 0.00% 0.00% 0 HRPC aim request
147 33 37 891 0.00% 0.00% 0.00% 0 HRPC axiom-mgmt
148 313 43574 7 0.00% 0.00% 0.00% 0 Hulc Axiom Mgmt
149 0 1 0 0.00% 0.00% 0.00% 0 HRPC backup inte
150 134 296 452 0.00% 0.00% 0.00% 0 OBFL TEMP obfl0
151 0 1 0 0.00% 0.00% 0.00% 0 HRPC cdp request
152 0 11 0 0.00% 0.00% 0.00% 0 HULC CISP Proces
153 290 2103 137 0.00% 0.00% 0.00% 0 HRPC dot1x reque
154 182 151 1205 0.00% 0.00% 0.00% 0 Dot1X Msg Retry
155 3656 6686 546 0.00% 0.00% 0.00% 0 HULC DOT1X Proce
156 0 1 0 0.00% 0.00% 0.00% 0 HRPC foxhound re
157 0 2 0 0.00% 0.00% 0.00% 0 Hulc Foxhound Pr
158 0 1 0 0.00% 0.00% 0.00% 0 HRPC lldp reques
159 2836 26 109076 0.00% 0.00% 0.00% 0 MACsec Process
160 0 1 0 0.00% 0.00% 0.00% 0 HRPC macsec requ
161 8 11 727 0.00% 0.00% 0.00% 0 HRPC system mtu
162 0 21 0 0.00% 0.00% 0.00% 0 HRPC rep request
163 8 9 888 0.00% 0.00% 0.00% 0 REP Helper Proc
164 0 1 0 0.00% 0.00% 0.00% 0 HRPC sdm request
165 0 1 0 0.00% 0.00% 0.00% 0 SMI MSG Retry Pr
166 0 1 0 0.00% 0.00% 0.00% 0 HRPC Smart Insta
167 108979 429007 254 0.16% 0.25% 0.16% 0 Hulc Storm Contr
168 0 2 0 0.00% 0.00% 0.00% 0 HSTP Sync Proces
169 0 1 0 0.00% 0.00% 0.00% 0 HRPC stp_cli req
170 17 54 314 0.00% 0.00% 0.00% 0 HRPC stp_state_s
171 0 2 0 0.00% 0.00% 0.00% 0 S/W Bridge Proce
172 0 54 0 0.00% 0.00% 0.00% 0 HRPC hudld reque
173 0 1 0 0.00% 0.00% 0.00% 0 HRPC vqpc reques
174 0 1 0 0.00% 0.00% 0.00% 0 HRPC iec_load_ba
175 1042 86589 12 0.00% 0.00% 0.00% 0 IEC Load balance
176 0 1 0 0.00% 0.00% 0.00% 0 HRPC l2pt qnq rp
177 0 74 0 0.00% 0.00% 0.00% 0 HRPC hled reques
178 18776722 2175625 8630 20.38% 19.84% 19.74% 0 Hulc LED Process
179 0 1 0 0.00% 0.00% 0.00% 0 HRPC hl3mm reque
180 1204 60584 19 0.00% 0.00% 0.00% 0 hl3mm
181 4743 64349 73 0.00% 0.00% 0.00% 0 HL3U bkgrd proce
182 10037 116433 86 0.00% 0.00% 0.00% 0 HRPC hl3u reques
183 1287 19962 64 0.00% 0.00% 0.00% 0 HIPV6 bkgrd proc
184 10197 116436 87 0.00% 0.00% 0.00% 0 HRPC IPv6 Unicas
185 218 19971 10 0.00% 0.00% 0.00% 0 HL3U PBR bkgrd p
186 9 730 12 0.00% 0.00% 0.00% 0 HL3U PBR n-h res
187 0 1 0 0.00% 0.00% 0.00% 0 HRPC obfl reques
188 19870 1492 13317 23.76% 21.80% 5.72% 1 SSH Process
189 0 1 0 0.00% 0.00% 0.00% 0 HRPC dtp request
190 0 1 0 0.00% 0.00% 0.00% 0 HRPC hmedianet p
191 0 1 0 0.00% 0.00% 0.00% 0 HRPC show_forwar
192 9 39 230 0.00% 0.00% 0.00% 0 HRPC snmp reques
193 115 20 5750 0.00% 0.00% 0.00% 0 HULC SNMP Proces
194 339352 17485 19408 0.32% 0.32% 0.31% 0 HQM Stack Proces
195 448315 150344 2981 0.48% 0.48% 0.52% 0 HRPC qos request
196 0 1 0 0.00% 0.00% 0.00% 0 HRPC span reques
197 24 61 393 0.00% 0.00% 0.00% 0 HRPC system post
198 26 41 634 0.00% 0.00% 0.00% 0 Hulc Reload Mana
199 596 1254 475 0.00% 0.00% 0.00% 0 HRPC hrcli-event
201 0 2 0 0.00% 0.00% 0.00% 0 image mgr
202 0 1 0 0.00% 0.00% 0.00% 0 HRFS OIR Proc
203 182 578 314 0.00% 0.00% 0.00% 0 HL2MCM
204 60 24 2500 0.00% 0.00% 0.00% 0 HL2MCM
205 4726 29048 162 0.00% 0.00% 0.00% 0 Hulc SM10G Infra
206 2319 2962 782 0.00% 0.00% 0.00% 0 AAA Server
207 34 68 500 0.00% 0.00% 0.00% 0 AAA ACCT Proc
208 0 1 0 0.00% 0.00% 0.00% 0 ACCT Periodic Pr
209 0 3 0 0.00% 0.00% 0.00% 0 AAA System Acct
210 0 1 0 0.00% 0.00% 0.00% 0 AUTH POLICY Fram
211 18493 124726 148 0.00% 0.05% 0.05% 0 Auth Manager
212 0 1 0 0.00% 0.00% 0.00% 0 Auth-proxy AAA B
213 0 292 0 0.00% 0.00% 0.00% 0 IP Admin SM Proc
214 25813 13901 1856 0.00% 0.02% 0.00% 0 CDP Protocol
215 0 1 0 0.00% 0.00% 0.00% 0 DiagCard3/-1
216 11854 134346 88 0.00% 0.00% 0.00% 0 CEF: IPv4 proces
217 0 2 0 0.00% 0.00% 0.00% 0 CMD HANDLER
218 0 2 0 0.00% 0.00% 0.00% 0 AAA Dictionary R
219 9 730 12 0.00% 0.00% 0.00% 0 DHCP Snooping
220 0 1 0 0.00% 0.00% 0.00% 0 DHCP Snooping db
221 302 4179 72 0.00% 0.00% 0.00% 0 Dot1x Mgr Proces
222 5334 10145 525 0.00% 0.00% 0.00% 0 EAP Framework
223 0 1 0 0.00% 0.00% 0.00% 0 EAP Test
224 0 2 0 0.00% 0.00% 0.00% 0 EAPoUDP Process
225 0 2 0 0.00% 0.00% 0.00% 0 CEF switching ba
226 0 9 0 0.00% 0.00% 0.00% 0 IP ARP Adjacency
227 26584 2161677 12 0.00% 0.04% 0.00% 0 IP ARP Retry Age
228 91461 431208 212 0.00% 0.37% 0.12% 0 IP Input
229 0 1 0 0.00% 0.00% 0.00% 0 ICMP event handl
230 1865 167924 11 0.00% 0.00% 0.00% 0 IP ARP Track
231 479 86839 5 0.00% 0.00% 0.00% 0 fnvRootHubPortSt
232 0 1 0 0.00% 0.00% 0.00% 0 tHUB
233 4569 857954 5 0.00% 0.00% 0.00% 0 MDFS MFIB Proces
234 0 1 0 0.00% 0.00% 0.00% 0 ADJ NSF process
235 0 1 0 0.00% 0.00% 0.00% 0 IPv6 ping proces
236 0 2 0 0.00% 0.00% 0.00% 0 KRB5 AAA
237 0 2 0 0.00% 0.00% 0.00% 0 MKA
238 0 1 0 0.00% 0.00% 0.00% 0 DiagCard2/-1
239 0 2 0 0.00% 0.00% 0.00% 0 REP Topology cha
240 0 8 0 0.00% 0.00% 0.00% 0 SMI Director DB
241 408 3920 104 0.00% 0.00% 0.00% 0 SMI CDP Update H
242 0 1 0 0.00% 0.00% 0.00% 0 SMI Backup Proce
243 0 2 0 0.00% 0.00% 0.00% 0 SMI IBC server p
244 0 1 0 0.00% 0.00% 0.00% 0 SMI IBC client p
245 0 2 0 0.00% 0.00% 0.00% 0 SMI IBC Download
246 187580 841581 222 0.00% 0.29% 0.20% 0 Spanning Tree
247 8 1471 5 0.00% 0.00% 0.00% 0 Spanning Tree St
248 0 2 0 0.00% 0.00% 0.00% 0 CDP Forward Proc
249 2397 516 4645 0.00% 0.00% 0.00% 0 802.1x switch
250 0 1 0 0.00% 0.00% 0.00% 0 802.1x Webauth F
251 1028 7174 143 0.00% 0.00% 0.00% 0 DTP Protocol
252 0 11 0 0.00% 0.00% 0.00% 0 HRPC IPv6 Host r
253 0 2 0 0.00% 0.00% 0.00% 0 IPv6 Platform Ho
254 0 1 0 0.00% 0.00% 0.00% 0 axiom-module cra
255 0 1 0 0.00% 0.00% 0.00% 0 HRPC dai request
256 228139 562989 405 0.00% 0.20% 0.18% 0 HULC DAI Process
257 0 1 0 0.00% 0.00% 0.00% 0 HRPC power down
258 0 1 0 0.00% 0.00% 0.00% 0 HRPC ip device t
259 0 1 0 0.00% 0.00% 0.00% 0 HRPC ip source g
260 0 1 0 0.00% 0.00% 0.00% 0 HULC IP Source g
261 0 1 0 0.00% 0.00% 0.00% 0 HRPC sisf reques
262 8 7 1142 0.00% 0.00% 0.00% 0 HULC SISF Proces
263 0 1 0 0.00% 0.00% 0.00% 0 HULC SISF Source
264 22821 86568 263 0.00% 0.02% 0.00% 0 PI MATM Aging Pr
265 31336 873528 35 0.00% 0.09% 0.02% 0 UDLD
266 69 2918 23 0.00% 0.00% 0.00% 0 Port-Security
267 0 2 0 0.00% 0.00% 0.00% 0 Switch Backup In
268 0 2 0 0.00% 0.00% 0.00% 0 IP Host Track Pr
269 0 1 0 0.00% 0.00% 0.00% 0 Link State Group
270 58 1456 39 0.00% 0.00% 0.00% 0 MMN bkgrd proces
271 140 8729 16 0.00% 0.00% 0.00% 0 Ethchnl
272 814 251 3243 0.00% 0.00% 0.00% 0 VMATM Callback
273 6667 63681 104 0.00% 0.00% 0.00% 0 IPC UDP Input
274 377 1759 214 0.00% 0.00% 0.00% 0 CEF background p
275 0 1 0 0.00% 0.00% 0.00% 0 fib_fib_bfd_sb e
276 0 1 0 0.00% 0.00% 0.00% 0 IP IRDP
277 9 1 9000 0.00% 0.00% 0.00% 0 CEF RF HULC Conv
278 0 6 0 0.00% 0.00% 0.00% 0 XDR background p
279 1560 2802 556 0.00% 0.00% 0.00% 0 XDR mcast
280 108 8728 12 0.00% 0.00% 0.00% 0 IP ACL XDR LC Ba
281 0 9 0 0.00% 0.00% 0.00% 0 XDR RP Ping Back
282 3881 5666 684 0.00% 0.01% 0.00% 0 XDR mcast rcv
283 66 1456 45 0.00% 0.00% 0.00% 0 XDR RP backgroun
284 59 16 3687 0.00% 0.00% 0.00% 0 IPC LC Message H
285 0 1 0 0.00% 0.00% 0.00% 0 XDR RP Test Back
286 0 4 0 0.00% 0.00% 0.00% 0 ADJ background
287 0 17 0 0.00% 0.00% 0.00% 0 MDFS LC Process
288 0 1 0 0.00% 0.00% 0.00% 0 Critical Auth
289 0 2 0 0.00% 0.00% 0.00% 0 Routing Topology
290 0 2 0 0.00% 0.00% 0.00% 0 IP RIB Update
291 0 2 0 0.00% 0.00% 0.00% 0 ADJ resolve proc
292 0 1 0 0.00% 0.00% 0.00% 0 IPv6 RIB Event H
293 285 5551 51 0.00% 0.00% 0.00% 0 TCP Timer
294 34 113 300 0.00% 0.00% 0.00% 0 TCP Protocols
295 25 86668 0 0.00% 0.00% 0.00% 0 Socket Timers
296 8 295 27 0.00% 0.00% 0.00% 0 HTTP CORE
297 0 1096 0 0.00% 0.00% 0.00% 0 Cluster L2
298 256 8730 29 0.00% 0.00% 0.00% 0 Cluster RARP
299 424 11909 35 0.00% 0.00% 0.00% 0 Cluster Base
300 5761 173363 33 0.16% 0.03% 0.00% 0 DHCPD Receive
301 0 2 0 0.00% 0.00% 0.00% 0 Dot1x Supplicant
302 0 2 0 0.00% 0.00% 0.00% 0 Dot1x Supplicant
303 0 2 0 0.00% 0.00% 0.00% 0 Dot1x Supplicant
304 0 3 0 0.00% 0.00% 0.00% 0 Flow Exporter Ti
305 0 2 0 0.00% 0.00% 0.00% 0 Multicast Offloa
306 0 1 0 0.00% 0.00% 0.00% 0 RARP Input
307 0 2 0 0.00% 0.00% 0.00% 0 static
308 0 1 0 0.00% 0.00% 0.00% 0 MAB Framework
309 904 8733 103 0.00% 0.00% 0.00% 0 QoS stats proces
310 8 754 10 0.00% 0.00% 0.00% 0 DHCPD Timer
311 0 2 0 0.00% 0.00% 0.00% 0 REP LSL Proc
312 0 2 0 0.00% 0.00% 0.00% 0 REP BPA/EPA Proc
313 0 5 0 0.00% 0.00% 0.00% 0 SNMP Timers
314 0 1 0 0.00% 0.00% 0.00% 0 HRPC dhcp snoopi
315 0 9 0 0.00% 0.00% 0.00% 0 HULC DHCP Snoopi
316 1416 10614 133 0.00% 0.00% 0.00% 0 HULC stack power
317 2299 34413 66 0.00% 0.00% 0.00% 0 HRPC stack power
318 150 164 914 0.00% 0.00% 0.00% 0 IGMPSN L2MCM
319 0 1 0 0.00% 0.00% 0.00% 0 IGMPSN MRD
320 82 130 630 0.00% 0.00% 0.00% 0 IGMPSN
321 0 1 0 0.00% 0.00% 0.00% 0 IGMPQR
322 8 3 2666 0.00% 0.00% 0.00% 0 L2TRACE SERVER
323 140 161 869 0.00% 0.00% 0.00% 0 MLDSN L2MCM
324 0 1 0 0.00% 0.00% 0.00% 0 MRD
325 0 1 0 0.00% 0.00% 0.00% 0 MLD_SNOOP
326 36 363 99 0.00% 0.00% 0.00% 0 HRPC hl2mcm igmp
327 17738 865912 20 0.00% 0.03% 0.00% 0 MDFS RP process
328 0 1 0 0.00% 0.00% 0.00% 0 EM Background Pr
329 0 2 0 0.00% 0.00% 0.00% 0 AAA Cached Serve
330 26 355 73 0.00% 0.00% 0.00% 0 HRPC hl2mcm mlds
331 0 3 0 0.00% 0.00% 0.00% 0 Collection proce
332 21894 54 405444 0.00% 0.00% 0.00% 0 NIST rng proc
333 1979 167923 11 0.00% 0.01% 0.00% 0 Timer Library
334 0 2 0 0.00% 0.00% 0.00% 0 ENABLE AAA
335 0 3 0 0.00% 0.00% 0.00% 0 LDAP process
336 0 2 0 0.00% 0.00% 0.00% 0 Crypto Support
337 0 1 0 0.00% 0.00% 0.00% 0 IPSECv6 PS Proc
338 0 2 0 0.00% 0.00% 0.00% 0 LINE AAA
339 43 141 304 0.00% 0.00% 0.00% 0 LOCAL AAA
340 74 165 448 0.00% 0.00% 0.00% 0 EPM MAIN PROCESS
341 8 20 400 0.00% 0.00% 0.00% 0 OBFL Cfg Dispatc
342 0 2 0 0.00% 0.00% 0.00% 0 TPLUS
343 13115 4365 3004 0.00% 0.01% 0.00% 0 HULC ISM FNF
344 43315 2521 17181 0.00% 0.00% 0.00% 0 hulc running con
345 436 30 14533 0.00% 0.00% 0.00% 0 crypto engine pr
346 2545 87153 29 0.00% 0.00% 0.00% 0 Crypto IKEv2
347 0 1 0 0.00% 0.00% 0.00% 0 IKEv2 AAA handle
348 0 1 0 0.00% 0.00% 0.00% 0 encrypt proc
349 0 3 0 0.00% 0.00% 0.00% 0 CRYPTO MAP FREE
350 0 1 0 0.00% 0.00% 0.00% 0 Crypto INT
351 0 2 0 0.00% 0.00% 0.00% 0 Crypto IKE Dispa
352 8 3 2666 0.00% 0.00% 0.00% 0 Crypto IKMP
353 0 1 0 0.00% 0.00% 0.00% 0 IPSEC manual key
354 349 4366 79 0.00% 0.00% 0.00% 0 IPSEC key engine
355 0 1 0 0.00% 0.00% 0.00% 0 Crypto ACL
356 0 1 0 0.00% 0.00% 0.00% 0 Crypto PAS Proc
357 0 1 0 0.00% 0.00% 0.00% 0 IPSec background
358 285 101 2821 0.00% 0.00% 0.00% 0 Crypto CA
359 0 1 0 0.00% 0.00% 0.00% 0 Crypto PKI-CRL
360 0 2 0 0.00% 0.00% 0.00% 0 MMON PROCESS
361 24659 2720047 9 0.16% 0.01% 0.00% 0 MMON MENG
362 0 1 0 0.00% 0.00% 0.00% 0 HRPC x_setup req
363 0 2 0 0.00% 0.00% 0.00% 0 REP Switch Helpe
364 0 1 0 0.00% 0.00% 0.00% 0 Licensing MIB pr
365 0 23 0 0.00% 0.00% 0.00% 0 VTP Trap Process
366 8 683 11 0.00% 0.00% 0.00% 0 ASP Process Crea
367 327 962 339 0.00% 0.00% 0.00% 0 AAA SEND STOP EV
368 0 1 0 0.00% 0.00% 0.00% 0 Test AAA Client
369 0 2 0 0.00% 0.00% 0.00% 0 DHCP Security He
370 8 2 4000 0.00% 0.00% 0.00% 0 EEM ED Routing
371 294 3215 91 0.00% 0.00% 0.00% 0 EEM ED Syslog
372 0 1 0 0.00% 0.00% 0.00% 0 Syslog Traps
373 431 49 8795 0.00% 0.00% 0.00% 0 OBFL UPTIME obfl
374 52 1456 35 0.00% 0.00% 0.00% 0 vlmap logger
375 0 1 0 0.00% 0.00% 0.00% 0 HCD Process
376 0 1 0 0.00% 0.00% 0.00% 0 HRPC cable diagn
377 0 1 0 0.00% 0.00% 0.00% 0 HRPC rsu request
378 323 93 3473 0.00% 0.00% 0.00% 0 SpanTree Helper
379 0 2 0 0.00% 0.00% 0.00% 0 DiagCard1/-1
380 0 1 0 0.00% 0.00% 0.00% 0 Online Diag EEM
381 18304 785407 23 0.00% 0.06% 0.01% 0 PM Callback
382 94 487 193 0.00% 0.00% 0.00% 0 HCMP sync proces
383 45 6994 6 0.00% 0.00% 0.00% 0 dhcp snooping sw
384 26 189 137 0.00% 0.00% 0.00% 0 SpanTree Flush
385 9 69 130 0.00% 0.00% 0.00% 0 EEM Server
386 17 2 8500 0.00% 0.00% 0.00% 0 Call Home proces
387 0 4 0 0.00% 0.00% 0.00% 0 CTS credentials
388 0 2 0 0.00% 0.00% 0.00% 0 EEM Policy Direc
389 0 1 0 0.00% 0.00% 0.00% 0 HRPC eee request
390 0 2 0 0.00% 0.00% 0.00% 0 EEM ED CLI
391 0 3 0 0.00% 0.00% 0.00% 0 EEM ED Config
392 0 3 0 0.00% 0.00% 0.00% 0 EEM ED Counter
393 8 3 2666 0.00% 0.00% 0.00% 0 EEM ED Env
394 0 3 0 0.00% 0.00% 0.00% 0 EM ED GOLD
395 0 3 0 0.00% 0.00% 0.00% 0 EEM ED Interface
396 0 3 0 0.00% 0.00% 0.00% 0 EEM ED IOSWD
397 0 3 0 0.00% 0.00% 0.00% 0 EEM ED Ipsla
398 0 3 0 0.00% 0.00% 0.00% 0 EEM ED None
399 0 3 0 0.00% 0.00% 0.00% 0 EEM ED OIR
400 76 1456 52 0.00% 0.00% 0.00% 0 Call Home Timer
401 0 3 0 0.00% 0.00% 0.00% 0 EEM ED SNMP
402 0 3 0 0.00% 0.00% 0.00% 0 EEM ED SNMP Obje
403 0 2 0 0.00% 0.00% 0.00% 0 EEM ED SNMP Noti
404 0 3 0 0.00% 0.00% 0.00% 0 EEM ED Test
405 26 2705 9 0.00% 0.00% 0.00% 0 EEM ED Timer
406 3470 1796 1932 0.00% 0.00% 0.00% 0 Syslog
407 0 37 0 0.00% 0.00% 0.00% 0 RBM CORE
408 0 2 0 0.00% 0.00% 0.00% 0 STP FAST TRANSIT
409 0 83 0 0.00% 0.00% 0.00% 0 CSRT RAPID TRANS
410 32 6 5333 0.00% 0.00% 0.00% 0 VLAN Manager
411 9 11 818 0.00% 0.00% 0.00% 0 CEF RP IPC Backg
412 100 4 25000 0.00% 0.00% 0.00% 0 OBFL MSG obfl0
413 0 27 0 0.00% 0.00% 0.00% 0 SSH Event handle
414 9903 5654 1751 0.00% 0.00% 0.00% 0 RADIUS
415 0 1 0 0.00% 0.00% 0.00% 0 DiagCard4/-1
416 0 1 0 0.00% 0.00% 0.00% 0 DiagCard5/-1
417 0 1 0 0.00% 0.00% 0.00% 0 DiagCard6/-1
418 1378 5563 247 0.00% 0.00% 0.00% 0 Dot1x Authentica
419 1610 6897 233 0.00% 0.00% 0.00% 0 Dot1x Authentica
420 923 1960 470 0.00% 0.00% 0.00% 0 IP SNMP
421 152 980 155 0.00% 0.00% 0.00% 0 PDU DISPATCHER
422 276 982 281 0.00% 0.00% 0.00% 0 SNMP ENGINE
423 0 2 0 0.00% 0.00% 0.00% 0 IP SNMPV6
424 0 1 0 0.00% 0.00% 0.00% 0 SNMP ConfCopyPro
425 764 111 6882 0.00% 0.00% 0.00% 0 SNMP Traps
426 68 1456 46 0.00% 0.00% 0.00% 0 DHCPD Database
427 0 1 0 0.00% 0.00% 0.00% 0 Connection Mgr
428 0 1 0 0.00% 0.00% 0.00% 0 LICENSE AGENT
429 408 1455 280 0.00% 0.00% 0.00% 0 RADIUS IO STATS
430 58 5 11600 0.00% 0.00% 0.00% 0 OBFL ENV obfl0
431 83 8746 9 0.00% 0.00% 0.00% 0 MACsec Device Pr
432 42288 1462 28924 0.00% 0.04% 0.00% 0 OBFL VOLT obfl0
433 274 5798 47 0.00% 0.00% 0.00% 0 OBFL I/O Buffer
435 1103 58791 18 0.00% 0.00% 0.00% 0 NTP
------------------ show process cpu history ------------------
11111
599999999999999999999000009999944444333334444433333333333333
922222111118888899999000009999944444555557777755555333334444
100 ********************
90 ******************************
80 ******************************
70 ******************************
60 *******************************
50 ******************************* *****
40 ***************************************************
30 **********************************************************
20 **********************************************************
10 **********************************************************
0....5....1....1....2....2....3....3....4....4....5....5....6
0 5 0 5 0 5 0 5 0 5 0
CPU% per second (last 60 seconds)
1
045444454445455456454544444445544445554444455544444554444444
014244939570204716609267997849984663119893502059796003393289
100 *
90 *
80 *
70 * *
60 # * **
50 # * ****** *************** *** ******** *********** *
40 #####*################################################*###
30 ##########################################################
20 ##########################################################
10 ##########################################################
0....5....1....1....2....2....3....3....4....4....5....5....6
0 5 0 5 0 5 0 5 0 5 0
CPU% per minute (last 60 minutes)
* = maximum CPU% # = average CPU%
1 1
555556505559555098555559
975760606569658093777769
100 * * ** *
90 * * ** *
80 * * *** *
70 * * *** *
60 ************************
50 ***************#********
40 ########################
30 ########################
20 ########################
10 ########################
0....5....1....1....2....2....3....3....4....4....5....5....6....6....7..
0 5 0 5 0 5 0 5 0 5 0 5 0
CPU% per hour (last 72 hours)
* = maximum CPU% # = average CPU%
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide