06-21-2012 04:53 AM - edited 03-04-2019 04:44 PM
Hi Friends,
Need your help. I am observing very high CPU utilization on my Cisco 7206VXR (NPE-G1) Router. We have done all the necessary troubleshooting but not able to identify root cause. I need your help to identify the root cause.
R1#sh process cpu | i 319
19 1546984 13195745 117 0.07% 0.04% 0.05% 0 ARP Input
293 24 3196 7 0.00% 0.00% 0.00% 0 TCP Listener
319 174635820 91704341 1904 2.00% 4.04% 3.55% 0 BGP Router
R1#sh process cpu | i 109
6 1092 3936 277 0.00% 0.00% 0.00% 0 Pool Manager
95 2404024 109044673 22 0.07% 0.03% 0.02% 0 Spanning Tree
109 284 139 2043 0.00% 0.00% 0.00% 0 PPP IP Route
R1#sh process cpu | i 105
105 184299056 43534715 4233 2.39% 4.00% 4.45% 0 IP RIB Update
R1#sh ver | i .bin
System image file is "disk2:c7200-adventerprisek9-mz.124-22.T5.bin"
R1#sh process cpu | e 0.00% 0.00% 0.00%
CPU utilization for five seconds: 37%/30%; one minute: 48%; five minutes: 48%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
2 84432 807096 104 0.07% 0.03% 0.02% 0 Load Meter
5 12508996 693791 18029 0.00% 0.22% 0.23% 0 Check heaps
19 1547136 13197135 117 0.07% 0.04% 0.05% 0 ARP Input
39 5212096 1553431 3355 0.31% 0.13% 0.12% 0 Net Background
43 51486852 4386344 11738 1.19% 1.17% 1.19% 0 Per-Second Jobs
49 215888 820053 263 0.00% 0.03% 0.02% 0 Compute load avg
50 1279180 76565 16707 0.00% 0.02% 0.00% 0 Per-minute Jobs
67 287428 13751297 20 0.07% 0.01% 0.00% 0 Voice PA Proc
82 32287968 216884136 148 0.39% 0.44% 0.43% 0 IP Input
95 2404188 109049242 22 0.00% 0.03% 0.02% 0 Spanning Tree
104 4330112 247267 17511 0.00% 0.09% 0.06% 0 IP Background
105 184317404 43536696 4233 1.27% 4.65% 4.64% 0 IP RIB Update
107 457488 5720165 79 0.07% 0.08% 0.08% 0 CEF: IPv4 proces
112 189312 22453864 8 0.07% 0.04% 0.05% 0 TCP Timer
140 853752 924776262 0 0.31% 0.29% 0.30% 0 HQF Shaper Backg
141 84108 39143458 2 0.00% 0.01% 0.00% 0 RBSCP Background
173 196688 437440 449 0.00% 0.01% 0.00% 0 TPLUS
188 27240072 56483059 482 0.31% 0.48% 0.50% 0 BGP I/O
190 27909872 316511 88181 0.00% 0.87% 0.84% 0 BGP Scanner
216 680 514 1322 0.23% 0.18% 0.08% 2 SSH Process
253 111224 39148448 2 0.00% 0.02% 0.00% 0 CCPROXY_CT
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
257 529752 2439497 217 0.07% 0.01% 0.00% 0 LDP
258 1166712 29138570 40 0.07% 0.07% 0.07% 0 TDP Hello
259 293012 1975923 148 0.00% 0.02% 0.00% 0 Tag Control
261 1567032 414671 3778 0.07% 0.04% 0.05% 0 MFI LFD Stats Pr
264 99176 4536258 21 0.00% 0.01% 0.00% 0 IGMP Input
265 120920 39470863 3 0.00% 0.01% 0.00% 0 Mwheel Process
266 758428 86705395 8 0.15% 0.11% 0.10% 0 PIM Process
273 335092 121538894 2 0.07% 0.07% 0.07% 0 PPP manager
274 287772 121539367 2 0.00% 0.05% 0.05% 0 PPP Events
276 25510540 15956971 1598 0.31% 0.60% 0.63% 0 Collection proce
279 62448 3983904 15 0.00% 0.01% 0.00% 0 MPLS HC Counter
285 75224 39354897 1 0.00% 0.01% 0.00% 0 FR Broadcast Out
294 1652096 15266545 108 0.00% 0.06% 0.06% 0 IP SNMP
295 731164 7538178 96 0.00% 0.03% 0.01% 0 PDU DISPATCHER
296 2756396 7552030 364 0.00% 0.20% 0.17% 0 SNMP ENGINE
309 811248 5034073 161 0.00% 0.01% 0.00% 0 OSPF-200 Router
313 4059664 13742716 295 0.07% 0.01% 0.00% 0 OSPF-480 Router
315 567256 4390468 129 0.00% 0.02% 0.00% 0 OSPF-590 Router
316 1982792 8404332 235 0.00% 0.01% 0.00% 0 OSPF-1 Router
319 174652228 91708521 1904 2.23% 3.78% 3.80% 0 BGP Router
327 677216 16678568 40 0.07% 0.05% 0.06% 0 OSPF-1 Hello
328 131448 2608238 50 0.07% 0.00% 0.00% 0 OSPF-1255 Hello
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
332 1515184 6444219 235 0.07% 0.01% 0.00% 0 OSPF-450 Router
335 21440 601 35673 0.00% 1.50% 1.83% 6 SSH Process
06-21-2012 07:39 AM
Can you clarify what you mean by high CPU utilisation i.e +80%. Can you do a "show proc cpu sorted" How much free memory does the router have? BGP appears to be enabled, so how big(how many routes) is the BGP routing table?
06-21-2012 08:06 AM
06-21-2012 10:36 AM
hi Tarun,
i had similar problem in the old days, using NPE-G2 running bgp and mpls. i did open a tac case, but then this case were handled by my team mate, so im not quite sure the workaround.
really interesting what you have here.
06-21-2012 12:05 PM
Can any body let me know the workaround. Eager to know the right solution
06-21-2012 05:42 PM
Disclaimer
The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.
Liability Disclaimer
In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.
Posting
What's the aggregate traffic volume when you see about 50% CPU? From your stats, looks like CPU is forwarding packets.
06-21-2012 09:43 PM
hi Tarun,
after contacting my colleages, it turns out that it was ios bug because of an incomplete mpls label. the solution was upgrading the ios to SRD.
i dont know if the environment are the same with yours.
regards,
06-22-2012 12:03 AM
It is one of the core router. Aggregrate traffic will be around 75-80 Mb.
Is this IOS Image having bug. Please confirm. I am not able to find same it Bug Tool Kit.
R1#sh ver | i .bin
System image file is "disk2:c7200-adventerprisek9-mz.124-22.T5.bin"
Rgds
06-22-2012 05:05 AM
hi Tarun,
http://www.cisco.com/en/US/docs/ios/12_4t/release/notes/124TCAVS3.html
CSCek71050
it was service provider feature set, SRB1
anyway, for NPE-G1 in 7206vxr, 75-80 Mb should be no problem at all.
regards,
06-23-2012 08:40 PM
Then what could be the root cause of High CPU.
Rgds
06-24-2012 12:02 AM
on my case, it was because the router using SRB1, ios bug when receiving incomplete mpls label packet, causing the cpu process become very high. the workaround was upgrading the to SRD releases.
im suggesting that you engage with cisco TAC team
regards,
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide