High CPU Processes on MPLS 2821

steven.t27 · ‎09-01-2011

Hello

I have a 2821 router which 99% CPU load. I check show processes it indicate that the Pool Manager and IP Input causing a lot of problem.

Can anyone help? Thanks

R-MPLS#sh proc

Load for five secs: 99%/29%; one minute: 99%; five minutes: 98%

Time source is NTP, 20:13:41.017 WST Thu Sep 1 2011

CPU utilization for five seconds: 99%/29%; one minute: 99%; five minutes: 98%

PID QTy PC Runtime (ms) Invoked uSecs Stacks TTY Process

1 Cwe 400AB0A4 0 3 0 5672/6000 0 Chunk Manager

2 Csp 4008F0D0 864168 5429090 159 2524/3000 0 Load Meter

3 Mwe 41FEF484 2620732 9724423 269 4324/6000 0 LDP

4 Mwe 421EE1E0 0 1 023372/24000 0 EDDRI_MAIN

5 Lst 400A7F68 26159996 2804359 9328 5392/6000 0 Check heaps

6 Cwe 400AF138 1401703808 252355086 5554 5480/6000 0 Pool Manager

7 Mst 411497AC 0 2 0 5496/6000 0 Timers

8 Mwe 401124D0 1396 452412 3 5704/6000 0 IPC Dynamic Cach

9 Mwe 401034B0 0 1 0 5700/6000 0 IPC Zone Manager

10 Mwe 40102658 111056 27104550 4 5616/6000 0 IPC Periodic Tim

11 Mwe 401024AC 97876 27104559 3 5640/6000 0 IPC Deferred Por

12 Mwe 4010318C 0 1 0 5564/6000 0 IPC Seat Manager

13 Mwe 4010873C 0 1 0 5668/6000 0 IPC BackPressure

14 Mwe 40259448 0 1 011668/12000 0 OIR Handler

15 Mwe 402F8624 0 1 023604/24000 0 Crash writer

16 Msi 403F98BC 131256 904780 145 5496/6000 0 Environmental mo

17 Mwe 40C68CC4 187904 1520583 123 4748/6000 0 ARP Input

18 Mwe 40CCB688 0 2 0 5444/6000 0 ATM Idle Timer

.....

76 0 1 0 0.00% 0.00% 0.00% 0 ACCT Periodic Pr

77 227924 3423030 66 0.00% 0.00% 0.00% 0 CDP Protocol

78 2524663424-1614061345 0 37.66% 37.35% 36.58% 0 IP Input

79 0 1 0 0.00% 0.00% 0.00% 0 ICMP event handl

80 0 4 0 0.00% 0.00% 0.00% 0 TurboACL

81 0 2 0 0.00% 0.00% 0.00% 0 TurboACL chunk

....

195 470407752-1600026626 0 2.52% 2.60% 2.57% 0 Tag Input

196 45592 2714562 16 0.00% 0.00% 0.00% 0 Tag Control

197 4516012 18962892 238 0.00% 0.00% 0.00% 0 IP SNMP

198 1287372 9446713 136 0.08% 0.00% 0.00% 0 PDU DISPATCHER

199 8860360 9514382 931 0.00% 0.02% 0.02% 0 SNMP ENGINE

200 0 2 0 0.00% 0.00% 0.00% 0 IP SNMPV6

201 0 1 0 0.00% 0.00% 0.00% 0 SNMP ConfCopyPro

202 0 1 0 0.00% 0.00% 0.00% 0 SNMP Traps

203 217736 27357624 7 0.00% 0.00% 0.00% 0 NTP

204 0 11 0 0.00% 0.00% 0.00% 0 AAA Accounting

205 0 3 0 0.00% 0.00% 0.00% 0 IP-EIGRP: PDM

206 674216 5865759 114 0.00% 0.00% 0.00% 0 IP-EIGRP: PDM

207 471488 57875748 8 0.00% 0.00% 0.00% 0 BGP Router

208 0 75 0 0.00% 0.00% 0.00% 0 Tagcon Addr

209 0 1 0 0.00% 0.00% 0.00% 0 IP-EIGRP: HELLO

210 789664 11731560 67 0.00% 0.00% 0.00% 0 IP-EIGRP: HELLO

211 283860 2693814 105 0.00% 0.00% 0.00% 0 BGP I/O

212 3950944 1811980 2180 0.00% 0.00% 0.00% 0 BGP Scanner

R-MPLS#

R-MPLS#sh ver

Load for five secs: 99%/29%; one minute: 99%; five minutes: 98%

Time source is NTP, 20:13:04.136 WST Thu Sep 1 2011

Cisco IOS Software, 2800 Software (C2800NM-SPSERVICESK9-M), Version 12.4(25a), RELEASE SOFTWARE (fc2)

Technical Support: http://www.cisco.com/techsupport

Compiled Fri 22-May-09 22:00 by prod_rel_team

ROM: System Bootstrap, Version 12.4(13r)T, RELEASE SOFTWARE (fc1)

R-MPLS uptime is 44 weeks, 6 days, 4 hours, 23 minutes

System returned to ROM by Reload Command

System restarted at 15:38:57 WST Fri Oct 22 2010

System image file is "flash:c2800nm-spservicesk9-mz.124-25a.bin"

This product contains cryptographic features and is subject to United

States and local country laws governing import, export, transfer and

use. Delivery of Cisco cryptographic products does not imply

third-party authority to import, export, distribute or use encryption.

Importers, exporters, distributors and users are responsible for

compliance with U.S. and local country laws. By using this product you

agree to comply with applicable laws and regulations. If you are unable

to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:

http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to

export@cisco.com.

Cisco 2821 (revision 53.51) with 251904K/10240K bytes of memory.

Processor board ID FHK1246F32R

2 Gigabit Ethernet interfaces

DRAM configuration is 64 bits wide with parity enabled.

239K bytes of non-volatile configuration memory.

62720K bytes of ATA CompactFlash (Read/Write)

Configuration register is 0x2102

R-MPLS#

Reza Sharifi · ‎09-01-2011

Hi,

Have a look at the release notes. The high CPU issue was supposed to be resolved in 12.2(25a), but may still be there.

Resolved Caveats—Cisco IOS Release 12.4(25a)

Cisco IOS Release 12.4(25a) is a rebuild release for Cisco IOS Release 12.4(25). The caveats in this section are resolved in Cisco IOS Release 12.4(25a) but may be open in previous Cisco IOS releases.

•CSCek77849

Symptoms: BGP convergence is very slow, and CPU utilization at BGP router process can reach 100% during the convergence at aggregation router. During normal operation, if BGP prefixes included in the aggregation flap, it will also produce high CPU utilization.

This issue shows the following tendencies:

1) The more of the component prefixes belonging to the aggregate-address entry, the slower the convergence at aggregation router. 2) The more of the duplicated aggregation component prefixes for aggregate- address entry, the slower convergence at aggregation router.

Conditions: Any releases would be affected if "aggregate-address" is configured, and routing updates involving aggregate components are received every few seconds.

Workaround: Remove the "aggregate-address".

Here is the doc:

http://www.cisco.com/en/US/docs/ios/12_4/release/notes/124MCAVS.html#wp312957

HTH

Peter Paluch · ‎09-01-2011

Hi Steven,

In addition to Reza's great reply, please have a close look at the following documents about troubleshooting high CPU load particularly with regard to the IP Input process, and on routers generally. It is possible that you are hitting one of the situations described in these docs.

http://www.cisco.com/en/US/products/hw/routers/ps359/products_tech_note09186a00801c2af3.shtml

http://www.cisco.com/en/US/products/hw/routers/ps133/products_tech_note09186a00800a70f2.shtml

I assume you are running wth IP CEF activated. Running without CEF can be a major source of these problems, so I am better asking to be sure.

Best regards,

Peter