CPU utilization for five seconds: 87%/84%; one minute: 92%; five minutes: 88%

[NBRK_Cisco]

Hi all,

please help

B1#sh proc cpu sort
CPU utilization for five seconds: 87%/84%; one minute: 92%; five minutes: 88%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
324 44008380 461987777 95 0.87% 0.86% 0.83% 0 IP SLAs XOS Even
101 14316976 376106982 38 0.39% 0.41% 0.39% 0 PSE2 UART period
105 15875472 245851843 64 0.39% 0.37% 0.35% 0 Ethernet Msec Ti
134 12748956 19954315 638 0.15% 0.16% 0.16% 0 IP Input
31 7286728 16542628 440 0.15% 0.20% 0.21% 0 ARP Input
14 2774724 1957715 1417 0.15% 0.02% 0.00% 0 Environmental mo
334 762216 2118276 359 0.07% 0.01% 0.00% 0 EIGRP-IPv4 Hello
104 528684 11777492 44 0.07% 0.01% 0.00% 0 Ethernet Timer C
314 143524 391554 366 0.07% 0.00% 0.00% 0 Track
26 6468 196925 32 0.07% 0.00% 0.00% 0 IPC Keep Alive M
48 3711372 14865058 249 0.07% 0.05% 0.07% 0 REVT background
87 4155008 7840412 529 0.07% 0.08% 0.07% 0 Netclock Backgro
69 2309956 1987942 1161 0.07% 0.02% 0.02% 0 Per-Second Jobs
130 2562112 60539016 42 0.07% 0.07% 0.07% 0 IPAM Manager
7 99888 459395 217 0.00% 0.00% 0.00% 0 Pool Manager
13 11431600 32631 350361 0.00% 0.15% 0.12% 0 Licensing Auto U
12 0 1 0 0.00% 0.00% 0.00% 0 Image License br
15 13424 391559 34 0.00% 0.00% 0.00% 0 IPC Event Notifi

#sh ver
Cisco IOS Software, C2900 Software (C2900-UNIVERSALK9-M), Version 15.2(1)T2, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2012 by Cisco Systems, Inc.
Compiled Sun 19-Feb-12 01:34 by prod_rel_team

ROM: System Bootstrap, Version 15.0(1r)M15, RELEASE SOFTWARE (fc1)

BSB_RTR_NB1 uptime is 3 weeks, 1 day, 19 hours, 13 minutes
System returned to ROM by reload at 00:30:46 ALM Sun Aug 13 2017
System restarted at 00:31:34 ALM Sun Aug 13 2017
System image file is "flash0:c2900-universalk9-mz.SPA.152-1.T2.bin"
Last reload type: Normal Reload

[Julio E. Moisa]

Hi

I have experienced situations where a frame is generating high CPU utilization, I suggest configure the following command under the interfaces (no more than 10 mins):

ip accounting mac-address input

ip accounting mac-address output

and verify the who is generating traffic through: show interface mac-accounting. 

Once it is identified you can block the Mac address. 

Hope it is useful

:-)

[Joseph W. Doherty]

You might be bumping into the physical forwarding capacity of the router.

What's your aggregate bandwidth being forwarded when this is happening?

[NBRK_Cisco]

Hi Joseph

thanks for your replay

we have only g0/0 connected as trunk port

and we have about 20 subinterfaces + 10 gre tunnels. + ipsec+nat

but all of these must be cef switched, not process switched. and even not fast switching.

so when cpu was 94%/91%  and input/output load g0/0 was about 27 mbitps

next day cpu was 45%/41%  load g0/0 was about 21 mbitps

and sometimes I see cpu was 33%/31%  load g0/0 was about 87 mbitps

[Joseph W. Doherty]

CEF switching doesn't mean a 2900 CPU doesn't work, it's just Cisco's "best".

Any chance there's fragmentation happening on you tunnels? That will drive up CPU, and in later IOS releases, I believe it's account for as interrupt CPU.

[NBRK_Cisco]

thanks for your again

I will check MTU and fragmentation

may be some server do not configured to support PMTU discavery or similar question

[Julio E. Moisa]

Hi 

Did you try with Ip accounting?

[Joseph W. Doherty]

Just in case you haven't seen it, https://www.cisco.com/c/en/us/support/docs/ip/generic-routing-encapsulation-gre/25885-pmtud-ipfrag.html?dtid=osscdc000283