High cpu utilization on 7206VXR/ NPG-1

murugananthamamir · ‎03-20-2013

Cpu always hitting 70% to 80 % and interrrupt traffic is 70%. Want to know the root cause for this issue.

Tech support is attached for your analyze. please help me here.

CPU utilization for five seconds: 73%/65%; one minute: 71%; five minutes: 67%

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

1 8 59 135 0.00% 0.00% 0.00% 0 Chunk Manager

2 58864 3290773 17 0.00% 0.00% 0.00% 0 Load Meter

3 1784 238001 7 0.07% 0.01% 0.00% 2 Virtual Exec

4 0 1 0 0.00% 0.00% 0.00% 0 EDDRI_MAIN

5 5855500 1701715 3440 0.00% 0.01% 0.00% 0 Check heaps

6 7032 95017 74 0.00% 0.00% 0.00% 0 Pool Manager

7 0 2 0 0.00% 0.00% 0.00% 0 AAA_SERVER_DEADT

8 0 2 0 0.00% 0.00% 0.00% 0 Timers

9 0 2 0 0.00% 0.00% 0.00% 0 Serial Backgroun

10 0 2 0 0.00% 0.00% 0.00% 0 ATM Idle Timer

11 0 2 0 0.00% 0.00% 0.00% 0 ATM AutoVC Perio

12 0 2 0 0.00% 0.00% 0.00% 0 ATM VC Auto Crea

13 0 2 0 0.00% 0.00% 0.00% 0 AAA high-capacit

14 710988 16441320 43 0.00% 0.00% 0.00% 0 EnvMon

15 0 1 0 0.00% 0.00% 0.00% 0 OIR Handler

16 580 274258 2 0.00% 0.00% 0.00% 0 IPC Dynamic Cach

17 0 1 0 0.00% 0.00% 0.00% 0 IPC Zone Manager

18 85464 16440614 5 0.00% 0.00% 0.00% 0 IPC Periodic Tim

19 47392 16440585 2 0.00% 0.00% 0.00% 0 IPC Deferred Por

20 0 1 0 0.00% 0.00% 0.00% 0 IPC Seat Manager

21 435020 3808098 114 0.00% 0.00% 0.00% 0 ARP Input

22 113028 4079632 27 0.00% 0.00% 0.00% 0 HC Counter Timer

23 0 2 0 0.00% 0.00% 0.00% 0 DDR Timers

24 4 4 1000 0.00% 0.00% 0.00% 0 Entity MIB API

25 0 1 0 0.00% 0.00% 0.00% 0 SERIAL A'detect

26 80888 16440570 4 0.00% 0.00% 0.00% 0 GraphIt

27 0 2 0 0.00% 0.00% 0.00% 0 Dialer event

28 0 3 0 0.00% 0.00% 0.00% 0 Critical Bkgnd

29 30112352 5746934 5239 0.07% 0.13% 0.10% 0 Net Background

30 252 1382 182 0.00% 0.00% 0.00% 0 Logger

31 152804 16440549 9 0.00% 0.00% 0.00% 0 TTY Background

32 9165828 16451271 557 0.15% 0.02% 0.00% 0 Per-Second Jobs

33 0 1 0 0.00% 0.00% 0.00% 0 Inode Table Dest

34 0 1 0 0.00% 0.00% 0.00% 0 CSP Timer

35 0 1 0 0.00% 0.00% 0.00% 0 SONET alarm time

36 0 2 0 0.00% 0.00% 0.00% 0 VNM DSPRM MAIN

37 0 1 0 0.00% 0.00% 0.00% 0 CES Line Conditi

38 0 2 0 0.00% 0.00% 0.00% 0 Flash MIB Update

39 0 2 0 0.00% 0.00% 0.00% 0 ATM OAM Input

40 0 2 0 0.00% 0.00% 0.00% 0 ATM OAM TIMER

41 4 418 9 0.00% 0.00% 0.00% 0 TurboACL

42 224 1964 114 0.00% 0.00% 0.00% 0 AAA Server

43 300 2630 114 0.00% 0.00% 0.00% 0 AAA ACCT Proc

44 0 1 0 0.00% 0.00% 0.00% 0 ACCT Periodic Pr

45 0 2 0 0.00% 0.00% 0.00% 0 AAA Dictionary R

46 822800444-1245556834 0 7.43% 6.98% 6.60% 0 IP Input

47 0 1 0 0.00% 0.00% 0.00% 0 ICMP event handl

48 512 3692 138 0.00% 0.00% 0.00% 0 TACACS+

49 0 1 0 0.00% 0.00% 0.00% 0 SSS Manager

50 6016 2193933 2 0.00% 0.00% 0.00% 0 SSS Test Client

51 0 3 0 0.00% 0.00% 0.00% 0 PPP Hooks

52 0 1 0 0.00% 0.00% 0.00% 0 X.25 Encaps Mana

53 0 1 0 0.00% 0.00% 0.00% 0 VPDN call manage

54 177856 295191 602 0.00% 0.00% 0.00% 0 IP Background

55 9348 274648 34 0.00% 0.00% 0.00% 0 IP RIB Update

56 0 2 0 0.00% 0.00% 0.00% 0 SNMP Timers

57 0 2 0 0.00% 0.00% 0.00% 0 PPP IP Route

58 0 2 0 0.00% 0.00% 0.00% 0 PPP IPCP

59 8896 274298 32 0.00% 0.00% 0.00% 0 Adj Manager

60 50216 1214216 41 0.00% 0.00% 0.00% 0 CEF process

61 260824 9730071 26 0.00% 0.00% 0.00% 0 TCP Timer

62 17064 71428 238 0.00% 0.00% 0.00% 0 TCP Protocols

63 0 1 0 0.00% 0.00% 0.00% 0 Socket Timers

64 336 54854 6 0.00% 0.00% 0.00% 0 HTTP CORE

65 0 1 0 0.00% 0.00% 0.00% 0 RARP Input

66 42879512 321782490 133 0.23% 0.29% 0.23% 0 IP NAT Ager

67 1997908 274201 7286 0.00% 0.00% 0.00% 0 IP Cache Ager

68 0 1 0 0.00% 0.00% 0.00% 0 COPS

69 0 2 0 0.00% 0.00% 0.00% 0 PPP SSS

70 4 2 2000 0.00% 0.00% 0.00% 0 PPP Bind

71 0 1 0 0.00% 0.00% 0.00% 0 PAD InCall

72 0 2 0 0.00% 0.00% 0.00% 0 X.25 Background

73 0 2 0 0.00% 0.00% 0.00% 0 Dialer Forwarder

74 144060 16440905 8 0.00% 0.00% 0.00% 0 RUDPV1 Main Proc

75 0 1 0 0.00% 0.00% 0.00% 0 bsm_timers

76 40448 16440542 2 0.00% 0.00% 0.00% 0 bsm_xmt_proc

77 0 2 0 0.00% 0.00% 0.00% 0 RLM groups Proce

78 0 3 0 0.00% 0.00% 0.00% 0 gk process

79 0 1 0 0.00% 0.00% 0.00% 0 Border Element p

80 3592 658237 5 0.00% 0.00% 0.00% 0 CRM_CALL_UPDATE_

81 12 43 279 0.00% 0.00% 0.00% 0 LOCAL AAA

82 0 2 0 0.00% 0.00% 0.00% 0 ENABLE AAA

83 0 2 0 0.00% 0.00% 0.00% 0 LINE AAA

84 11628 16331 712 0.00% 0.00% 0.00% 0 TPLUS

85 484 33334 14 0.00% 0.00% 0.00% 0 CEF Scanner

86 0 2 0 0.00% 0.00% 0.00% 0 ASNL

87 0 2 0 0.00% 0.00% 0.00% 0 VSP_MGR

88 0 1 0 0.00% 0.00% 0.00% 0 VoIP AAA

89 0 1 0 0.00% 0.00% 0.00% 0 QOS_MODULE_MAIN

90 0 1 0 0.00% 0.00% 0.00% 0 RPMS_PROC_MAIN

91 0 1 0 0.00% 0.00% 0.00% 0 CCVPM_HTSP

92 0 1 0 0.00% 0.00% 0.00% 0 CCVPM_R2

93 0 1 0 0.00% 0.00% 0.00% 0 CCSWVOICE

94 0 1 0 0.00% 0.00% 0.00% 0 sssapp

95 0 1 0 0.00% 0.00% 0.00% 0 http client proc

96 170744 3958550 43 0.00% 0.00% 0.00% 0 traffic_shape

97 0 2 0 0.00% 0.00% 0.00% 0 Proxy Session Ap

98 3076 1645411 1 0.00% 0.00% 0.00% 0 RMON Recycle Pro

99 0 2 0 0.00% 0.00% 0.00% 0 RMON Deferred Se

100 0 1 0 0.00% 0.00% 0.00% 0 SYSMGT Events

101 196 956 205 0.00% 0.00% 0.00% 0 AAA SEND STOP EV

102 0 1 0 0.00% 0.00% 0.00% 0 RMON Packets

103 10137352-337816004 0 0.07% 0.02% 0.00% 0 SAA Event Proces

104 0 1 0 0.00% 0.00% 0.00% 0 VPDN Scal

105 60864 16431137 3 0.00% 0.00% 0.00% 0 trunk conditioni

106 0 1 0 0.00% 0.00% 0.00% 0 trunk conditioni

108 190748 9746448 19 0.00% 0.00% 0.00% 0 Net Input

109 155028 3290773 47 0.00% 0.00% 0.00% 0 Compute load avg

110 3366928 283081 11893 0.00% 0.00% 0.00% 0 Per-minute Jobs

111 2856 137133 20 0.00% 0.00% 0.00% 0 CC-API_VCM

112 1238408 163822020 7 0.00% 0.00% 0.00% 0 CCPROXY_CT

115 0 2 0 0.00% 0.00% 0.00% 0 IP Flow Backgrou

117 40 223 179 0.00% 0.00% 0.00% 0 TCP Listener

118 8396 509053 16 0.00% 0.00% 0.00% 0 Kron Process

119 614248 6974492 88 0.00% 0.00% 0.00% 0 IP SNMP

120 154524 3451334 44 0.00% 0.00% 0.00% 0 PDU DISPATCHER

121 1439416 3452097 416 0.00% 0.00% 0.00% 0 SNMP ENGINE

122 0 1 0 0.00% 0.00% 0.00% 0 SNMP ConfCopyPro

123 380 389 976 0.00% 0.00% 0.00% 0 SNMP Traps

124 2784720 112449663 24 0.00% 0.02% 0.00% 0 NAT MIB Helper

125 197792 16968400 11 0.00% 0.00% 0.00% 0 NTP

126 0 1 0 0.00% 0.00% 0.00% 0 xcpa-driver

127 711364 48065369 14 0.00% 0.00% 0.00% 0 BGP Router

128 1371696 19810711 69 0.07% 0.01% 0.00% 0 BGP I/O

129 2527848 1098635 2300 0.00% 0.00% 0.00% 0 BGP Scanner

------------------ show process cpu history ------------------

Prithvi-II 09:36:02 AM Monday Mar 18 2013 IST

7777777777777777777666666666677777888888888877777777776666

3333000002222222222555556666633333888880000000000666665555

100

90 *****

80 ********** *****

70 ***********************************************************

60 ************************************************************

50 ************************************************************

40 ************************************************************

30 ************************************************************

20 ************************************************************

10 ************************************************************

0....5....1....1....2....2....3....3....4....4....5....5....

0 5 0 5 0 5 0 5 0 5

CPU% per second (last 60 seconds)

8788778787767776666666655555665655555556555555554554444454

8700006800683007907435179594249298558400698598424142566638

100

90 * *

80 **** #** *

70 ####**#####*##*** * *

60 #######################**** *#******* ******* *

50 ######################################*########* *# ********

40 ############################################################

30 ############################################################

20 ############################################################

10 ############################################################

0....5....1....1....2....2....3....3....4....4....5....5....

0 5 0 5 0 5 0 5 0 5

CPU% per minute (last 60 minutes)

* = maximum CPU% # = average CPU%

6431233113222422223212332333233113222444444355533342233224534669999999

0419341677964791622194329210526840857467010439877485732653735880698868

100 *******

90 **#*#*#**

80 *#######*

70 **#########

60 * ** * **#########

50 * * ** *** * * **##########

40 ** * * * ****** ***** * ** *###########

30 #** ** *** ** * * ********* ***********#****************###########

20 ##****************************** ******#########**********##############

10 ########################################################################

0....5....1....1....2....2....3....3....4....4....5....5....6....6....7.

0 5 0 5 0 5 0 5 0 5 0 5 0

CPU% per hour (last 72 hours)

* = maximum CPU% # = average CPU%

paolo bevilacqua · ‎03-20-2013

You have enabled a lot of features, NAT , QoS, flow esport, ACL just to name a few. And handling traffic that can probably have burst of some hundreds of mbps, see drops.

So all in all CPU utilization seems normal.