cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2845
Views
0
Helpful
2
Replies

High CPU Usage on C4507

Dian Rubens
Level 1
Level 1

Hi community.

 

A 4507 switch has high CPU issues, if we introduce the command show processes cpu, the outputs says that a process called Cat4k Mgmt LoPri consumes aprox 80 to 90%:

 

4507#show processes cpu
CPU utilization for five seconds: 94%/4%; one minute: 95%; five minutes: 97%
 PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process

 69           0           3          0  0.00%  0.00%  0.00%   0 ARP HA           
  70           0           2          0  0.00%  0.00%  0.00%   0 SISF HA Process  
  71           0           4          0  0.00%  0.00%  0.00%   0 MSP HA           
  72           0           2          0  0.00%  0.00%  0.00%   0 MSP HA IPC flow  
  73           0           1          0  0.00%  0.00%  0.00%   0 DHCPC HA         
  74           0           1          0  0.00%  0.00%  0.00%   0 DHCPD HA         
  75           0           1          0  0.00%  0.00%  0.00%   0 Network-rf Notif
  76    61748992   169810858        363  4.31%  4.32%  4.31%   0 Cat4k Mgmt HiPri
  77   350784471   392501329        893 78.07% 79.72% 81.35%   0 Cat4k Mgmt LoPri
  78         880      395330          2  0.00%  0.00%  0.00%   0 Galios Reschedul
  79           0           1          0  0.00%  0.00%  0.00%   0 IOS SGACL Helper
  80         596         537       1109  0.00%  0.00%  0.00%   0 GaliosObflLogger
  81           0           4          0  0.00%  0.00%  0.00%   0 IOS ACL Helper   
  82           0           1          0  0.00%  0.00%  0.00%   0 GalRedMsgProc    
  83           8          16        500  0.00%  0.00%  0.00%   0 RF Master Main T
  84           0          11          0  0.00%  0.00%  0.00%   0 RF Master Status

 

Following the guide for High CPU utilization Catalyst 4500 switches (https://www.cisco.com/c/en/us/support/docs/switches/catalyst-4000-series-switches/65591-cat4500-high-cpu.html), we introduce the command show platform health with the item named K5CpuMan Review consuming an 88% of CPU.

 

4507#show platform health
                     %CPU   %CPU    RunTimeMax   Priority  Average %CPU  Total
                     Target Actual Target Actual   Fg   Bg 5Sec Min Hour  CPU
RkiosObflMan           0.50   0.00      4      0  100  500    0   0    0  0:31
VSI slot-03            1.00   0.05      6      0  100  500    0   0    0  9:28

K5 L2 Hardware Addre   2.00   0.00     20     10  100  500    0   0    0  2549:05
K5 L2 Hardware Mac L   1.00   0.00      2      0  100  500    0   0    0  0:00
K5RetStatsMan Review   2.00   0.00      5      0  100  500    0   0    0  0:00
K5CpuMan Review       30.00  88.37     30     21  100  500  166 167  124  5353:16
K5ForerunnerPacketMa   2.00   4.81      4      0  100  500    5   5    4  367:29
K5ForerunnerPacketMa   2.00   0.28      4      0  100  500    0   0    0  28:26

 

Introducing the command show platform cpu packet statistics we saw that the switch is dropping massive packets on l2 router (the complete output below):

 

4507#show platform cpu packet statistics


RkGenericPacketMan:
Packet allocation failures: 0
Packet Buffer(SW Common) allocation failures: 0
Packet Buffer(SW ESMP) allocation failures: 0
Packet Buffer(SW EOBC) allocation failures: 0
Packet Buffer(SW SupToSup) allocation failures: 0

Packets Dropped In Processing Overall

Total                5 sec avg 1 min avg 5 min avg 1 hour avg
-------------------- --------- --------- --------- ----------
            84346886        42        23        23         25

Packets Dropped In Processing by CPU event

Event             Total                5 sec avg 1 min avg 5 min avg 1 hour avg
----------------- -------------------- --------- --------- --------- ----------
Sa Miss                       65560165         0         0         0          0
L2 Router                     18773630        42        23        22         24
Input Acl Fwd                       58         0         0         0          0
Input ACl Copy                   11226         0         0         0          0
Output Acl Copy                   1807         0         0         0          0

Packets Dropped In Processing by Priority

Priority          Total                5 sec avg 1 min avg 5 min avg 1 hour avg
----------------- -------------------- --------- --------- --------- ----------
Normal                        18781484        42        23        22         24
Medium                        65560165         0         0         0          0
High                              5237         0         0         0          0

Packets Dropped In Processing by Reason

Reason             Total                5 sec avg 1 min avg 5 min avg 1 hour avg
------------------ -------------------- --------- --------- --------- ----------
STPDrop                             473         0         0         0          0
AclActionDrop                      1802         0         0         0          0
NoDstPorts                            5         0         0         0          0
Tx Mode Drop                   65570976         0         0         0          0
Extra packet due to RxProc hwAssisted Bridging             18773630        42        23        22         24

Total packet queues 64

Packets Received by Packet Queue
          
Queue                  Total           5 sec avg 1 min avg 5 min avg 1 hour avg
---------------------- --------------- --------- --------- --------- ----------
Esmp                          17334707        28        20        17          9
Input ACL fwd(snooping)       288909409       260       242       195        187
Host Learning                 65554306         0         0         0          0
L2 Control                     9084507        15         8        10          1
Input ACL log, unreach           11226         0         0         0          0
Ttl Expired                        119         0         0         0          0
Output ACL log, unreach            1807         0         0         0          0
Adj SameIf Fail                 457675         6         0         0          0
Bfd                                 73         0         0         0          0
L2 router to CPU, 7         6264331074      6337     19380     17625      15668
L3 Glean, 7                    1309226         5         0         0          0
L3 Fwd, 7                           80         0         0         0          0
L3 Receive, 7                  1604009         7         2         2          1

Packets Dropped by Packet Queue

Queue                  Total           5 sec avg 1 min avg 5 min avg 1 hour avg
---------------------- --------------- --------- --------- --------- ----------
Esmp                               386         0         0         0          0
Host Learning                     1040         0         0         0          0
Adj SameIf Fail                    895         0         0         0          0
L2 router to CPU, 7           59168345        33        97        29          7

 

Anyone knows what is the association of the last line? What can be causing the overprocesing of the CPU?

 

Thanks.

2 Replies 2

Damien Miller
VIP Alumni
VIP Alumni
Dian,

Are you running VSS? The "SA Miss" is very high which leads me to believe one of the two bugs I linked below. You didn't mention the sup or ios you're running but I find these aren't always accurate on the public facing reports anyways. Are you running and SPAN sessions on the chassis? sh monitor session will spit that out. I would suggest a TAC case if you have an active agreement.

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCtb44674
https://bst.cloudapps.cisco.com/bugsearch/bug/CSCuh50329

Leo Laohoo
Hall of Fame
Hall of Fame
Post the complete output to the following commands:
1. sh proc cpu sort | ex 0.00
2. sh version