cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
6265
Views
0
Helpful
3
Replies

High CPU on Cisco 4500

sachin goel
Level 1
Level 1

I have couple of 4500 switches which are running high CPU. Though i know it is process Cat4k Mgmt LoPri which is eating up CPU cycles, i am unable to determine what exactly is the root cause and how to fix it.

Below are the commands for your reference. How do i see which packets are being sent to CPU and what interface is sending them and why.

Cat4500-sw1#sh proc cpu sorted 5min | ex 0.00%  0.00%  0.00%
CPU utilization for five seconds: 100%/0%; one minute: 99%; five minutes: 91%
PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
  49   3924333802675976363          0 95.66% 95.51% 85.66%   0 Cat4k Mgmt LoPri
  48  1496541140 550408613       2718  2.39%  2.59%  3.71%   0 Cat4k Mgmt HiPri
104   5036533161297436277        388  1.16%  1.04%  1.05%   0 Spanning Tree   
  37   239957700  77642786       3090  0.23%  0.24%  0.23%   0 IDB Work        
   7    72783040  10613546       6857  0.00%  0.10%  0.11%   0 Check heaps     
195    45645708 295308027        154  0.07%  0.07%  0.07%   0 SNMP ENGINE     
193    47040588 668827756         70  0.07%  0.05%  0.05%   0 IP SNMP         
  93    10969624 126007784         87  0.07%  0.03%  0.05%   0 CDP Protocol    
100     9995124 313542997         31  0.07%  0.03%  0.00%   0 IP Input        
  94         108       392        275  0.07%  0.02%  0.00%   1 SSH Process     
164     4732892 655867157          7  0.07%  0.01%  0.00%   0 PM Callback     
  55     3418728  17432919        196  0.00%  0.01%  0.00%   0 Compute load avg
  41    21243520   1559662      13620  0.00%  0.01%  0.00%   0 Per-minute Jobs 
  79     3919768 101059994         38  0.00%  0.01%  0.00%   0 DTP Protocol   

Cat4500-its-sw1#sh platform cpu packet statistics al           
Packets Dropped In Hardware By CPU Subport (txQueueNotAvail)

CPU Subport  TxQueue 0       TxQueue 1       TxQueue 2       TxQueue 3
------------ --------------- --------------- --------------- ---------------
           0               0               0               0               0
           1               0               0               0               0
           2               0               0               0               0
           3               0               0               0               0
           4               0               0               0               0
           5               0               0               0               0
           6               0               0               0               0
           7               0               0               0               0

Packets Dropped In Hardware By CPU Subport (txQueueNotAvail)

CPU Subport  TxQueue 4       TxQueue 5       TxQueue 6       TxQueue 7
------------ --------------- --------------- --------------- ---------------
           0               0               0               0               0
           1               0               0               0               0
           2               0               0               0               0
           3               0               0               0               0
           4               0               0               0               0
           5               0               0               0               0
           6               0               0               0               0
           7               0               0               0               0


RkiosSysPacketMan:
Packet allocation failures: 0
Packet Buffer(Software Common) allocation failures: 0
Packet Buffer(Software ESMP) allocation failures: 0
Packet Buffer(Software EOBC) allocation failures: 0
Packet Buffer(Software SupToSup) allocation failures: 0
IOS Packet Buffer Wrapper allocation failures: 0

Packets Enqueued Overall

Total                5 sec avg 1 min avg 5 min avg 1 hour avg
-------------------- --------- --------- --------- ----------
                   0         0         0         0          0

Packets Not Enqueued Overall

Total                5 sec avg 1 min avg 5 min avg 1 hour avg
-------------------- --------- --------- --------- ----------
                   0         0         0         0          0

Packets Dropped In Processing Overall

Total                5 sec avg 1 min avg 5 min avg 1 hour avg
-------------------- --------- --------- --------- ----------
         25285077341         5         2         3          6

Packets Not Enqueued by CPU event

Event             Total                5 sec avg 1 min avg 5 min avg 1 hour avg
----------------- -------------------- --------- --------- --------- ----------
Unknown                              0         0         0         0          0
Sa Miss                              0         0         0         0          0
PVMapping Miss                       0         0         0         0          0
Nfe                                  0         0         0         0          0
CTS                                  0         0         0         0          0
MTU Check Fail                       0         0         0         0          0
ESMP                                 0         0         0         0          0
L2 Control                           0         0         0         0          0
Adj Same If                          0         0         0         0          0
Non Arpa                             0         0         0         0          0
Expired Ttl                          0         0         0         0          0
Ip Option                            0         0         0         0          0
Input If Fail                        0         0         0         0          0
Ucast Rpf Fail                       0         0         0         0          0
SrcIdx Check Fail                    0         0         0         0          0
Ipv6 Scope Fail                      0         0         0         0          0
L2 Bridge                            0         0         0         0          0
L2 Router                            0         0         0         0          0
L3 Glean                             0         0         0         0          0
L3 Forward                           0         0         0         0          0
L3 Receive                           0         0         0         0          0
Input Acl Fwd                        0         0         0         0          0
Input ACl Copy                       0         0         0         0          0
Input Acl Punt                       0         0         0         0          0
Input Acl Err                        0         0         0         0          0
Output Acl Fwd                       0         0         0         0          0
Output Acl Copy                      0         0         0         0          0
Output Acl Punt                      0         0         0         0          0
Output Acl Err                       0         0         0         0          0
Sw Packet w/ TxInfo                    0         0         0         0          0
Sw Packet for Bridge                    0         0         0         0          0
Sw Cpu Span                          0         0         0         0          0

Packets Dropped In Processing by CPU event

Event             Total                5 sec avg 1 min avg 5 min avg 1 hour avg
----------------- -------------------- --------- --------- --------- ----------
Unknown                              0         0         0         0          0
Sa Miss                    24702605024         3         1         2          0
PVMapping Miss                       0         0         0         0          0
Nfe                                  0         0         0         0          0
CTS                                  0         0         0         0          0
MTU Check Fail                       0         0         0         0          0
ESMP                                 0         0         0         0          0
L2 Control                           0         0         0         0          0
Adj Same If                          0         0         0         0          0
Non Arpa                             0         0         0         0          0
Expired Ttl                          0         0         0         0          0
Ip Option                            0         0         0         0          0
Input If Fail                        0         0         0         0          0
Ucast Rpf Fail                       0         0         0         0          0
SrcIdx Check Fail                    0         0         0         0          0
Ipv6 Scope Fail                      0         0         0         0          0
L2 Bridge                            0         0         0         0          0
L2 Router                            0         0         0         0          0
L3 Glean                             0         0         0         0          0
L3 Forward                           0         0         0         0          0
L3 Receive                           0         0         0         0          0
Input Acl Fwd                582472317         2         0         0          2
Input ACl Copy                       0         0         0         0          0
Input Acl Punt                       0         0         0         0          0
Input Acl Err                        0         0         0         0          0
Output Acl Fwd                       0         0         0         0          0
Output Acl Copy                      0         0         0         0          0
Output Acl Punt                      0         0         0         0          0
Output Acl Err                       0         0         0         0          0
Sw Packet w/ TxInfo                    0         0         0         0          0
Sw Packet for Bridge                    0         0         0         0          0
Sw Cpu Span                          0         0         0         0          0

Packets Not Enqueued by Priority

Priority          Total                5 sec avg 1 min avg 5 min avg 1 hour avg
----------------- -------------------- --------- --------- --------- ----------
Unknown                              0         0         0         0          0
Normal                               0         0         0         0          0
Medium                               0         0         0         0          0
High                                 0         0         0         0          0
Crucial                              0         0         0         0          0
Super Crucial                        0         0         0         0          0

Packets Dropped In Processing by Priority

Priority          Total                5 sec avg 1 min avg 5 min avg 1 hour avg
----------------- -------------------- --------- --------- --------- ----------
Unknown                              0         0         0         0          0
Normal                       319883415         0         0         0          0
Medium                     24702605024         3         1         2          0
High                         262588902         2         0         0          1
Crucial                              0         0         0         0          0
Super Crucial                        0         0         0         0          0

Packets Not Enqueued by Reason

Reason            Total                5 sec avg 1 min avg 5 min avg 1 hour avg
---------------- --------------------- --------- --------- --------- ----------
RxQueueNotAvail                      0         0         0         0          0
RxNoBuffersAvail                     0         0         0         0          0
RxBadCpuEvent                        0         0         0         0          0
Testing                              0         0         0         0          0

Packets Dropped In Processing by Reason

Reason             Total                5 sec avg 1 min avg 5 min avg 1 hour avg
------------------ -------------------- --------- --------- --------- ----------
NoPimPhyportMap                       0         0         0         0          0
NoPimPhyport                          0         0         0         0          0
UnimplementedEvt                      0         0         0         0          0
InvalidEvt                            0         0         0         0          0
SrcAddrTableFilt                      0         0         0         0          0
NoL2Vlan                              0         0         0         0          0
STPDrop                             153         0         0         0          0
UnknownLyr-Bridge                     0         0         0         0          0
UnknownLyr-OutAcl                     0         0         0         0          0
L2DstDrop                             0         0         0         0          0
L2DstDropOutAcl                       0         0         0         0          0
AclNoVlan                             0         0         0         0          0
AclActionDrop                         0         0         0         0          0
AclActionRedirect                     0         0         0         0          0
AclActionUnsupptd                     0         0         0         0          0
UnknownBridgeAct                      0         0         0         0          0
NoTxInfo                              0         0         0         0          0
NoDstPorts                    582472185         2         0         0          2
NoFloodPorts                          0         0         0         0          0
ControlNoL2DstEnt                     0         0         0         0          0
InvalidAclAction                      0         0         0         0          0
AclInputErr                           0         0         0         0          0
OutAclEvntOnInAcl                     0         0         0         0          0
AclOutputErr                          0         0         0         0          0
ReservedPort                          0         0         0         0          0
EsmpNoTxInfo                          0         0         0         0          0
UnknownSource                         0         0         0         0          0
NoDriverTxPacketInfo                    0         0         0         0          0
NoPimPort                             0         0         0         0          0
RouteLookupFailed                     0         0         0         0          0
FragPktAllocFailed                    0         0         0         0          0
Dont Fragment Drop                    0         0         0         0          0
IpHdrException                        0         0         0         0          0
SpanEvent                             0         0         0         0          0
Dot1xUnauthCtrlPkt                    0         0         0         0          0
No Sw port for ARP                    0         0         0         0          0
No Sw port for ESMP                    0         0         0         0          0
Tx Mode Drop                24702605003         3         1         2          0
Invalid TxInfo For Replica                    0         0         0         0          0
Qos Policer Drop                      0         0         0         0          0
Bad Crc                               0         0         0         0          0
Invalid EVC packet                    0         0         0         0          0
Unknown EVC packet                    0         0         0         0          0
Invalid Vlan packet                    0         0         0         0          0
Efp not present for Tx packet                    0         0         0         0          0
No space available for EVC shim hdr                    0         0         0         0          0

Total packet queues 64

Packets Received by Packet Queue

Queue                  Total           5 sec avg 1 min avg 5 min avg 1 hour avg
---------------------- --------------- --------- --------- --------- ----------
Mtu Fail                             0         0         0         0          0
Esmp                                 0         0         0         0          0
Input ACL fwd(snooping)               0         0         0         0          0
L2 bridge to CPU, 0                  0         0         0         0          0
L2 router to CPU, 0                  0         0         0         0          0
L3 Glean, 0                          0         0         0         0          0
L3 Fwd, 0                            0         0         0         0          0
L3 Receive, 0                        0         0         0         0          0
Host Learning                        0         0         0         0          0
L2 Control                           0         0         0         0          0
Input ACL log, unreach               0         0         0         0          0
Unknown                              0         0         0         0          0
L2 router to CPU, 1                  0         0         0         0          0
L3 Glean, 1                          0         0         0         0          0
L3 Fwd, 1                            0         0         0         0          0
L3 Receive, 1                        0         0         0         0          0
Unknown                              0         0         0         0          0
Ip Option                            0         0         0         0          0
Input ACL sw processing               0         0         0         0          0
Unknown                              0         0         0         0          0
L2 router to CPU, 2                  0         0         0         0          0
L3 Glean, 2                          0         0         0         0          0
L3 Fwd, 2                            0         0         0         0          0
L3 Receive, 2                        0         0         0         0          0
PV Mapping Miss                      0         0         0         0          0
Ttl Expired                          0         0         0         0          0
Input ACL Error                      0         0         0         0          0
Unknown                              0         0         0         0          0
L2 router to CPU, 3                  0         0         0         0          0
L3 Glean, 3                          0         0         0         0          0
L3 Fwd, 3                            0         0         0         0          0
L3 Receive, 3                        0         0         0         0          0
Unknown                              0         0         0         0          0
Non Arpa                             0         0         0         0          0
Output ACL fwd(snooping)               0         0         0         0          0
Unknown                              0         0         0         0          0
L2 router to CPU, 4                  0         0         0         0          0
L3 Glean, 4                          0         0         0         0          0
L3 Fwd, 4                            0         0         0         0          0
L3 Receive, 4                        0         0         0         0          0
InputIf Fail                         0         0         0         0          0
UcastRpfFail                         0         0         0         0          0
Output ACL log, unreach               0         0         0         0          0
Unknown                              0         0         0         0          0
L2 router to CPU, 5                  0         0         0         0          0
L3 Glean, 5                          0         0         0         0          0
L3 Fwd, 5                            0         0         0         0          0
L3 Receive, 5                        0         0         0         0          0
Ipv6 Scope Check Fail                0         0         0         0          0
SrcIndex Check Fail                  0         0         0         0          0
Output ACL sw processing               0         0         0         0          0
Unknown                              0         0         0         0          0
L2 router to CPU, 6                  0         0         0         0          0
L3 Glean, 6                          0         0         0         0          0
L3 Fwd, 6                            0         0         0         0          0
L3 Receive, 6                        0         0         0         0          0
Unknown                              0         0         0         0          0
Adj SameIf Fail                      0         0         0         0          0
Output ACL Error                     0         0         0         0          0
Unknown                              0         0         0         0          0
L2 router to CPU, 7                  0         0         0         0          0
L3 Glean, 7                          0         0         0         0          0
L3 Fwd, 7                            0         0         0         0          0
L3 Receive, 7                        0         0         0         0          0
         
Packets Dropped by Packet Queue

Queue                  Total           5 sec avg 1 min avg 5 min avg 1 hour avg
---------------------- --------------- --------- --------- --------- ----------
Mtu Fail                             0         0         0         0          0
Esmp                                 0         0         0         0          0
Input ACL fwd(snooping)               0         0         0         0          0
L2 bridge to CPU, 0                  0         0         0         0          0
L2 router to CPU, 0                  0         0         0         0          0
L3 Glean, 0                          0         0         0         0          0
L3 Fwd, 0                            0         0         0         0          0
L3 Receive, 0                        0         0         0         0          0
Host Learning                        0         0         0         0          0
L2 Control                           0         0         0         0          0
Input ACL log, unreach               0         0         0         0          0
Unknown                              0         0         0         0          0
L2 router to CPU, 1                  0         0         0         0          0
L3 Glean, 1                          0         0         0         0          0
L3 Fwd, 1                            0         0         0         0          0
L3 Receive, 1                        0         0         0         0          0
Unknown                              0         0         0         0          0
Ip Option                            0         0         0         0          0
Input ACL sw processing               0         0         0         0          0
Unknown                              0         0         0         0          0
L2 router to CPU, 2                  0         0         0         0          0
L3 Glean, 2                          0         0         0         0          0
L3 Fwd, 2                            0         0         0         0          0
L3 Receive, 2                        0         0         0         0          0
PV Mapping Miss                      0         0         0         0          0
Ttl Expired                          0         0         0         0          0
Input ACL Error                      0         0         0         0          0
Unknown                              0         0         0         0          0
L2 router to CPU, 3                  0         0         0         0          0
L3 Glean, 3                          0         0         0         0          0
L3 Fwd, 3                            0         0         0         0          0
L3 Receive, 3                        0         0         0         0          0
Unknown                              0         0         0         0          0
Non Arpa                             0         0         0         0          0
Output ACL fwd(snooping)               0         0         0         0          0
Unknown                              0         0         0         0          0
L2 router to CPU, 4                  0         0         0         0          0
L3 Glean, 4                          0         0         0         0          0
L3 Fwd, 4                            0         0         0         0          0
L3 Receive, 4                        0         0         0         0          0
InputIf Fail                         0         0         0         0          0
UcastRpfFail                         0         0         0         0          0
Output ACL log, unreach               0         0         0         0          0
Unknown                              0         0         0         0          0
L2 router to CPU, 5                  0         0         0         0          0
L3 Glean, 5                          0         0         0         0          0
L3 Fwd, 5                            0         0         0         0          0
L3 Receive, 5                        0         0         0         0          0
Ipv6 Scope Check Fail                0         0         0         0          0
SrcIndex Check Fail                  0         0         0         0          0
Output ACL sw processing               0         0         0         0          0
Unknown                              0         0         0         0          0
L2 router to CPU, 6                  0         0         0         0          0
L3 Glean, 6                          0         0         0         0          0
L3 Fwd, 6                            0         0         0         0          0
L3 Receive, 6                        0         0         0         0          0
Unknown                              0         0         0         0          0
Adj SameIf Fail                      0         0         0         0          0
Output ACL Error                     0         0         0         0          0
Unknown                              0         0         0         0          0
L2 router to CPU, 7                  0         0         0         0          0
L3 Glean, 7                          0         0         0         0          0
L3 Fwd, 7                            0         0         0         0          0
L3 Receive, 7                        0         0         0         0          0

Packets Transmitted from CPU per Output Interface

Interface              Total           5 sec avg 1 min avg 5 min avg 1 hour avg
---------------------- --------------- --------- --------- --------- ----------
Gi1/1                                0         0         0         0          0
Gi1/2                                0         0         0         0          0
Gi1/3                                0         0         0         0          0
Gi1/4                                0         0         0         0          0
Gi1/5                                0         0         0         0          0
Gi1/6                                0         0         0         0          0
Gi1/7                                0         0         0         0          0
Gi1/8                                0         0         0         0          0
Gi1/9                                0         0         0         0          0
Gi1/10                               0         0         0         0          0
Gi1/11                               0         0         0         0          0
Gi1/12                               0         0         0         0          0
Gi1/13                               0         0         0         0          0
Gi1/14                               0         0         0         0          0
Gi1/15                               0         0         0         0          0
Gi1/16                               0         0         0         0          0
Gi1/17                               0         0         0         0          0
Gi1/18                               0         0         0         0          0
Gi1/19                               0         0         0         0          0
Gi1/20                               0         0         0         0          0
Gi1/21                               0         0         0         0          0
Gi1/22                               0         0         0         0          0
Gi1/23                               0         0         0         0          0
Gi1/24                               0         0         0         0          0
Gi1/25                               0         0         0         0          0
Gi1/26                               0         0         0         0          0
Gi1/27                               0         0         0         0          0
Gi1/28                               0         0         0         0          0
Gi1/29                               0         0         0         0          0
Gi1/30                               0         0         0         0          0
Gi1/31                               0         0         0         0          0
Gi1/32                               0         0         0         0          0
Gi1/33                               0         0         0         0          0
Gi1/34                               0         0         0         0          0
Gi1/35                               0         0         0         0          0
Gi1/36                               0         0         0         0          0
Gi1/37                               0         0         0         0          0
Gi1/38                               0         0         0         0          0
Gi1/39                               0         0         0         0          0
Gi1/40                               0         0         0         0          0
Gi1/41                               0         0         0         0          0
Gi1/42                               0         0         0         0          0
Gi1/43                               0         0         0         0          0
Gi1/44                               0         0         0         0          0
Gi1/45                               0         0         0         0          0
Gi1/46                               0         0         0         0          0
Gi1/47                               0         0         0         0          0
Gi1/48                               0         0         0         0          0
Gi2/1                                0         0         0         0          0
Gi2/2                                0         0         0         0          0
Gi2/3                                0         0         0         0          0
Gi2/4                                0         0         0         0          0
Gi2/5                                0         0         0         0          0
Gi2/6                                0         0         0         0          0
Gi2/7                                0         0         0         0          0
Gi2/8                                0         0         0         0          0
Gi2/9                                0         0         0         0          0
Gi2/10                               0         0         0         0          0
Gi2/11                               0         0         0         0          0
Gi2/12                               0         0         0         0          0
Gi2/13                               0         0         0         0          0
Gi2/14                               0         0         0         0          0
Gi2/15                               0         0         0         0          0
Gi2/16                               0         0         0         0          0
Gi2/17                               0         0         0         0          0
Gi2/18                               0         0         0         0          0
Gi2/19                               0         0         0         0          0
Gi2/20                               0         0         0         0          0
Gi2/21                               0         0         0         0          0
Gi2/22                               0         0         0         0          0
Gi2/23                               0         0         0         0          0
Gi2/24                               0         0         0         0          0
Gi2/25                               0         0         0         0          0
Gi2/26                               0         0         0         0          0
Gi2/27                               0         0         0         0          0
Gi2/28                               0         0         0         0          0
Gi2/29                               0         0         0         0          0
Gi2/30                               0         0         0         0          0
Gi2/31                               0         0         0         0          0
Gi2/32                               0         0         0         0          0
Gi2/33                               0         0         0         0          0
Gi2/34                               0         0         0         0          0
Gi2/35                               0         0         0         0          0
Gi2/36                               0         0         0         0          0
Gi2/37                               0         0         0         0          0
Gi2/38                               0         0         0         0          0
Gi2/39                               0         0         0         0          0
Gi2/40                               0         0         0         0          0
Gi2/41                               0         0         0         0          0
Gi2/42                               0         0         0         0          0
Gi2/43                               0         0         0         0          0
Gi2/44                               0         0         0         0          0
Gi2/45                               0         0         0         0          0
Gi2/46                               0         0         0         0          0
Gi2/47                               0         0         0         0          0
Gi2/48                               0         0         0         0          0
Gi5/1                                0         0         0         0          0
Gi5/2                                0         0         0         0          0
Gi5/3                                0         0         0         0          0
Gi5/4                                0         0         0         0          0
Gi5/5                                0         0         0         0          0
Gi5/6                                0         0         0         0          0
Gi5/7                                0         0         0         0          0
Gi5/8                                0         0         0         0          0
Gi5/9                                0         0         0         0          0
Gi5/10                               0         0         0         0          0
Gi5/11                               0         0         0         0          0
Gi5/12                               0         0         0         0          0
Gi5/13                               0         0         0         0          0
Gi5/14                               0         0         0         0          0
Gi5/15                               0         0         0         0          0
Gi5/16                               0         0         0         0          0
Gi5/17                               0         0         0         0          0
Gi5/18                               0         0         0         0          0
Gi5/19                               0         0         0         0          0
Gi5/20                               0         0         0         0          0
Gi5/21                               0         0         0         0          0
Gi5/22                               0         0         0         0          0
Gi5/23                               0         0         0         0          0
Gi5/24                               0         0         0         0          0
Gi5/25                               0         0         0         0          0
Gi5/26                               0         0         0         0          0
Gi5/27                               0         0         0         0          0
Gi5/28                               0         0         0         0          0
Gi5/29                               0         0         0         0          0
Gi5/30                               0         0         0         0          0
Gi5/31                               0         0         0         0          0
Gi5/32                               0         0         0         0          0
Gi5/33                               0         0         0         0          0
Gi5/34                               0         0         0         0          0
Gi5/35                               0         0         0         0          0
Gi5/36                               0         0         0         0          0
Gi5/37                               0         0         0         0          0
Gi5/38                               0         0         0         0          0
Gi5/39                               0         0         0         0          0
Gi5/40                               0         0         0         0          0
Gi5/41                               0         0         0         0          0
Gi5/42                               0         0         0         0          0
Gi5/43                               0         0         0         0          0
Gi5/44                               0         0         0         0          0
Gi5/45                               0         0         0         0          0
Gi5/46                               0         0         0         0          0
Gi5/47                               0         0         0         0          0
Gi5/48                               0         0         0         0          0
Gi6/1                                0         0         0         0          0
Gi6/2                                0         0         0         0          0
Gi6/3                                0         0         0         0          0
Gi6/4                                0         0         0         0          0
Gi6/5                                0         0         0         0          0
Gi6/6                                0         0         0         0          0
Gi6/7                                0         0         0         0          0
Gi6/8                                0         0         0         0          0
Gi6/9                                0         0         0         0          0
Gi6/10                               0         0         0         0          0
Gi6/11                               0         0         0         0          0
Gi6/12                               0         0         0         0          0
Gi6/13                               0         0         0         0          0
Gi6/14                               0         0         0         0          0
Gi6/15                               0         0         0         0          0
Gi6/16                               0         0         0         0          0
Gi6/17                               0         0         0         0          0
Gi6/18                               0         0         0         0          0
Gi6/19                               0         0         0         0          0
Gi6/20                               0         0         0         0          0
Gi6/21                               0         0         0         0          0
Gi6/22                               0         0         0         0          0
Gi6/23                               0         0         0         0          0
Gi6/24                               0         0         0         0          0
Gi6/25                               0         0         0         0          0
Gi6/26                               0         0         0         0          0
Gi6/27                               0         0         0         0          0
Gi6/28                               0         0         0         0          0
Gi6/29                               0         0         0         0          0
Gi6/30                               0         0         0         0          0
Gi6/31                               0         0         0         0          0
Gi6/32                               0         0         0         0          0
Gi6/33                               0         0         0         0          0
Gi6/34                               0         0         0         0          0
Gi6/35                               0         0         0         0          0
Gi6/36                               0         0         0         0          0
Gi6/37                               0         0         0         0          0
Gi6/38                               0         0         0         0          0
Gi6/39                               0         0         0         0          0
Gi6/40                               0         0         0         0          0
Gi6/41                               0         0         0         0          0
Gi6/42                               0         0         0         0          0
Gi6/43                               0         0         0         0          0
Gi6/44                               0         0         0         0          0
Gi6/45                               0         0         0         0          0
Gi6/46                               0         0         0         0          0
Gi6/47                               0         0         0         0          0
Gi6/48                               0         0         0         0          0
Gi7/1                                0         0         0         0          0
Gi7/2                                0         0         0         0          0
Gi7/3                                0         0         0         0          0
Gi7/4                                0         0         0         0          0
Gi7/5                                0         0         0         0          0
Gi7/6                                0         0         0         0          0
Gi7/7                                0         0         0         0          0
Gi7/8                                0         0         0         0          0
Gi7/9                                0         0         0         0          0
Gi7/10                               0         0         0         0          0
Gi7/11                               0         0         0         0          0
Gi7/12                               0         0         0         0          0
Gi7/13                               0         0         0         0          0
Gi7/14                               0         0         0         0          0
Gi7/15                               0         0         0         0          0
Gi7/16                               0         0         0         0          0
Gi7/17                               0         0         0         0          0
Gi7/18                               0         0         0         0          0
Gi7/19                               0         0         0         0          0
Gi7/20                               0         0         0         0          0
Gi7/21                               0         0         0         0          0
Gi7/22                               0         0         0         0          0
Gi7/23                               0         0         0         0          0
Gi7/24                               0         0         0         0          0
Gi7/25                               0         0         0         0          0
Gi7/26                               0         0         0         0          0
Gi7/27                               0         0         0         0          0
Gi7/28                               0         0         0         0          0
Gi7/29                               0         0         0         0          0
Gi7/30                               0         0         0         0          0
Gi7/31                               0         0         0         0          0
Gi7/32                               0         0         0         0          0
Gi7/33                               0         0         0         0          0
Gi7/34                               0         0         0         0          0
Gi7/35                               0         0         0         0          0
Gi7/36                               0         0         0         0          0
Gi7/37                               0         0         0         0          0
Gi7/38                               0         0         0         0          0
Gi7/39                               0         0         0         0          0
Gi7/40                               0         0         0         0          0
Gi7/41                               0         0         0         0          0
Gi7/42                               0         0         0         0          0
Gi7/43                               0         0         0         0          0
Gi7/44                               0         0         0         0          0
Gi7/45                               0         0         0         0          0
Gi7/46                               0         0         0         0          0
Gi7/47                               0         0         0         0          0
Gi7/48                               0         0         0         0          0
Te3/1                                0         0         0         0          0
Te3/2                                0         0         0         0          0
Gi3/3                                0         0         0         0          0
Gi3/4                                0         0         0         0          0
Gi3/5                                0         0         0         0          0
Gi3/6                                0         0         0         0          0
Te4/1                                0         0         0         0          0
Te4/2                                0         0         0         0          0
Gi4/3                                0         0         0         0          0
Gi4/4                                0         0         0         0          0
Gi4/5                                0         0         0         0          0
Gi4/6                                0         0         0         0          0

Packets Received at CPU per Input Interface

Interface              Total           5 sec avg 1 min avg 5 min avg 1 hour avg
---------------------- --------------- --------- --------- --------- ----------
Gi1/1                                0         0         0         0          0
Gi1/2                                0         0         0         0          0
Gi1/3                                0         0         0         0          0
Gi1/4                                0         0         0         0          0
Gi1/5                                0         0         0         0          0
Gi1/6                                0         0         0         0          0
Gi1/7                                0         0         0         0          0
Gi1/8                                0         0         0         0          0
Gi1/9                                0         0         0         0          0
Gi1/10                               0         0         0         0          0
Gi1/11                               0         0         0         0          0
Gi1/12                               0         0         0         0          0
Gi1/13                               0         0         0         0          0
Gi1/14                               0         0         0         0          0
Gi1/15                               0         0         0         0          0
Gi1/16                               0         0         0         0          0
Gi1/17                               0         0         0         0          0
Gi1/18                               0         0         0         0          0
Gi1/19                               0         0         0         0          0
Gi1/20                               0         0         0         0          0
Gi1/21                               0         0         0         0          0
Gi1/22                               0         0         0         0          0
Gi1/23                               0         0         0         0          0
Gi1/24                               0         0         0         0          0
Gi1/25                               0         0         0         0          0
Gi1/26                               0         0         0         0          0
Gi1/27                               0         0         0         0          0
Gi1/28                               0         0         0         0          0
Gi1/29                               0         0         0         0          0
Gi1/30                               0         0         0         0          0
Gi1/31                               0         0         0         0          0
Gi1/32                               0         0         0         0          0
Gi1/33                               0         0         0         0          0
Gi1/34                               0         0         0         0          0
Gi1/35                               0         0         0         0          0
Gi1/36                               0         0         0         0          0
Gi1/37                               0         0         0         0          0
Gi1/38                               0         0         0         0          0
Gi1/39                               0         0         0         0          0
Gi1/40                               0         0         0         0          0
Gi1/41                               0         0         0         0          0
Gi1/42                               0         0         0         0          0
Gi1/43                               0         0         0         0          0
Gi1/44                               0         0         0         0          0
Gi1/45                               0         0         0         0          0
Gi1/46                               0         0         0         0          0
Gi1/47                               0         0         0         0          0
Gi1/48                               0         0         0         0          0
Gi2/1                                0         0         0         0          0
Gi2/2                                0         0         0         0          0
Gi2/3                                0         0         0         0          0
Gi2/4                                0         0         0         0          0
Gi2/5                                0         0         0         0          0
Gi2/6                                0         0         0         0          0
Gi2/7                                0         0         0         0          0
Gi2/8                                0         0         0         0          0
Gi2/9                                0         0         0         0          0
Gi2/10                               0         0         0         0          0
Gi2/11                               0         0         0         0          0
Gi2/12                               0         0         0         0          0
Gi2/13                               0         0         0         0          0
Gi2/14                               0         0         0         0          0
Gi2/15                               0         0         0         0          0
Gi2/16                               0         0         0         0          0
Gi2/17                               0         0         0         0          0
Gi2/18                               0         0         0         0          0
Gi2/19                               0         0         0         0          0
Gi2/20                               0         0         0         0          0
Gi2/21                               0         0         0         0          0
Gi2/22                               0         0         0         0          0
Gi2/23                               0         0         0         0          0
Gi2/24                               0         0         0         0          0
Gi2/25                               0         0         0         0          0
Gi2/26                               0         0         0         0          0
Gi2/27                               0         0         0         0          0
Gi2/28                               0         0         0         0          0
Gi2/29                               0         0         0         0          0
Gi2/30                               0         0         0         0          0
Gi2/31                               0         0         0         0          0
Gi2/32                               0         0         0         0          0
Gi2/33                               0         0         0         0          0
Gi2/34                               0         0         0         0          0
Gi2/35                               0         0         0         0          0
Gi2/36                               0         0         0         0          0
Gi2/37                               0         0         0         0          0
Gi2/38                               0         0         0         0          0
Gi2/39                               0         0         0         0          0
Gi2/40                               0         0         0         0          0
Gi2/41                               0         0         0         0          0
Gi2/42                               0         0         0         0          0
Gi2/43                               0         0         0         0          0
Gi2/44                               0         0         0         0          0
Gi2/45                               0         0         0         0          0
Gi2/46                               0         0         0         0          0
Gi2/47                               0         0         0         0          0
Gi2/48                               0         0         0         0          0
Gi5/1                                0         0         0         0          0
Gi5/2                                0         0         0         0          0
Gi5/3                                0         0         0         0          0
Gi5/4                                0         0         0         0          0
Gi5/5                                0         0         0         0          0
Gi5/6                                0         0         0         0          0
Gi5/7                                0         0         0         0          0
Gi5/8                                0         0         0         0          0
Gi5/9                                0         0         0         0          0
Gi5/10                               0         0         0         0          0
Gi5/11                               0         0         0         0          0
Gi5/12                               0         0         0         0          0
Gi5/13                               0         0         0         0          0
Gi5/14                               0         0         0         0          0
Gi5/15                               0         0         0         0          0
Gi5/16                               0         0         0         0          0
Gi5/17                               0         0         0         0          0
Gi5/18                               0         0         0         0          0
Gi5/19                               0         0         0         0          0
Gi5/20                               0         0         0         0          0
Gi5/21                               0         0         0         0          0
Gi5/22                               0         0         0         0          0
Gi5/23                               0         0         0         0          0
Gi5/24                               0         0         0         0          0
Gi5/25                               0         0         0         0          0
Gi5/26                               0         0         0         0          0
Gi5/27                               0         0         0         0          0
Gi5/28                               0         0         0         0          0
Gi5/29                               0         0         0         0          0
Gi5/30                               0         0         0         0          0
Gi5/31                               0         0         0         0          0
Gi5/32                               0         0         0         0          0
Gi5/33                               0         0         0         0          0
Gi5/34                               0         0         0         0          0
Gi5/35                               0         0         0         0          0
Gi5/36                               0         0         0         0          0
Gi5/37                               0         0         0         0          0
Gi5/38                               0         0         0         0          0
Gi5/39                               0         0         0         0          0
Gi5/40                               0         0         0         0          0
Gi5/41                               0         0         0         0          0
Gi5/42                               0         0         0         0          0
Gi5/43                               0         0         0         0          0
Gi5/44                               0         0         0         0          0
Gi5/45                               0         0         0         0          0
Gi5/46                               0         0         0         0          0
Gi5/47                               0         0         0         0          0
Gi5/48                               0         0         0         0          0
Gi6/1                                0         0         0         0          0
Gi6/2                                0         0         0         0          0
Gi6/3                                0         0         0         0          0
Gi6/4                                0         0         0         0          0
Gi6/5                                0         0         0         0          0
Gi6/6                                0         0         0         0          0
Gi6/7                                0         0         0         0          0
Gi6/8                                0         0         0         0          0
Gi6/9                                0         0         0         0          0
Gi6/10                               0         0         0         0          0
Gi6/11                               0         0         0         0          0
Gi6/12                               0         0         0         0          0
Gi6/13                               0         0         0         0          0
Gi6/14                               0         0         0         0          0
Gi6/15                               0         0         0         0          0
Gi6/16                               0         0         0         0          0
Gi6/17                               0         0         0         0          0
Gi6/18                               0         0         0         0          0
Gi6/19                               0         0         0         0          0
Gi6/20                               0         0         0         0          0
Gi6/21                               0         0         0         0          0
Gi6/22                               0         0         0         0          0
Gi6/23                               0         0         0         0          0
Gi6/24                               0         0         0         0          0
Gi6/25                               0         0         0         0          0
Gi6/26                               0         0         0         0          0
Gi6/27                               0         0         0         0          0
Gi6/28                               0         0         0         0          0
Gi6/29                               0         0         0         0          0
Gi6/30                               0         0         0         0          0
Gi6/31                               0         0         0         0          0
Gi6/32                               0         0         0         0          0
Gi6/33                               0         0         0         0          0
Gi6/34                               0         0         0         0          0
Gi6/35                               0         0         0         0          0
Gi6/36                               0         0         0         0          0
Gi6/37                               0         0         0         0          0
Gi6/38                               0         0         0         0          0
Gi6/39                               0         0         0         0          0
Gi6/40                               0         0         0         0          0
Gi6/41                               0         0         0         0          0
Gi6/42                               0         0         0         0          0
Gi6/43                               0         0         0         0          0
Gi6/44                               0         0         0         0          0
Gi6/45                               0         0         0         0          0
Gi6/46                               0         0         0         0          0
Gi6/47                               0         0         0         0          0
Gi6/48                               0         0         0         0          0
Gi7/1                                0         0         0         0          0
Gi7/2                                0         0         0         0          0
Gi7/3                                0         0         0         0          0
Gi7/4                                0         0         0         0          0
Gi7/5                                0         0         0         0          0
Gi7/6                                0         0         0         0          0
Gi7/7                                0         0         0         0          0
Gi7/8                                0         0         0         0          0
Gi7/9                                0         0         0         0          0
Gi7/10                               0         0         0         0          0
Gi7/11                               0         0         0         0          0
Gi7/12                               0         0         0         0          0
Gi7/13                               0         0         0         0          0
Gi7/14                               0         0         0         0          0
Gi7/15                               0         0         0         0          0
Gi7/16                               0         0         0         0          0
Gi7/17                               0         0         0         0          0
Gi7/18                               0         0         0         0          0
Gi7/19                               0         0         0         0          0
Gi7/20                               0         0         0         0          0
Gi7/21                               0         0         0         0          0
Gi7/22                               0         0         0         0          0
Gi7/23                               0         0         0         0          0
Gi7/24                               0         0         0         0          0
Gi7/25                               0         0         0         0          0
Gi7/26                               0         0         0         0          0
Gi7/27                               0         0         0         0          0
Gi7/28                               0         0         0         0          0
Gi7/29                               0         0         0         0          0
Gi7/30                               0         0         0         0          0
Gi7/31                               0         0         0         0          0
Gi7/32                               0         0         0         0          0
Gi7/33                               0         0         0         0          0
Gi7/34                               0         0         0         0          0
Gi7/35                               0         0         0         0          0
Gi7/36                               0         0         0         0          0
Gi7/37                               0         0         0         0          0
Gi7/38                               0         0         0         0          0
Gi7/39                               0         0         0         0          0
Gi7/40                               0         0         0         0          0
Gi7/41                               0         0         0         0          0
Gi7/42                               0         0         0         0          0
Gi7/43                               0         0         0         0          0
Gi7/44                               0         0         0         0          0
Gi7/45                               0         0         0         0          0
Gi7/46                               0         0         0         0          0
Gi7/47                               0         0         0         0          0
Gi7/48                               0         0         0         0          0
Te3/1                                0         0         0         0          0
Te3/2                                0         0         0         0          0
Gi3/3                                0         0         0         0          0
Gi3/4                                0         0         0         0          0
Gi3/5                                0         0         0         0          0
Gi3/6                                0         0         0         0          0
Te4/1                                0         0         0         0          0
Te4/2                                0         0         0         0          0
Gi4/3                                0         0         0         0          0
Gi4/4                                0         0         0         0          0
Gi4/5                                0         0         0         0          0
Gi4/6                                0         0         0         0          0

3 Replies 3

Dale Miller
Cisco Employee
Cisco Employee

From the outputs you provided I suspect packets are being software processed Input Acl punted packets are being dropped. These could be control packets like IGMP, OSPF etc. Or there is a ACL defined that is logging. If the problem is happening now can you collect  show platform health and show platform cpu packet statistics?

If it is traffic being punted to the CPU try using the following debug: (built-in-sniffer)

     - debug platform packet all receive buffer (safe to use in high CPU condition that is what it is built for)

     - show platform CPU packet buffered to see what packets are being punted.

The output will provide src / dst  and rx interface, etc...

Here is a whitepaper on TS high CPU on the 4500. Refer to the built in sniffer tool for more details.

http://www.cisco.com/en/US/products/hw/switches/ps663/products_tech_note09186a00804cef15.shtml


Regards,

Dale

Thanks for the reply Dale. I have few further queries.

Can you let me about this output from show plat cpu packet stat  all.Apart from Input Acl Fwd event, i can see event Sa Miss and Packets  Dropped In Processing by reasons NoDstPorts and Tx Mode Drop

.

.

.

Packets Dropped In Processing by CPU event

Event             Total                5 sec avg 1 min avg 5 min avg 1 hour avg
----------------- -------------------- --------- --------- --------- ----------
Unknown                              0         0         0         0          0
Sa Miss                    24702624823        11         1         2          0

.

.

.

.

Packets Dropped In Processing by Reason

Reason             Total                5 sec avg 1 min avg 5 min avg 1 hour avg
------------------ -------------------- --------- --------- --------- ----------
NoDstPorts                    582487025         4         0         0          2

Tx Mode Drop                24702624802        11         1         2          0

.

.

.

Here is the output of show plat health

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Cat4500-sw1#  sh platform health
                      %CPU   %CPU    RunTimeMax   Priority  Average %CPU  Total
                      Target Actual Target Actual   Fg   Bg 5Sec Min Hour  CPU
GalChassisVp-review    3.00   0.14     10     45  100  500    0   0    0  1504:04
S2w-JobEventSchedule  10.00   0.00     10      0  100  500    0   0    0  0:00
Stub-JobEventSchedul  10.00   0.00     10      0  100  500    0   0    0  0:00
Lj-poll                1.00   0.02      2      0  100  500    0   0    0  267:57
StatValueMan Update    1.00   0.08      1      0  100  500    0   0    0  753:25
Pim-review             0.10   0.00      1      0  100  500    0   0    0  66:01
Ebm-host-review        1.00   0.00      8      7  100  500    0   0    0  3408:27
Ebm-host-util-review   1.00   0.00     10      0  100  500    0   0    0  2:13
Ebm-port-review        0.10   0.00      1      0  100  500    0   0    0  5:35
Protocol-aging-revie   0.20   0.00      2      0  100  500    0   0    0  0:18
EbmHostRedundancyMan   2.00   0.00     20      0  100  500    0   0    0  0:00
Acl-Flattener          1.00   0.00     10      0  100  500    0   0    0  0:00
IrmFibThrottler Thro   2.00   0.00      7      0  100  500    0   0    0  90:02
GalChassisVp Ondeman   2.00   0.00      2      0  100  500    0   0    0  0:00
KxAclPathMan create/   1.00   0.00     10      5  100  500    0   0    0  10:20
KxAclPathMan update    2.00   0.00     10      0  100  500    0   0    0  6:44
KxAclPathMan reprogr   1.00   0.00      2      0  100  500    0   0    0  0:00
GalK5TatooineStatsMa   0.70   0.01      4      0  100  500    0   0    0  274:11
MOL FastDropReview     2.00   0.00     15      0  100  500    0   0    0  0:00
RkiosL3PortMan AclFe   2.00   0.00     15      1  100  500    0   0    0  0:00
K5L3FlcMan FwdEntry    2.00   0.00     15      2  100  500    0   0    0  0:00
FwdEntry Zombie Revi   2.00   0.00     15      0  100  500    0   0    0  0:00
K5L3FlcMan Cam Shuff   4.00   0.00     25      0  100  500    0   0    0  0:00
K5L3Unciast IFE Revi   2.00   0.00     15      0  100  500    0   0    0  0:00
K5L3UnicastRpf IFE R   2.00   0.00     15      0  100  500    0   0    0  0:00
K5L3Unicast Fwd Entr   2.00   0.00     15      0  100  500    0   0    0  0:00
K5L3McastMan IrmMfib   2.00   0.00     15      0  100  500    0   0    0  0:00
K5L3McastMan ImeSync   2.00   0.00     15      0  100  500    0   0    0  3:42
K5L3Unicast Fwd Entr   2.00   0.00     15      0  100  500    0   0    0  0:00
K5L3Unicast Adj Grou   2.00   0.00     15      0  100  500    0   0    0  0:00
K5L3Unicast Adj Chan   2.00   0.00     15      0  100  500    0   0    0  0:00
K5L3Unicast Adj Tabl   2.00   8.38     15      8  100  500    9  10    7  68125:26
K5L3AdjStatsMan Revi   2.00   0.49     10      7  100  500    0   0    0  2929:28
K5L3McastMan RetSync   1.00   0.00      8      0  100  500    0   0    0  0:00
K5FlcHitMan review     2.00   0.02      5      4  100  500    0   0    0  1876:42
K5PortMan Regular Re   2.00   0.20     15     12  100  500    0   0    0  4508:03
K5PortMan Ondemand L   3.00   0.51     30      4  100  500    0   0    0  4894:51
K5PortMan Stats Revi   2.00   0.07     15      5  100  500    0   0    0  503:17
K5PortMan Tx Queue R   3.00   0.00     15      0  100  500    0   0    0  0:00
K5L2 Vlan Table Revi   2.00   0.00     12      8  100  500    0   0    0  1853:53
K5 L2 Aging Table Re   2.00   0.04     20      6  100  500    0   0    0  1594:39
K5 L2 Unicast Addres   2.00   0.00     10      5  100  500    0   0    0  16:46
K5 L2 Multicast Addr   2.00   0.00     10      0  100  500    0   0    0  17:13
K5 L2 Hardware Addre   2.00   0.00     10      8  100  500    0   1    1  12840:29
K5RetStatsMan Review   2.00   0.00      5      0  100  500    0   0    0  0:00
K5CpuMan Review       30.00   7.64     30   2001  100  500   15  20   34  530517:33
K5ForerunnerPacketMa   1.50   0.23      4      4  100  500    1   1    0  4169:38
K5ForerunnerPacketMa   0.70   0.28      4      4  100  500    0   0    0  2864:58
K5QosDhmMan Rate DBL   2.00   0.00      7      0  100  500    0   0    0  0:00
K5QosPolicerStatsMan   1.00   0.00     10      0  100  500    0   0    0  54:12
K5VlanStatsReview      2.00   0.51     10      6  100  500    0   0    0  7990:12
K5AclMan-labeledFlat   1.00   0.00     10      0  100  500    0   0    0  0:00
K5AclLabelMan-punted   1.00   0.00     10      0  100  500    0   0    0  6:52
K5AclCamMan stale en   1.00   0.00     10      0  100  500    0   0    0  0:00
K5AclCamStatsMan hw    3.00   0.10     10      5  100  500    0   0    0  4412:49
VSI DMA Local Jawa R   1.00   0.05      6      4  100  500    0   0    0  607:22
VSI DMA Remote Jawa    1.00   0.06      6      0  100  500    0   0    0  605:10
VSI DMA Darkside Rev   1.00   0.00      6      0  100  500    0   0    0  0:00
VSI DMA Slot-01 Revi   1.00   0.26      6      4  100  500    0   0    0  2901:13
VSI DMA Slot-02 Revi   1.00   0.26      6      4  100  500    0   0    0  2895:43
VSI DMA Slot-05 Revi   1.00   0.27      6      4  100  500    0   0    0  2948:02
VSI DMA Slot-06 Revi   1.00   0.25      6      4  100  500    0   0    0  2957:10
VSI DMA Slot-07 Revi   1.00   0.26      6      4  100  500    0   0    0  2905:59
GbmHostTable Review    2.00   0.00      8      0  100  500    0   0    0  17:56
GbmHostTableUtilizat   2.00   0.00      8      0  100  500    0   0    0  0:00
RkiosPortMan Port Re   2.00   0.17     12      6  100  500    0   0    0  3454:12
Rkios Module State R   4.00   0.02     40      2  100  500    0   0    0  283:17
Rkios Online Diag Re   4.00   0.02     40      0  100  500    0   0    0  276:13
RkiosIpPbr IrmPort R   2.00   0.00     10      5  100  500    0   0    0  368:38
RkiosIpWccp IrmWccpW   2.00   0.00     10      0  100  500    0   0    0  53:44
RkiosAclMan Review     3.00   0.03     30      0  100  500    0   0    0  535:53
MatMan Review          0.50   0.00      4      0  100  500    0   0    0  0:00
LocalJawaVsiMan VsiR   0.20   0.00      2      0  100  500    0   0    0  0:00
QuackMan( 254 ) Revi   4.00   0.00     20      4  100  500    0   0    0  0:02
GlmBridgeMan(2) revi   0.50   0.01      2      0  100  500    0   0    0  106:46
Stub periodic global   0.50   0.00      5      0  100  500    0   0    0  2:20
Stub ondemand global   0.50   0.00      5      0  100  500    0   0    0  0:00
Xgstub Stats Review    0.50   0.09      5      4  100  500    0   0    0  1059:45
EpmPortGroup(2) stat   0.50   0.01      2      0  100  500    0   0    0  95:41
EpmPortGroup(2) PimE   0.50   0.00      4      0  100  500    0   0    0  0:00
EpmPortGroup(2) on d   0.50   0.00      4      8  100  500    0   0    0  0:00
X2Controller(2)        0.50   0.00      0      0  100  500    0   0    0  9:22
PluggableLinecard(2)   0.50   0.00      1      0  100  500    0   0    0  0:00
PluggableLinecard(2)   0.50   0.00      1      0  100  500    0   0    0  0:00
EpmPluggableGroup(2)   0.50   0.00      2      0  100  500    0   0    0  4:44
QuackMan( 2 ) Review   4.00   0.00     20      0  100  500    0   0    0  0:00
GlmBridgeMan(0) revi   0.50   0.00      2      0  100  500    0   0    0  108:09
GlmBridgeMan(1) revi   0.50   0.00      2      4  100  500    0   0    0  108:17
RemoteJawaVsiMan Vsi   0.20   0.00      2      0  100  500    0   0    0  0:00
Stub periodic global   0.50   0.00      5      0  100  500    0   0    0  2:24
Stub ondemand global   0.50   0.00      5      0  100  500    0   0    0  0:10
Xgstub Stats Review    0.50   0.11      5      4  100  500    0   0    0  1201:52
EthPhyControllerMan(   0.40   0.00      4      0  100  500    0   0    0  17:49
EthPhyControllerMan(   0.20   0.00      2      2  100  500    0   0    0  8:36
EthPoeControllerMan(   0.20   0.00      2      0  100  500    0   0    0  4:47
EthPoeControllerMan(   0.20   0.00      2      0  100  500    0   0    0  5:28
EpmPortGroup(0) stat   0.50   0.00      2      4  100  500    0   0    0  215:40
EpmPortGroup(0) PimE   0.50   0.00      4      0  100  500    0   0    0  0:00
EpmPortGroup(0) on d   0.50   0.00      4      8  100  500    0   0    0  0:03
GlmBridgeMan(3) revi   0.50   0.01      2      0  100  500    0   0    0  106:54
Stub periodic global   0.50   0.00      5      0  100  500    0   0    0  2:24
Stub ondemand global   0.50   0.00      5      0  100  500    0   0    0  0:17
Xgstub Stats Review    0.50   0.11      5      4  100  500    0   0    0  1201:10
EthPhyControllerMan(   0.40   0.00      4      4  100  500    0   0    0  17:40
EthPhyControllerMan(   0.20   0.00      2      2  100  500    0   0    0  8:24
EthPoeControllerMan(   0.20   0.00      2      0  100  500    0   0    0  4:47
EthPoeControllerMan(   0.20   0.00      2      0  100  500    0   0    0  5:29
EpmPortGroup(1) stat   0.50   0.03      2      4  100  500    0   0    0  212:10
EpmPortGroup(1) PimE   0.50   0.00      4      0  100  500    0   0    0  0:00
EpmPortGroup(1) on d   0.50   0.00      4      4  100  500    0   0    0  0:06
GlmBridgeMan(4) revi   0.50   0.00      2      0  100  500    0   0    0  111:48
GlmBridgeMan(5) revi   0.50   0.00      2      0  100  500    0   0    0  112:33
Stub periodic global   0.50   0.00      5      0  100  500    0   0    0  2:21
Stub ondemand global   0.50   0.00      5      0  100  500    0   0    0  0:00
Xgstub Stats Review    0.50   0.09      5      0  100  500    0   0    0  1058:55
EpmPortGroup(3) stat   0.50   0.00      2      4  100  500    0   0    0  95:40
EpmPortGroup(3) PimE   0.50   0.00      4      0  100  500    0   0    0  0:00
EpmPortGroup(3) on d   0.50   0.00      4      6  100  500    0   0    0  0:00
GlmBridgeMan(6) revi   0.50   0.00      2      4  100  500    0   0    0  108:02
Stub periodic global   0.50   0.00      5      0  100  500    0   0    0  2:24
Stub ondemand global   0.50   0.00      5      4  100  500    0   0    0  38:16
Xgstub Stats Review    0.50   0.11      5      4  100  500    0   0    0  1201:17
EthPhyControllerMan(   0.40   0.00      4      1  100  500    0   0    0  19:34
EthPhyControllerMan(   0.20   0.00      2      2  100  500    0   0    0  15:26
EthPoeControllerMan(   0.20   0.00      2      0  100  500    0   0    0  4:46
EthPoeControllerMan(   0.20   0.00      2      0  100  500    0   0    0  5:29
EpmPortGroup(4) stat   0.50   0.00      2      0  100  500    0   0    0  100:24
EpmPortGroup(4) PimE   0.50   0.00      4      0  100  500    0   0    0  0:00
EpmPortGroup(4) on d   0.50   0.00      4      3  100  500    0   0    0  8:09
Stub periodic global   0.50   0.00      5      0  100  500    0   0    0  2:24
Stub ondemand global   0.50   0.00      5      0  100  500    0   0    0  43:03
Xgstub Stats Review    0.50   0.11      5      4  100  500    0   0    0  1200:22
EthPhyControllerMan(   0.40   0.00      4      0  100  500    0   0    0  18:29
EthPhyControllerMan(   0.20   0.00      2      2  100  500    0   0    0  16:07
EthPoeControllerMan(   0.20   0.00      2      0  100  500    0   0    0  4:48
EthPoeControllerMan(   0.20   0.00      2      0  100  500    0   0    0  5:31
EpmPortGroup(5) stat   0.50   0.01      2      4  100  500    0   0    0  41:18
EpmPortGroup(5) PimE   0.50   0.00      4      0  100  500    0   0    0  0:00
EpmPortGroup(5) on d   0.50   0.00      4      3  100  500    0   0    0  10:17
Stub periodic global   0.50   0.00      5      0  100  500    0   0    0  2:24
Stub ondemand global   0.50   0.00      5      0  100  500    0   0    0  0:00
Xgstub Stats Review    0.50   0.11      5      4  100  500    0   0    0  1200:34
EthPhyControllerMan(   0.40   0.00      4      1  100  500    0   0    0  18:48
EthPhyControllerMan(   0.20   0.00      2      2  100  500    0   0    0  7:40
EthPoeControllerMan(   0.20   0.00      2      0  100  500    0   0    0  4:47
EthPoeControllerMan(   0.20   0.00      2      0  100  500    0   0    0  5:29
EpmPortGroup(6) stat   0.50   0.00      2      0  100  500    0   0    0  27:28
EpmPortGroup(6) PimE   0.50   0.00      4      0  100  500    0   0    0  0:00
EpmPortGroup(6) on d   0.50   0.00      4      0  100  500    0   0    0  0:00
X2Controller(3)        0.50   0.00      0      0  100  500    0   0    0  9:20
PluggableLinecard(3)   0.50   0.00      1      0  100  500    0   0    0  0:00
PluggableLinecard(3)   0.50   0.00      1      0  100  500    0   0    0  0:00
EpmPluggableGroup(3)   0.50   0.00      2      0  100  500    0   0    0  4:35
QuackMan( 3 ) Review   4.00   0.00     20      0  100  500    0   0    0  0:00
QuackMan( 0 ) Review   4.00   0.00     20      0  100  500    0   0    0  0:00
QuackMan( 1 ) Review   4.00   0.00     20      0  100  500    0   0    0  0:00
QuackMan( 4 ) Review   4.00   0.00     20      0  100  500    0   0    0  0:01
QuackMan( 5 ) Review   4.00   0.00     20      0  100  500    0   0    0  0:01
QuackMan( 6 ) Review   4.00   0.00     20      0  100  500    0   0    0  0:01
                      -------------
%CPU Totals          234.69  21.45

                       Allocation ceiling        Current allocation
                        ------------------        ------------------
                        kbytes    % in use        kbytes    % in use

Linecard 1's Store        600.00      99%          596.36      100%
Linecard 2's Store        600.00      99%          596.36      100%
Linecard 3's Store        600.00      34%          205.72      100%
Linecard 4's Store        600.00      34%          205.72      100%
Linecard 5's Store        600.00      99%          596.36      100%
Linecard 6's Store        600.00      99%          596.36      100%
Linecard 7's Store        600.00      99%          596.36      100%
TSM objects            ------------------        ------------------
PacketInfoItem            859.37       0%            4.29        0%
VbufNodes2400              80.50       0%            0.00        0%
VbufNodes1600              55.50       0%            3.46        0%
VbufNodes400              288.00       1%            4.50       87%
VbufNodes64                60.00       0%            0.46        0%
VbufNodes4200              68.37      43%           29.91      100%
PacketBufRaw              184.29     100%          184.29      100%
PacketBufRaw             4644.56     100%         4644.56      100%
RkiosSysPacketBuf         250.00       0%            1.12        0%
Packet                   2322.42       2%           54.73       98%
IndexCache                800.39       0%            0.00        0%
K5AclOpDescNode         10752.00       0%            0.00        0%
K5AclLabelListNode        768.00       0%            0.00        0%
K5AclIpv6PackedAddrH     1024.00       0%            0.00        0%
K5AdjGroups               960.00       0%            0.13      100%
K5L3FlcEntryAvlTree2    12902.39       0%            0.46      100%
IrmFibUnicastRpfList     5120.00       0%            0.15      100%
K5L3FwdEntrys           29120.00       0%            1.09      100%
K5L3FwdEntryAvlTree2    12480.00       0%            0.37      100%
K5L3FwdTreeEntrys       21840.00       0%            1.31      100%
K5L3FwdTreeEntryAvlT    49920.00       0%            0.23      100%
IrmMfibFastDropFlowM      576.00       0%            0.00        0%
K5QosTxQueSelTableBl       14.00       2%            0.38      100%
K5QosPolicerBlockNod        2.00       0%            0.00        0%
K5QosPolicerBlockMem       69.00       0%            0.00        0%
K5QosPolicerMemAlloc      448.00       0%            0.00        0%
K5QosFeatureInfoList     1536.00       0%            0.00        0%
K5QosLabelToFeatureE     1792.00       0%            0.00        0%
K5QosPathFeatureInfo      384.00       0%            0.00        0%
K5CpuPacketInfoItem       859.37       0%            0.00        0%
K5TxPacketInfo            384.00       0%            2.36        0%
MatEntrys                7680.00       0%            0.46      100%
MatEntryTableIterato        1.12       0%            0.03        0%
RkiosAclMan NamedGal       43.75       0%            0.00        0%
LpipAceListNode           192.00       0%            0.00        0%
Rkios Acl VlanMaps        128.00       0%            0.00        0%
Rkios Acl VlanMapEnt      859.37       0%            0.00        0%
Rkios EVC EFPs           1280.00       0%            0.00        0%
Rkios GBM BridgeDoma      256.00       0%            0.00        0%
Rkios GBM PseudoPort      768.00       0%            0.00        0%
Rkios QoS PolicyMaps      356.32       0%            0.00        0%
Rkios QoS ClassMaps       896.00       0%            0.02      100%
AclToIosFilterMapLis      384.00       0%            0.00        0%
Rkios QoS Policers        456.00       0%            0.00        0%
RkiosTableMap Galios        3.00       0%            0.00        0%
PimPhyports              1000.00      25%          257.00      100%
PimPorts                  890.62      32%          285.89      100%
PimModules                156.00       2%            4.26      100%
PimSlots                    5.00       2%            0.13      100%
PimChassis                 38.37       6%            2.39      100%
PimQuack                    0.35      26%            0.09      100%
EbmVlans                 4832.00       0%           35.39      100%
EbmVlanGroupEntrys       1920.00       0%            0.35       33%
EbmPorts                  240.00      30%           74.29      100%
EbmPortHostEntrys         384.00       0%            0.00        0%
EbmIeNodes                536.00       0%            4.71      100%
EbmPortVlanAclFeatur     1536.00       0%            0.00        0%
EbmSortedHostTableIt        1.87       0%            0.05        0%
EbmSortedGroupTableI        1.75       0%            0.05        0%
EbmHostRedundancyMan     1082.81       0%            0.00        0%
IrmVrfs                   317.00       0%            1.23      100%
IrmFibLoadBalances       1152.00       0%            0.00        0%
IrmFibAdjs               1920.00       0%            0.50      100%
IrmPortEtherAddrEntr      500.00       0%            0.00        0%
IrmFibEntries           12288.00       0%            0.51      100%
IrmMfibEntryMemMan      13312.00       0%            0.10      100%
IrmWccpMemMan              34.06       0%            0.00        0%
AclOp                    2048.00       0%            0.00        0%
AclOpAceSet              2559.96       0%            0.00        0%
AclClassifier            1280.00       0%            1.17      100%
AclFeature               2570.04       0%            1.63      100%
Acl                      1408.00       0%            1.28      100%
Ace24                    9215.85       0%            3.58      100%
Ace48                   15359.76       0%            0.11      100%
AceActionDescStorage     1023.98       0%            0.00        0%
AclListNode               256.00       0%            0.23      100%
AceListNode             25600.00       0%            0.03        0%
AclClassifierActionL     1152.00       0%            0.91      100%
AclLayerFeatureListN      512.00       0%            0.00        0%
AclClassifierListNod      256.00       0%            0.00        0%
TableMapMan NameToTa       27.00       0%            0.00        0%
TableMapAllocator         116.00       0%            0.00        0%
FlatAcl                   512.00       0%            0.00        0%
FlatAce24               10752.00       0%            0.00        0%
FlatAce48               16896.00       0%            0.00        0%
FlatAceActionListNod   256000.00       0%            0.07        0%
FlatAclOpSetStorage      3072.00       0%            0.00        0%
FlatAclCacheNode         4224.00       0%            0.00        0%
FlatAclListNode           256.00       0%            0.00        0%
QosFeatureClassifier      135.26       0%            0.00        0%
QosFeatureClassifier     2367.14       0%            0.00        0%
QosClassifierActionL      896.00       0%            0.00        0%
QosPortVlanAclFeatur     1152.00       0%            0.00        0%
QosEvcEfpAllocator       4352.00       0%            0.00        0%
QoS Policers             5624.00       0%            0.00        0%
MOL EVC EFPs             9728.00       0%            0.00        0%
EvcEfpEncapVlanRange      512.00       0%            0.00        0%
MOL GBM BridgeDomain     2048.00       0%            0.00        0%
MOL GBM PseudoPort       5376.00       0%            0.00        0%
GbmSortedHostTableIt        1.75       0%            0.00        0%
GbmBridgeDomainHostE    12288.00       0%            0.00        0%
KxAclPath                1024.00       0%            0.00        0%
KxAclPathListNode         256.00       0%            0.00        0%
KxAclConstPathListNo      256.00       0%            0.00        0%
EpmPortGroups               0.81      87%            0.71      100%
EpmPortMans                61.27      99%           61.03      100%
KxAclLabeledFlatAcl      3968.00       0%            0.00        0%
KxAclLabeledFlatAclE     3072.00       0%            0.00        0%
EbmVlanHostEntrys        3437.50       9%          384.12       83%
VsiBuffers3072            300.00       0%            3.00        0%
VsiBuffers1024            750.00      64%          486.00      100%
VsiBuffers128             250.00       0%           21.50        0%
VsiBuffers16               31.25       3%           22.00        4%
VsiTransactions(Max        82.03       4%            8.32       40%
VsiTransactions(Max       106.64      35%           88.96       42%
VsiTransactions(Max        15.42       0%            0.18        0%
VsiTransactions(Max         5.90       0%            0.00        0%
VsiTransactions(Max         4.83       0%            0.00        0%
VsiTransactionRespon       11.89      18%            4.88       44%
VsiS2wRequestMans          54.68       0%            5.07        0%
VsiVliRequestMans         147.65      26%           50.83       75%
VsiMdio22RequestMans      273.43       0%          105.00        0%
VsiMdio45RequestMans       27.34       0%            0.15        0%
VsiJtagRequestMans          4.92       0%            0.00        0%
GalGbicEntrys              11.85       0%            0.09      100%
EbmVlanBroadcastFloo     1408.00       0%           10.31      100%
Event Nodes               128.00       0%            0.00        0%
Event Nodes               128.00       0%            0.21        0%
RkisoIpPbrRouteMaps        97.65       0%            0.00        0%
Rkios EVC EFPs            768.00       0%            0.00        0%
K5AclLabelSignatureM    11648.00       0%            0.00        0%
K5AclLabelMapEntryPa     1408.00       0%            0.00        0%
TableMapMan NameToTa       27.00       0%            0.00        0%
TableMapAllocator         116.00       0%            0.00        0%
InpTosMarkTbl BlockA        1.75      12%            0.21      100%
InpCosMarkTbl BlockA        1.75      12%            0.21      100%
InpExpMarkTbl BlockA        1.75      12%            0.21      100%
OutTosMarkTbl BlockA        1.75      12%            0.21      100%
OutCosMarkTbl BlockA        1.75      12%            0.21      100%
OutExpMarkTbl BlockA        1.75      12%            0.21      100%
CommandTables              48.00      14%            6.75      100%
                        ------------------        ------------------
TSM totals             645533.02       1%         6870.98       95%

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Sachin,

The SA Miss is a packet flooded in the vlan and a copy sent the CPU for mac learning. The Tx mode drop (almost exact match to the amount of drops for SA MIss ) is that same packet getting dropped. Since the packet was already bridged by hardware, software drops the packet after doing source learning. The NoDstPorts are packets that cannot be forwarded. Example TTL=1 or hairpinned traffic that cannot be forwarded out the same interface it was received.

I can see from the outputs you're running the command when the CPU is normal. These outputs will help determine the root cause while the problem is happening. From the information so far I think it is best you get a idea of what traffic is punted to the CPU when running high. The debug packet all recieve buffer would be the ideal tool to use. You can also do an inband span and monitor all traffic to the CPU.

Once you identify what traffic is causing the problem it will be much easier to identifiy the root cause.  Right now there isn't enough information to tell.

Take Care,

Dale

Review Cisco Networking for a $25 gift card