cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
755
Views
0
Helpful
5
Replies

Four Switches with Stack 3750 10/1000 CPU usage 100 % simultaniously

Hi All,

sh processes cpu history, What is the Cause of this problem, becuase of this Network is getting down in entire network,

    6668966666756922212222124422212222222122222222222222122122

    1210938912210656383872837721491432223864124205354827445540

100     *        *                                           

90     *        *                                           

80    *#        *                                           

70    *# **  *  *                                           

60 ****#****** **                                           

50 ****#*********          **                               

40 ****#*********          **                               

30 ###########**#**   **   **            *      * * * *  *  

20 ##############**#**********************************# *****

10 ##########################################################

   0....5....1....1....2....2....3....3....4....4....5....5....

             0    5    0    5    0    5    0    5    0    5   

               CPU% per minute (last 60 minutes)

              * = maximum CPU%   # = average CPU%

    5343266623434556338343324238354342332343433333233282535256435437334344

    2518730354524518700109392450901216679522763520415847061734059563216436

100                                                                      

90                                                                      

80                   *        *                      *                  

70                *  *        *                      *            *     

60      ***     * *  *        *                      *      *  *  *     

50 *    ***  *  ***  *        * *          *         * * * **  ** *  *  *

40 **** ***  * ***** * **  * ***** * ** ** ** *    * * *** ********  * **

30 ************************* ******************** ***********************

20 **********************************************************************

10 ######################################################################

   0....5....1....1....2....2....3....3....4....4....5....5....6....6....7.

             0    5    0    5    0    5    0    5    0    5    0    5    0

                   CPU% per hour (last 72 hours)

                  * = maximum CPU%   # = average CPU%

5 Replies 5

acampbell
VIP Alumni
VIP Alumni

Aman,

There are many items that could cause this issue.

Like spanning tree loops etc

Try

show proc cpu sorted

This should output the league table of what is consuming the CPU,

with the greediest at the top.

Regards,
Alex.
Please rate useful posts.

Regards, Alex. Please rate useful posts.

Leo Laohoo
Hall of Fame
Hall of Fame

Would be nice if you post what IOS you are currently running. 

InayathUlla Sharieff
Cisco Employee
Cisco Employee

Hi Aman,

In addition please provide us the following outputs:

show version

show proc cpu sorted | ex 0.0

show proc cpu history

show controllers cpu-interfac

show platform port-asic stats drop

Regards,

Inayath

Please rate useful posts.

here is the out put of

show version

Cisco IOS Software, C3750 Software (C3750-IPSERVICESK9-M), Version 12.2(55)SE6, RELEASE SOFTWARE (fc1)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2012 by Cisco Systems, Inc.

Compiled Mon 23-Jul-12 13:22 by prod_rel_team

Image text-base: 0x01000000, data-base: 0x02F00000

ROM: Bootstrap program is C3750 boot loader

BOOTLDR: C3750 Boot Loader (C3750-HBOOT-M) Version 12.2(44)SE5, RELEASE SOFTWARE (fc1)

RDCCI37F0 uptime is 18 weeks, 3 days, 11 hours, 27 minutes

System returned to ROM by power-on

System restarted at 21:27:44 MSK Fri Sep 28 2012

System image file is "flash:c3750-ipservicesk9-mz.122-55.SE6/c3750-ipservicesk9-mz.122-55.SE6.bin"

This product contains cryptographic features and is subject to United

States and local country laws governing import, export, transfer and

use. Delivery of Cisco cryptographic products does not imply

third-party authority to import, export, distribute or use encryption.

Importers, exporters, distributors and users are responsible for

compliance with U.S. and local country laws. By using this product you

agree to comply with applicable laws and regulations. If you are unable

to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:

http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to

export@cisco.com.

cisco WS-C3750-48P (PowerPC405) processor (revision J0) with 131072K bytes of memory.

Processor board ID CAT1018R31J

Last reset from power-on

2 Virtual Ethernet interfaces

192 FastEthernet interfaces

16 Gigabit Ethernet interfaces

The password-recovery mechanism is enabled.

512K bytes of flash-simulated non-volatile configuration memory.

Base ethernet MAC Address       : 00:17:E0:67:2F:00

Motherboard assembly number     : 73-9675-11

Power supply part number        : 341-0029-04

Motherboard serial number       : CAT1017086B

Power supply serial number      : LIT1012032Y

Model revision number           : J0

Motherboard revision number     : A0

--More--                           Model number                    : WS-C3750-48PS-S

System serial number            : CAT1018R31J

SFP Module assembly part number : 73-7757-03

SFP Module revision Number      : A0

SFP Module serial number        : CAT10171KTE

Top Assembly Part Number        : 800-25858-03

Top Assembly Revision Number    : F0

Version ID                      : V05

CLEI Code Number                : COM1W00ARB

Hardware Board Revision Number  : 0x01

Switch Ports Model              SW Version            SW Image                

------ ----- -----              ----------            ----------              

     1 52    WS-C3750-48P       12.2(55)SE6           C3750-IPSERVICESK9-M    

     2 52    WS-C3750-48P       12.2(55)SE6           C3750-IPSERVICESK9-M    

*    3 52    WS-C3750-48P       12.2(55)SE6           C3750-IPSERVICESK9-M    

     4 52    WS-C3750-48P       12.2(55)SE6           C3750-IPSERVICESK9-M    

Switch 01

---------

Switch Uptime                   : 18 weeks, 3 days, 11 hours, 27 minutes

Base ethernet MAC Address       : 00:14:F2:00:FA:00

Motherboard assembly number     : 73-9675-17

Power supply part number        : 341-0029-05

Motherboard serial number       : CAT09260Y05

Power supply serial number      : DTN125140EL

Model revision number           : H0

Motherboard revision number     : A0

Model number                    : WS-C3750-48PS-S

System serial number            : CAT0926N2S0

SFP Module assembly part number : 73-7757-03

SFP Module revision number      : A0

SFP Module serial number        : CAT09241KM8

Top assembly part number        : 800-25858-08

Top assembly revision number    : A0

Version ID                      : V10

CLEI Code Number                : COMAH10BRA

Switch 02

---------

Switch Uptime                   : 18 weeks, 3 days, 11 hours, 27 minutes

Base ethernet MAC Address       : 00:17:E0:FB:4A:00

Motherboard assembly number     : 73-9675-11

Power supply part number        : 341-0029-04

--More--                           Motherboard serial number       : CAT1018018H

Power supply serial number      : LIT10120362

Model revision number           : J0

Motherboard revision number     : A0

Model number                    : WS-C3750-48PS-S

System serial number            : CAT1018R2YB

SFP Module assembly part number : 73-7757-03

SFP Module revision number      : A0

SFP Module serial number        : CAT10171JBM

Top assembly part number        : 800-25858-03

Top assembly revision number    : F0

Version ID                      : V05

CLEI Code Number                : COM1W00ARB

Switch 04

---------

Switch Uptime                   : 18 weeks, 3 days, 11 hours, 16 minutes

Base ethernet MAC Address       : 00:1E:49:F8:D3:80

Motherboard assembly number     : 73-9675-11

Power supply part number        : 341-0029-05

Motherboard serial number       : FDO114507GV

Power supply serial number      : LIT11410GXR

Model revision number           : K0

Motherboard revision number     : B0

Model number                    : WS-C3750-48PS-S

System serial number            : FDO1145X2JG

SFP Module assembly part number : 73-7757-03

SFP Module revision number      : A0

SFP Module serial number        : FDO114503J7

Top assembly part number        : 800-25858-03

Top assembly revision number    : J0

Version ID                      : V05

CLEI Code Number                : COM1W00ARB

Configuration register is 0xF

RDCCI37F0#show proc cpu sorted | ex 0.0

CPU utilization for five seconds: 11%/1%; one minute: 12%; five minutes: 12%

PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process

   9   208999294 229139920        912  2.39%  1.72%  1.90%   0 ARP Input       

  68    36294624 586768083         61  0.63%  0.35%  0.29%   0 Fifo Error Detec

103    29029344 149237236        194  0.47%  0.23%  0.18%   0 hpm main process

251    73618720  24512848       3003  0.31%  0.63%  0.60%   0 Marvell wk-a Pow

146    64054978 264479575        242  0.31%  0.52%  0.55%   0 Hulc LED Process

231    16197753  17096266        947  0.31%  0.15%  0.14%   0 IPC LC Message H

107    27695859  23198743       1193  0.15%  0.24%  0.27%   0 hpm counter proc

207    25784237  77440696        332  0.15%  0.23%  0.31%   0 IP Input        

108    20824663  72331071        287  0.15%  0.20%  0.17%   0 HRPC pm-counters

221    20326958  76187435        266  0.15%  0.15%  0.15%   0 Spanning Tree   

156    35462717  17854228       1986  0.15%  0.33%  0.32%   0 HRPC qos request

201    17863448  18592442        960  0.15%  0.15%  0.15%   0 CDP Protocol    

RDCCI37F0#show proc cpu history

    1111111111111111111111111111111111112222211111111111111111

    5111111111100000000000000000000000004444433333111110000000

100                                                          

90                                                          

80                                                          

70                                                          

60                                                          

50                                                          

40                                                          

30                                                          

20 *                                   *****                

10 **********************************************************

   0....5....1....1....2....2....3....3....4....4....5....5....

             0    5    0    5    0    5    0    5    0    5   

               CPU% per second (last 60 seconds)

    2222211222222222222122221222221212212822222212212221222222

    4042187111333211105451337112259340373021111394690039371631

100                                                          

90                                                          

80                                      *                   

70                                      *                   

60                                      *                   

50                                      *                   

40                                      *                   

30                   * *        *       *        *      * * 

20 ******************* ************ ****#********************

10 ##########################################################

   0....5....1....1....2....2....3....3....4....4....5....5....

             0    5    0    5    0    5    0    5    0    5   

               CPU% per minute (last 60 minutes)

              * = maximum CPU%   # = average CPU%

    9346443443534333543843433363652237344353995343266623434556338343324238

    2693431017037991798427582019567671827274992518730354524518700109392450

100                                         **                           

90 *                                       **                           

80 *                  *                    **                  *        *

70 *                  *        *    *      **               *  *        *

60 *  *            *  *      * **   *    * **     ***     * *  *        *

50 * **      * *   ** *  *   * **   *  * * ***    ***  *  ***  *        *

40 ****** **** *** ********  ****  ***** * ****** ***  * ***** * **  * **

30 ******************************************************************* **

20 ****************************************##****************************

10 ######################################################################

--More--                              0....5....1....1....2....2....3....3....4....4....5....5....6....6....7.

             0    5    0    5    0    5    0    5    0    5    0    5    0

                   CPU% per hour (last 72 hours)

                  * = maximum CPU%   # = average CPU%

RDCCI37F0#show controllers cpu-interfac

ASIC    Rxbiterr   Rxunder    Fwdctfix   Txbuflos   Rxbufloc   Rxbufdrain

-------------------------------------------------------------------------

ASIC0     0          0          0          0          0          0        

ASIC1     0          0          0          0          0          0        

HOL Fix Counts

--------------

No Fixes:          0 Added:          0 In Use:          0 Both:          0

CPU Heartbeat Statistics

Tx Success Tx Fail    1st Thr    2nd Thr    Unthr      RetryCtMax

---------- ---------- ---------- ---------- ---------- ----------

222242642          0          0          0          0          1

Rx Delay

         0          1          2          3          4

---------- ---------- ---------- ---------- ----------

222242642          0          0          0          0

AddlDelay AdvanceCnt

---------- ----------

         0          0

Rx Retries by RetryCount

         0          1          2          3          4          5          6

---------- ---------- ---------- ---------- ---------- ---------- ----------

222242642          0          0          0          0          0          0

         7          8          9

---------- ---------- ----------

         0          0          0

AddlRetry

----------

         0

cpu-queue-frames  retrieved  dropped    invalid    hol-block  stray

----------------- ---------- ---------- ---------- ---------- ----------

rpc               614263200  0          0          0          0        

stp               72674914   0          0          0          0        

ipc               99470973   0          0          0          0        

routing protocol  289267059  0          0          0          0        

L2 protocol       6113187    0          0          0          0        

remote console    1340       0          0          0          0        

--More--                           sw forwarding     18246      0          0          0          0        

host              564528     0          0          0          0        

broadcast         345288241  0          0          0          0        

cbt-to-spt        0          0          0          0          0        

igmp snooping     15822592   0          0          0          0        

icmp              0          0          0          0          0        

logging           0          0          0          0          0        

rpf-fail          0          0          0          0          0        

dstats            0          0          0          0          0        

cpu heartbeat     222242665  0          0          0          0        

cpu-queue         static inuse static added

----------------- ------------ ------------

rpc               0            0          

stp               0            0          

ipc               0            0          

routing protocol  0            0          

L2 protocol       0            0          

remote console    0            0          

sw forwarding     0            0          

host              0            0          

broadcast         0            0          

cbt-to-spt        0            0          

igmp snooping     0            0          

icmp              0            0          

logging           0            0          

rpf-fail          0            0          

dstats            0            0          

cpu heartbeat     0            0          

Supervisor ASIC receive-queue parameters

----------------------------------------

queue 0 maxrecevsize 7E0 pakhead 55C2F9C paktail 47EB6F8

queue 1 maxrecevsize 7E0 pakhead 4899BE0 paktail 489A318

queue 2 maxrecevsize 7E0 pakhead 562DF28 paktail 562DB8C

queue 3 maxrecevsize 7E0 pakhead 5A66FEC paktail 5A6E36C

queue 4 maxrecevsize 7E0 pakhead 48AB568 paktail 575821C

queue 5 maxrecevsize 7E0 pakhead 4AEA7C8 paktail 5A513D4

queue 6 maxrecevsize 7E0 pakhead 4AFBE84 paktail 5A5BF10

queue 7 maxrecevsize 7E0 pakhead 4A8D0E8 paktail 4A981C4

queue 8 maxrecevsize 7E0 pakhead 5A01A94 paktail 4A9A5DC

queue 9 maxrecevsize 7E0 pakhead 5955BB8 paktail 5955BB8

queue A maxrecevsize 7E0 pakhead 4986D60 paktail 594D4F0

queue B maxrecevsize 7E0 pakhead 4B11310 paktail 5A7307C

queue C maxrecevsize 7E0 pakhead 49A4278 paktail 5965D78

queue D maxrecevsize 7E0 pakhead 499BBE0 paktail 595581C

--More--                            queue E maxrecevsize 0 pakhead 0 paktail 0

queue F maxrecevsize 7E0 pakhead 5943388 paktail 497DC18

Supervisor ASIC exception status

--------------------------------

Receive overrun    00000000   Transmit overrun 00000000

FrameSignatureErr  00000000   MicInitialize    00000000

BadFrameErr        00000000   LenExceededErr   00000000

BadJumboSegments   00000000

Supervisor ASIC Mic Registers

------------------------------

MicDirectPollInfo             80000200

MicIndicationsReceived        00000000

MicInterruptsReceived        00000000

MicPcsInfo                   0000001F

MicPlbMasterConfiguration    00000000

MicRxFifosAvailable          00000000

MicRxFifosReady              0000BFFF

MicTimeOutPeriod:    FrameTOPeriod: 00000EA6    DirectTOPeriod: 00004000

MicTransmFramesCopied        00000003

MicTxFifosAvailable          00000003

MicConfiguration:    Conf flag: 00000110    Interrupt Flag: 0000000A

MicReceiveFifoAssignmen    Queue 0 - 7: 00000000    Queue 8 - 15:00000000

MicReceiveFramesReady:    FrameAvailable: 00000001    frameAvaiMask: 00000000

MicException:   

    Exception_flag    00000000

    Message-1    00000000

    Message-2    00000000

    Message-3    00000000

MicIntRxFifo:

    ReadPtr        00000748    WritePtr    00000748

    WHeadPtr    00000748    TxFifoDepth    C0000800

MicIntTxFifo:

    ReadPtr        000008F0    WritePtr    000008F0

    WHeadPtr    000008F0    TxFifoDepth    C0000800

MicDecodeInfo:

Fifo0:    address:    03FF4000 asic_num:    00000100

Fifo1:    address:    03FF4400 asic_num:    00000101

MicTransmitFifoInfo:

Fifo0:     StartPtrs:    066C9000    ReadPtr:    066C91F8

    WritePtrs:    066C91F8    Fifo_Flag:    8A800800

    Weights:    001E001E

Fifo1:     StartPtrs:    06427400    ReadPtr:    06427590

    WritePtrs:    06427590    Fifo_Flag:    89800400

    Weights:    000A000A

MicReceiveFifoInfo:

--More--                           Fifo0:    StartPtr:    066DD000    ReadPtr:    066DDED0

    WritePtrs:    066DDF30    Fifo_Flag:    8B000FA0

    writeHeaderPtr:    066DDF30

Fifo1:    StartPtr:    069DA800    ReadPtr:    069DAB78

    WritePtrs:    069DAB78    Fifo_Flag:    89800400

    writeHeaderPtr:    069DAB78

Fifo2:    StartPtr:    06993000    ReadPtr:    06993018

    WritePtrs:    06993018    Fifo_Flag:    89800400

    writeHeaderPtr:    06993018

Fifo3:    StartPtr:    06BEF800    ReadPtr:    06BEF980

    WritePtrs:    06BEF980    Fifo_Flag:    89800400

    writeHeaderPtr:    06BEF980

Fifo4:    StartPtr:    06A22000    ReadPtr:    06A22138

    WritePtrs:    06A22138    Fifo_Flag:    89800400

    writeHeaderPtr:    06A22138

Fifo5:    StartPtr:    06B5EE00    ReadPtr:    06B5EFE0

    WritePtrs:    06B5EFE0    Fifo_Flag:    88800200

    writeHeaderPtr:    06B5EFE0

Fifo6:    StartPtr:    06BA5C00    ReadPtr:    06BA5E30

    WritePtrs:    06BA5E30    Fifo_Flag:    89800400

    writeHeaderPtr:    06BA5E30

Fifo7:    StartPtr:    06ACE800    ReadPtr:    06ACE9A0

    WritePtrs:    06ACE9A0    Fifo_Flag:    89800400

    writeHeaderPtr:    06ACE9A0

Fifo8:    StartPtr:    06B38E00    ReadPtr:    06B38E30

    WritePtrs:    06B38E30    Fifo_Flag:    88800200

    writeHeaderPtr:    06B38E30

Fifo9:    StartPtr:    066DB4D8    ReadPtr:    066DB4D8

    WritePtrs:    066DB4D8    Fifo_Flag:    82800008

    writeHeaderPtr:    066DB4D8

Fifo10:    StartPtr:    06A6B800    ReadPtr:    06A6B870

    WritePtrs:    06A6B870    Fifo_Flag:    88800200

    writeHeaderPtr:    06A6B870

Fifo11:    StartPtr:    066C8F00    ReadPtr:    066C8F00

    WritePtrs:    066C8F00    Fifo_Flag:    86800080

    writeHeaderPtr:    066C8F00

Fifo12:    StartPtr:    06A98800    ReadPtr:    06A98B00

    WritePtrs:    06A98800    Fifo_Flag:    89000100

    writeHeaderPtr:    06A98800

Fifo13:    StartPtr:    06427880    ReadPtr:    06427880

    WritePtrs:    06427880    Fifo_Flag:    86800080

    writeHeaderPtr:    06427880

Fifo14:    StartPtr:    00000000    ReadPtr:    00000000

    WritePtrs:    00000000    Fifo_Flag:    00800000

    writeHeaderPtr:    00000000

Fifo15:    StartPtr:    06992F80    ReadPtr:    06992F90

--More--                               WritePtrs:    06992F90    Fifo_Flag:    84800020

    writeHeaderPtr:    06992F90

===========================================================

Complete Board Id:0x4006

===========================================================

RDCCI37F0#show platform port-asic stats drop

Port-asic Port Drop Statistics - Summary

========================================

  RxQueue 0 Drop Stats: 0

  RxQueue 1 Drop Stats: 0

  RxQueue 2 Drop Stats: 0

  RxQueue 3 Drop Stats: 0

  Port  0 TxQueue Drop Stats: 0

  Port  1 TxQueue Drop Stats: 0

  Port  2 TxQueue Drop Stats: 0

  Port  3 TxQueue Drop Stats: 0

  Port  4 TxQueue Drop Stats: 43101743

  Port  5 TxQueue Drop Stats: 23173181

  Port  6 TxQueue Drop Stats: 159216307

  Port  7 TxQueue Drop Stats: 0

  Port  8 TxQueue Drop Stats: 79275931

  Port  9 TxQueue Drop Stats: 159240537

  Port 10 TxQueue Drop Stats: 0

  Port 11 TxQueue Drop Stats: 65443677

  Port 12 TxQueue Drop Stats: 43845906

  Port 13 TxQueue Drop Stats: 0

  Port 14 TxQueue Drop Stats: 0

  Port 15 TxQueue Drop Stats: 23250063

  Port 16 TxQueue Drop Stats: 23113365

  Port 17 TxQueue Drop Stats: 79946637

  Port 18 TxQueue Drop Stats: 16162287

  Port 19 TxQueue Drop Stats: 0

  Port 20 TxQueue Drop Stats: 23124609

  Port 21 TxQueue Drop Stats: 0

  Port 22 TxQueue Drop Stats: 23080029

  Port 23 TxQueue Drop Stats: 0

  Port 24 TxQueue Drop Stats: 23078915

  Port 25 TxQueue Drop Stats: 0

  Supervisor TxQueue Drop Statistics

    Queue  0: 0

    Queue  1: 0

    Queue  2: 0

    Queue  3: 105753

    Queue  4: 0

    Queue  5: 0

    Queue  6: 0

    Queue  7: 20877

    Queue  8: 150143

    Queue  9: 0

--More--                               Queue 10: 0

--More--                               Queue 11: 0

    Queue 12: 0

    Queue 13: 0

    Queue 14: 0

    Queue 15: 0

Port-asic Port Drop Statistics - Details

========================================

  RxQueue Drop Statistics

    Queue 0

    Weight 0 Frames: 0

    Weight 1 Frames: 0

    Weight 2 Frames: 0

    Queue 1

    Weight 0 Frames: 0

    Weight 1 Frames: 0

    Weight 2 Frames: 0

    Queue 2

    Weight 0 Frames: 0

    Weight 1 Frames: 0

    Weight 2 Frames: 0

    Queue 3

    Weight 0 Frames: 0

    Weight 1 Frames: 0

    Weight 2 Frames: 0

  Port 0 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

  Port 1 TxQueue Drop Statistics

    Queue 0

--More--                                 Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

  Port 2 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

  Port 3 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

--More--                                 Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

  Port 4 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 38592

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 43063151

  Port 5 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 31010

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 23142171

  Port 6 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

--More--                                 Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 83682

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 159132625

  Port 7 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

  Port 8 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 30633

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

--More--                                 Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 79245298

  Port 9 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 83672

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 159156865

  Port 10 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

  Port 11 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

--More--                                 Weight 0 Frames 47446

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 65396231

  Port 12 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 38774

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 43807132

  Port 13 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

--More--                                 Weight 2 Frames 0

  Port 14 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

  Port 15 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 30601

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 23219462

  Port 16 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 31014

      Weight 1 Frames 0

--More--                                 Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 23082351

  Port 17 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 53038

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 79893599

  Port 18 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 20670

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 16141617

--More--                             Port 19 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

  Port 20 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 31004

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 23093605

  Port 21 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

--More--                                 Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

  Port 22 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 31013

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 23049016

  Port 23 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

  Port 24 TxQueue Drop Statistics

    Queue 0

--More--                                 Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 31032

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 23047883

  Port 25 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

  Supervisor TxQueue Drop Statistics

    Queue  0: 0

    Queue  1: 0

    Queue  2: 0

    Queue  3: 105753

    Queue  4: 0

    Queue  5: 0

    Queue  6: 0

    Queue  7: 20877

    Queue  8: 150143

    Queue  9: 0

    Queue 10: 0

    Queue 11: 0

--More--                               Queue 12: 0

    Queue 13: 0

    Queue 14: 0

    Queue 15: 0

RDCCI37F0#  show platform port-asic stats drop                                                                                                             exit

HI Aman,

I believe the switches are working fine now.

The CPU on the switches seems to be normal:

RDCCI37F0#show proc cpu sorted | ex 0.0

CPU utilization for five seconds: 11%/1%; one minute: 12%; five minutes: 12%

   5111111111100000000000000000000000004444433333111110000000

100                                                         

90                                                         

80                                                         

70                                                         

60                                                         

50                                                         

40                                                         

30                                                         

20 *                                   *****               

10 **********************************************************

   0....5....1....1....2....2....3....3....4....4....5....5....

             0    5    0    5    0    5    0    5    0    5  

               CPU% per second (last 60 seconds)

    2222211222222222222122221222221212212822222212212221222222

    4042187111333211105451337112259340373021111394690039371631

100                                                         

90                                                         

80                                      *                  

70                                      *                  

60                                      *                  

50                                      *                  

40                                      *                  

30                   * *        *       *        *      * *

20 ******************* ************ ****#********************

10 ##########################################################

   0....5....1....1....2....2....3....3....4....4....5....5....

             0    5    0    5    0    5    0    5    0    5  

               CPU% per minute (last 60 minutes)

              * = maximum CPU%   # = average CPU%

Its quite difficult to findout what causing the CPU to go high as it has drop back to normal.

I request you to kindly log a tac case when you see the CPU going high and we can look into this live and findout the rootcause.

I can recommend few steps which you can look out during the high cpu:

http://www.cisco.com/en/US/products/hw/switches/ps5023/products_tech_note09186a00807213f5.shtml

This is the one of the link which helps you to troubleshoot the high cpu.

Check the cpu utilization and findout it is due to process or interupt:

Eg: 99%11%( Were 99 is total cpu utizliation and 11 % is due to interupt and 99-11 is due to process).

2) CPU has 16 Queus on 3750 hence check which Q is punting the traffic (show controllers cpu-interface- under this command check which is hte Q value which is getting increased.)

3) look for TX Q drop which is getting increaeed under show platform port asic stats drop.

4)There is some debugs but I would suggest you do it with the TAC engineer.

Hope this helps.

REgards

Inayath

Review Cisco Networking for a $25 gift card