cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
24547
Views
0
Helpful
27
Replies

High CPU Utilization on 7606

sabuz_banik
Level 1
Level 1

Dear concern,

Our Router  7606 is in a problem with high CPU Utilization. We are using one EBGP  and one IBGP session. We are also running OSPF with area 0 between two  IBGP router to exchange few local networks. When bandwidth cross 100 MB  than only CPU use become 100%. when bandwidth consumption is 80MB CPU  use become 80% end so on.

Pls find the bellow out put and assist us.

----------------------------------------------------------------------------------------------------------------------

Out put is when bandwidth consumption is 34MB

----------------------------------------------------------------------------------------------------------------------

CPU utilization for five seconds: 44%/43%; one minute: 42%; five minutes: 30%

PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process

  78          44        9625          4  0.39%  0.03%  0.00%   2 Virtual Exec

  13          40         375        106  0.07%  0.00%  0.00%   0 ARP Input

  30         116        1098        105  0.07%  0.01%  0.00%   0 IPC Seat Manager

164         524       42660         12  0.07%  0.04%  0.05%   1 Virtual Exec

186           0       22945          0  0.07%  0.03%  0.02%   0 ACE Tunnel Task

259          12       89401          0  0.07%  0.10%  0.08%   0 Ethernet Msec Ti

   1           0           8          0  0.00%  0.00%  0.00%   0 Chunk Manager

   2           4         156         25  0.00%  0.01%  0.00%   0 Load Meter

   4           0          47          0  0.00%  0.00%  0.00%   0 SCP async: Draco

   5           0         173          0  0.00%  0.00%  0.00%   0 Retransmission o

   6           0           3          0  0.00%  0.00%  0.00%   0 IPC ISSU Dispatc

   7           0           1          0  0.00%  0.00%  0.00%   0 PF Redun ICC Req

   8        8684         466      18635  0.00%  0.86%  0.81%   0 Check heaps

   9           0           1          0  0.00%  0.00%  0.00%   0 Pool Manager

  10           0           1          0  0.00%  0.00%  0.00%   0 DiscardQ Backgro

  11           0           2          0  0.00%  0.00%  0.00%   0 Timers

  12           4         226         17  0.00%  0.00%  0.00%   0 WATCH_AFS

  14           8         853          9  0.00%  0.00%  0.00%   0 ARP Background

  15           0           2          0  0.00%  0.00%  0.00%   0 ATM Idle Timer

  16           0           1          0  0.00%  0.00%  0.00%   0 ATM ASYNC PROC

  End = e   Freeze = f

Enter Command:

-------------------------------------------------------------------------------------------------------------------------------

RTLSGR1#sh memory statistics

                Head    Total(b)     Used(b)     Free(b)   Lowest(b)  Largest(b)

Processor   1322CA94   1692218732   522133152   1170085580   1168835904   1169364236

      I/O   78000000   134217728    50319772    83897956    83897956    83896252

---------------------------------------------------------------------------------------------------------------------------------------

RTLSGR1#sh cef drop

% Command accepted but obsolete, see 'show (ip|ipv6) cef switching statistics [feature]'

IPv4 CEF Drop Statistics

Slot  Encap_fail  Unresolved Unsupported    No_route      No_adj  ChkSum_Err

RP             0           0         422          10           4           0

5/0            0           0           0           0           0           0

3/0            0           0           0           0           0           0

RTLSGR1#

-------------------------------------------------------------------------------------------------------------

SUP

RSP720-3C-GE

--------------------------------------------------------------------------------------------------------------------

IOS

c7600rsp72043_rp-ADVENTERPRISEK9-M

---------------------------------------------------------------------------------------------------------------------

Regards

1 Accepted Solution

Accepted Solutions

Please don't forget to mark the query as answered if you are satisfied with the answer

View solution in original post

27 Replies 27

Raju Sekharan
Cisco Employee
Cisco Employee

The reason for high cpu is interrupts here (44%/43%)

When did you start seeing this issue? Did you make any change prior to this issue

Can you upload the following output

1. Show tech

2. Show interfaces | inc up|rate|drop

The router was configured one month back and was running well.One week back customer complain with dropping the packet through ICMP/Ping. After issuing sh proc cpu moni we found CPU uses increase parallal to bandwidth consumption.

--------------------------------------------------------------

RTLSGR1#

RTLSGR1#Show interfaces | inc up|rate|drop

GigabitEthernet5/1 is up, line protocol is up (connected)

  Full-duplex, 1000Mb/s, media type is LH

  Input queue: 0/75/4426/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  5 minute input rate 117567000 bits/sec, 151845 packets/sec

  5 minute output rate 113281000 bits/sec, 144974 packets/sec

     0 unknown protocol drops

GigabitEthernet5/2 is up, line protocol is up (connected)

  Full-duplex, 1000Mb/s, media type is LH

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  5 minute input rate 43863000 bits/sec, 68152 packets/sec

  5 minute output rate 48144000 bits/sec, 75019 packets/sec

     0 unknown protocol drops

Loopback0 is up, line protocol is up

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  5 minute input rate 0 bits/sec, 0 packets/sec

  5 minute output rate 0 bits/sec, 0 packets/sec

     0 unknown protocol drops

  Keepalive not supported

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  5 minute input rate 0 bits/sec, 0 packets/sec

  5 minute output rate 0 bits/sec, 0 packets/sec

     0 unknown protocol drops

-----------------------------------------------------------------------------------------------------------------------------------------------

RTLSGR1#sh tech-support

------------------ show clock ------------------

*07:10:26.950 UTC Sun Sep 30 2012

------------------ show version ------------------

Cisco IOS Software, c7600rsp72043_rp Software (c7600rsp72043_rp-ADVENTERPRISEK9-M), Version 12.2(33)SRE6, RELEASE SOFTWARE (fc1)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2012 by Cisco Systems, Inc.

Compiled Thu 15-Mar-12 05:31 by prod_rel_team

ROM: System Bootstrap, Version 12.2(33r)SRD5, RELEASE SOFTWARE (fc1)

BOOTLDR: Cisco IOS Software, c7600rsp72043_rp Software (c7600rsp72043_rp-ADVENTERPRISEK9-M), Version 12.2(33)SRE6, RELEASE SOFTWARE (fc1)

RTLSGR1 uptime is 21 hours, 10 minutes

Uptime for this control processor is 21 hours, 10 minutes

System returned to ROM by reload (SP by reload)

System image file is "bootdisk:c7600rsp72043-adventerprisek9-mz.122-33.SRE6.bin"

Last reload type: Normal Reload

This product contains cryptographic features and is subject to United

States and local country laws governing import, export, transfer and

use. Delivery of Cisco cryptographic products does not imply

third-party authority to import, export, distribute or use encryption.

Importers, exporters, distributors and users are responsible for

compliance with U.S. and local country laws. By using this product you

agree to comply with applicable laws and regulations. If you are unable

to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:

http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to

export@cisco.com.

Cisco CISCO7606-S (M8500) processor (revision 1.0) with 1900544K/131072K bytes of memory.

Processor board ID FOX1249G0P1

BASEBOARD: RSP720

CPU: MPC8548_E, Version: 2.0, (0x80390020)

CORE: E500, Version: 2.0, (0x80210020)

CPU:1200MHz, CCB:400MHz, DDR:200MHz,

L1:    D-cache 32 kB enabled

        I-cache 32 kB enabled

Last reset from power-on

1 Virtual Ethernet interface

2 Gigabit Ethernet interfaces

3964K bytes of non-volatile configuration memory.

507024K bytes of Internal ATA PCMCIA card (Sector size 512 bytes).

Configuration register is 0x2102

------------------ show running-config ------------------

Building configuration...

Current configuration : 3282 bytes

!

! Last configuration change at 20:13:17 UTC Sat Sep 29 2012 by rtl

!

version 12.2

service timestamps debug datetime msec

service timestamps log datetime msec

service password-encryption

service counters max age 10

!

hostname RTLSGR1

!

boot-start-marker

boot system flash sup-bootdisk:c7600rsp72043-adventerprisek9-mz.122-33.SRE6.bin

boot-end-marker

!

enable secret 5

enable password 7

!

no aaa new-model

!

!

!

no ip source-route

!

!

no ip domain lookup

!

!

!

!

vtp mode transparent

mls rp ip

no mls flow ip

no mls flow ipv6

no mls rate-limit multicast ipv4 fib-miss

no mls rate-limit multicast ipv4 partial

no mls rate-limit unicast acl vacl-log

no mls rate-limit unicast ip rpf-failure

no mls rate-limit unicast ip icmp unreachable no-route

no mls rate-limit unicast ip icmp unreachable acl-drop

no mls rate-limit unicast ip errors

no mls rate-limit all ttl-failure

no mls rate-limit all mtu-failure

mls cef error action reset

multilink bundle-name authenticated

!

!

!

!

!

spanning-tree mode pvst

spanning-tree extend system-id

diagnostic cns publish cisco.cns.device.diag_results

diagnostic cns subscribe cisco.cns.device.diag_commands

username das privilege 15 secret 5

username rtl privilege 15 secret 5

!

redundancy

main-cpu

  auto-sync running-config

mode sso

!

vlan internal allocation policy ascending

!

ip ssh version 2

!

!

class-map match-all icmp

  match protocol icmp

!

policy-map icmp-policy

  class icmp

    priority 100

  class class-default

    fair-queue

!

!

!

!

!

!

!

!

interface Loopback0

ip address 192.168.18.3 255.255.255.240

!

interface GigabitEthernet5/1

description Internet With NetroNext

ip address 103.14.80.19 255.255.255.240

speed nonegotiate

!

interface GigabitEthernet5/2

ip address 103.12.212.130 255.255.255.252

speed nonegotiate

!

interface Vlan1

no ip address

shutdown

!

router ospf 1

log-adjacency-changes

redistribute static subnets route-map rtl

network 103.14.80.16 0.0.0.15 area 0

network 192.168.18.0 0.0.0.15 area 0

!

router bgp 58636

no synchronization

bgp router-id 103.12.212.130

bgp log-neighbor-changes

network 103.12.212.128 mask 255.255.255.252

network 103.14.80.0 mask 255.255.252.0

network 103.14.80.16 mask 255.255.255.240

neighbor 103.12.212.129 remote-as 132337

neighbor 192.168.18.1 remote-as 58636

neighbor 192.168.18.1 update-source Loopback0

neighbor 192.168.18.1 soft-reconfiguration inbound

no auto-summary

!

no ip forward-protocol udp netbios-ns

no ip forward-protocol udp netbios-dgm

!

no ip http server

no ip http secure-server

ip route 103.14.80.0 255.255.252.0 Null0

ip route 103.14.80.48 255.255.255.240 103.14.80.20

ip route 192.168.100.0 255.255.255.0 103.14.80.49

ip route 192.168.101.0 255.255.255.0 103.14.80.49

ip route 192.168.102.0 255.255.255.0 103.14.80.49

!

!

ip prefix-list netronext-out seq 5 permit 103.14.80.0/24

ip prefix-list netronext-out seq 10 deny 0.0.0.0/0 le 32

access-list 10 permit 192.168.100.0

access-list 10 permit 192.168.101.0

access-list 10 permit 192.168.102.0

no cdp run

!

route-map rtl permit 10

match ip address 10

!

snmp-server community RO

!

!

control-plane

!

!

line con 0

line vty 0 4

login local

length 0

international

transport input telnet

!

exception crashinfo buffersize 80

!

end

------------------ show stacks ------------------

Minimum process stacks:

Free/Size   Name

5548/6000   OIR IOS Process

5532/6000   ISSU Infra API Delayed Registration Process

4488/6000   IPC ISSU Versioning Process

3508/6000   IPC ISSU Receive Process

5444/6000   IPC Zone Manager

5544/6000   ICC Retry Q

3236/6000   IPC Split Init

4284/6000   RTTY Client Registration

3856/6000   PF ISSU PRST media read proc

5552/6000   CDP BLOB

5508/6000   FM HA Sync

5508/6000   L3 Manager HA

2144/6000   c6k ivrfs

11068/12000  MRIB IPv4 Init Process

11480/12000  MRIB IPv6 Init Process

11076/12000  MFIB RP IPC

5116/6000   ICC Slave Comp. Up

3396/6000   eobc_init_process

4912/6000   DRMI Master Reg. Slave Process

5552/6000   PM MP Process

4940/6000   EARL INFO CAPABILITY process

5552/6000   Low Batt Process

5492/6000   C6K ENV RP init

5000/6000   SPAN Subsystem

5364/6000   PostOfficeNet

5548/6000   SASL MAIN

11052/12000  Router Init

10892/12000  CDP Protocol

53996/60000  EEM TCL Proc

53796/60000  EEM Auto Registration Proc

11536/12000  cdp init process

6320/12000  Init

5032/6000   Draco DFS Port Registation Proc

4840/6000   IPC LC Port Opener

5548/6000   MPLS ACL RP CFG Background

5340/6000   RADIUS INITCONFIG

4780/6000   LCC Configure

9504/12000  draco-oir:slot 5

4028/6000   BGP Open

4832/6000   SLB RF Active Proc

10512/12000  draco-oir:slot 3

5496/6000   URPF stats

2552/3000   Rom Random Update Process

33720/36000  TCP Command

10816/12000  Exec

5516/6000   Link Status process

5940/12000  Virtual Exec

5528/6000   FCP Router

5536/6000   moduleTrap Generation Process

19912/24000  show techsupport huge_stack

Interrupt level stacks:

Level    Called Unused/Size  Name

  1   253516503   6364/9000  dma Intr handler

  2     1192726   6476/9000  eobc etsec1 tx interrupt

  3           0   9000/9000  Low Battery Interrupt

  4       18402   8916/9000  16550 Con/Aux Interrupt

  5           0   9000/9000  SP to RP Interrupt

  6     3538172   8924/9000  Timebase Reference Interrupt

  7    19127678   8812/9000  M8500_TIMER_INTERRUPT

***************************************************

******* Information of Last System Crash **********

***************************************************

Using bootdisk:crashinfo.

%Error opening bootdisk:crashinfo (File not found)

***************************************************

****** Information of Last System Crash - SP ******

***************************************************

The last crashinfo failed to be written.

Please verify the exception crashinfo configuration

the filesytem devices, and the free space on the

filesystem devices.

Using crashinfo_FAILED.

%Error opening disk0:crashinfo_FAILED (File not found)

------------------ show interfaces ------------------

GigabitEthernet5/1 is up, line protocol is up (connected)

  Hardware is C7600 1Gb 802.3, address is 0021.a00b.1f80 (bia 0021.a00b.1f80)

  Description: Internet With NetroNext

  Internet address is 103.14.80.19/28

  MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,

     reliability 255/255, txload 35/255, rxload 36/255

  Encapsulation ARPA, loopback not set

  Keepalive set (10 sec)

  Full-duplex, 1000Mb/s, media type is LH

  input flow-control is off, output flow-control is on

  Clock mode is auto

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input 00:00:00, output 00:00:00, output hang never

  Last clearing of "show interface" counters never

  Input queue: 0/75/4426/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 145078000 bits/sec, 187358 packets/sec

  5 minute output rate 140346000 bits/sec, 179956 packets/sec

  L2 Switched: ucast: 21922 pkt, 1587633 bytes - mcast: 28897 pkt, 3332971 bytes

  L3 in Switched: ucast: 1945640473 pkt, 284679948179 bytes - mcast: 0 pkt, 0 bytes mcast

  L3 out Switched: ucast: 7186984243 pkt, 630745022136 bytes mcast: 0 pkt, 0 bytes

     5385099034 packets input, 561321660323 bytes, 0 no buffer

     Received 62921 broadcasts (0 IP multicasts)

     0 runts, 0 giants, 3 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog, 0 multicast, 0 pause input

     0 input packets with dribble condition detected

     7186568266 packets output, 630714825156 bytes, 0 underruns

     0 output errors, 0 collisions, 2 interface resets

     0 unknown protocol drops

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier, 0 pause output

     0 output buffer failures, 0 output buffers swapped out

GigabitEthernet5/2 is up, line protocol is up (connected)

  Hardware is C7600 1Gb 802.3, address is 0021.a00b.1f80 (bia 0021.a00b.1f80)

  Internet address is 103.12.212.130/30

  MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,

     reliability 255/255, txload 15/255, rxload 13/255

  Encapsulation ARPA, loopback not set

  Keepalive set (10 sec)

  Full-duplex, 1000Mb/s, media type is LH

  Media-type configured as SFP connector

  input flow-control is off, output flow-control is on

  Clock mode is auto

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input 00:00:00, output 00:00:00, output hang never

  Last clearing of "show interface" counters never

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 54458000 bits/sec, 85172 packets/sec

  5 minute output rate 59194000 bits/sec, 92575 packets/sec

  L2 Switched: ucast: 798508 pkt, 74558632 bytes - mcast: 1 pkt, 64 bytes

  L3 in Switched: ucast: 4478303555 pkt, 361957334907 bytes - mcast: 0 pkt, 0 bytes mcast

  L3 out Switched: ucast: 473908163 pkt, 38519838944 bytes mcast: 0 pkt, 0 bytes

     4479526586 packets input, 362293052779 bytes, 0 no buffer

     Received 3 broadcasts (0 IP multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 126 overrun, 0 ignored

     0 watchdog, 0 multicast, 0 pause input

     0 input packets with dribble condition detected

     4938216931 packets output, 398118007574 bytes, 0 underruns

     0 output errors, 0 collisions, 2 interface resets

     0 unknown protocol drops

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier, 4 pause output

     0 output buffer failures, 0 output buffers swapped out

Loopback0 is up, line protocol is up

  Hardware is Loopback

  Internet address is 192.168.18.3/28

  MTU 1514 bytes, BW 8000000 Kbit/sec, DLY 5000 usec,

     reliability 255/255, txload 1/255, rxload 1/255

  Encapsulation LOOPBACK, loopback not set

  Keepalive set (10 sec)

  Last input 11:13:25, output never, output hang never

  Last clearing of "show interface" counters never

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/0 (size/max)

  5 minute input rate 0 bits/sec, 0 packets/sec

  5 minute output rate 0 bits/sec, 0 packets/sec

     0 packets input, 0 bytes, 0 no buffer

     Received 0 broadcasts (0 IP multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort

     473 packets output, 89549 bytes, 0 underruns

     0 output errors, 0 collisions, 0 interface resets

     0 unknown protocol drops

     0 output buffer failures, 0 output buffers swapped out

Vlan1 is administratively down, line protocol is down

  Hardware is EtherSVI, address is 0021.a00b.1f80 (bia 0021.a00b.1f80)

  MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,

     reliability 255/255, txload 1/255, rxload 1/255

  Encapsulation ARPA, loopback not set

  Keepalive not supported

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input never, output never, output hang never

  Last clearing of "show interface" counters never

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 0 bits/sec, 0 packets/sec

  5 minute output rate 0 bits/sec, 0 packets/sec

     0 packets input, 0 bytes, 0 no buffer

     Received 0 broadcasts (0 IP multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 packets output, 0 bytes, 0 underruns

     0 output errors, 1 interface resets

     0 unknown protocol drops

     0 output buffer failures, 0 output buffers swapped out

------------------ show interfaces history ------------------

------------------ show controllers ------------------

        Hardware is RSP720 EOBC

Interface information:

        Interface EOBC0/0(idb 0x17CCC6D0)

        108696 packets input, 27128580 bytes

        4 broadcasts received

        126939 packets output, 45936031 bytes

        4300 broadcasts sent

Dev_Instance=0x183C6B98 406612888

Receive/Transmit Buffer Descriptors

        RingSz  Shadow   StartBD  LastBD   Head  Tail Pend

  Rx0    512   183C7534  79C43CC0 79C44CB8   152   0    0

  Tx0    512   0         79C44D00 79C45CF8   475   475  0

  Rx Drop On Busy Disabled

Hash match table for multicast (in use 1, maximum 64 entries):

    Entry 0 MAC Addr = 0100.8300.0000

ETSEC counters in Hardware

64b         = 685        64b-127b    = 227        128b-255b   = 51

256b-511b   = 609        512b-1023b  = 565        1024b-1518b = 937

1519b-1522b = 0          RxBytes     = 0          RxPkt       = 49

RxFCSErr    = 0          RxMCast     = 27         RxBCast     = 4

RxCntrlFr   = 0          RxPauseFR   = 0          RxUnkop     = 0

RxAlignErr  = 0          RxLenErr    = 0          RxCodeErr   = 0

RxSenseErr  = 0          RxUnder     = 0          RxOver      = 0

RxFrag      = 10         RxJab       = 0          RxDrop      = 0

RxFilerRej  = 0          Txbytes     = 0          TxPkt       = 987

TxMCast     = 202        TxBCast     = 0          TxPause     = 0

TxDef       = 205        TxCol       = 29         TxMCol      = 3

TxLateCol   = 0          TxExCol     = 0          TxTotCol    = 38

TxDrop      = 0          TxJab       = 0          TxFCS       = 0

TxOver      = 0          TxUnder     = 0          TxFrg       = 0

------------------ show user ------------------

    Line       User       Host(s)              Idle       Location

   1 vty 0     rtl        idle                 00:03:48 203.76.109.154

*  2 vty 1     rtl        idle                 00:00:01 203.76.109.154

  Interface    User               Mode         Idle     Peer Address

------------------ show data-corruption ------------------

No data inconsistency errors have been recorded.

------------------ show memory statistics ------------------

                Head    Total(b)     Used(b)     Free(b)   Lowest(b)  Largest(b)

Processor   1322CA94   1692218732   522457996   1169760736   1167289628   1168873196

      I/O   78000000   134217728    50320188    83897540    83838016    83831612

------------------ show process memory ------------------

Processor Pool Total: 1692218732 Used:  522444180 Free: 1169774552

      I/O Pool Total:  134217728 Used:   50320144 Free:   83897584

PID TTY  Allocated      Freed    Holding    Getbufs    Retbufs Process

   0   0  263483816   13296604  238688968       7490  171325823 *Init*

   0   0      14068    4875544      14068         11         11 *Sched*

   0   0  159652980  162595780    1497596        193        155 *Dead*

   0   0          0          0     677692          0          0 *MallocLite*

   1   0     517920          0     525148          0          0 Chunk Manager

   2   0        236        236       4228          0          0 Load Meter

   3   0          0          0       7092          0          0 FCP Router

   4   0     293812     293812       7092       3617       4797 SCP async: Draco

   5   0          0          0       7228          0          0 Retransmission o

   6   0      15056          0       7228          0          0 IPC ISSU Dispatc

   7   0          0          0       7228          0          0 PF Redun ICC Req

   8   0       3360        236      10352          0          0 Check heaps

   9   0      82936     212216      90164       5560          0 Pool Manager

  10   0          0      16160       7228          0          0 DiscardQ Backgro

  11   0        236        236       7228          0          0 Timers

  12   0          0          0       4228          0          0 WATCH_AFS

  13   0      66120      35624      37724      15618      15618 ARP Input

  14   0     987328     962224      32332         21         21 ARP Background

  15   0        236        236       7228          0          0 ATM Idle Timer

  16   0          0          0       7228          0          0 ATM ASYNC PROC

  17   0          0          0       7228          0          0 AAA_SERVER_DEADT

  18   0          0          0      13228          0          0 Policy Manager

  19   0      35392      10336      33488        124        124 Entity MIB API

  20   0     598996          0     609224        176        144 EEM ED Syslog

  21   0          0          0       7228          0          0 IFS Agent Manage

  22   0         80       1604       7228          0          0 IPC Event Notifi

  23   0          0          0       7228          0          0 IPC Mcast Pendin

  24   0          0          0       7228          0          0 IPC Dynamic Cach

  25   0          0          0       7228          0          0 IPC Service NonC

  26   0     122128     115760      13516        104        104 PF_Split Sync Pr

  27   0          0          0       7228     138630     138630 IPC Periodic Tim

  28   0      24460          0       8572          0          0 IPC Deferred Por

  29   0          0          0       7228          0          0 IPC Process leve

  30   0     350760     213028     167956     113119     113119 IPC Seat Manager

  31   0          0          0       7228          0          0 IPC Check Queue

  32   0     216344          0      19248        566        552 IPC Seat RX Cont

  33   0      12516       5384      18372          0          0 IPC Seat TX Cont

  34   0      19184        496      16536      11275      11275 IPC Keep Alive M

  35   0          0          0       7228          0          0 IPC Loadometer

  36   0          0          0       7228          0          0 Compute SRP rate

  37   0          0          0       7228          0          0 RO Notify Timers

  38   0          0          0       4228          0          0 RMI RM Notify Wa

  39   0          0          0       7228          0          0 CEF MIB API

  40   0        236        236       7228          0          0 DDR Timers

  41   0          0          0       7228          0          0 ifIndex Receive

  42   0          0          0       7228          0          0 SENSOR-MGR event

  43   0        236        236       7228          0          0 Serial Backgroun

  44   0          0          0      25228          0          0 Crash writer

  45   0        304          0       7532          0          0 Exception contro

  46   0       3416       2984       7228         22         22 RPC pf-split-rp

  47   0    4882872    4882872       7228      39746      39746 EnvMon

  48   0        236        236       7228          0          0 GraphIt

  49   0     310092     132988     160812       1091       1091 rf proxy rp agen

  50   0        236        236       7228          0          0 rf proxy message

  51   0      15760       2276      20200        187        187 Entity MIB C6k P

  52   0       5484        236       7228          2          2 client_entity_se

  53   0          0          0       7228          0          0 SERIAL A'detect

  54   0          0          0       7228          0          0 Snmp ICC Process

  55   0     134844       6636       8228         44         44 Cat6k SNMP

  56   0        236       1032       7228          0          0 Cat6k SNMP Trap

  57   0       8820       2892      12704          6          6 RPC idprom-MP

  58   0          0          0       7228          0          0 Dynamic ARP Insp

  59   0          0          0       7228          0          0 Critical Bkgnd

  60   0      34916        708      15524          2          2 Net Background

  61   0       2472       2156      13544          0          0 IDB Work

  62   0        236        236      13228        320        320 Logger

  63   0      13664       3196       7228          0          0 TTY Background

  64   0       2268       1720       7776         10         10 IF-MGR control p

  65   0          0          0       7228          0          0 IF-MGR event pro

  66   0        472        604       7228          0          0 RPC rf_rfmib_not

  67   0      80432      61164      26496        352        352 Env action

  68   0          0          0       7228          0          0 EOBC ETSEC FREEZ

  69   0        236        236       7228          0          0 Flash ICC Server

  70   0          0          0       7228          0          0 ICC Nego

  71   0        224          0       7452          0          0 itasca

  72   0          0          0       7228       7060       7060 Net Input

  73   0      46548     130276      15188          0          0 draco-oir-proces

  75   0        696        700       7452          4          4 svclc

  76   0          0          0       7228          0          0 IPC ISSU Version

  77   0          0          0       7228          0          0 IPC ISSU Version

  78   1    2606832    2579140      14712      13299      13303 Virtual Exec

  79   0          0          0       7228          0          0 IPC ISSU Version

  80   0          0          0       7228          0          0 IPC ISSU Version

  81   0          0          0       7228          0          0 IPC ISSU Version

  82   0          0          0       7228          0          0 IPC ISSU Version

  83   0          0          0       7228          0          0 IPC ISSU Version

  84   0          0          0       7228          0          0 IPC ISSU Version

  85   0          0          0       7228          0          0 IPC ISSU Version

  86   0          0          0       7228          0          0 IPC ISSU Version

  87   0          0          0       7228          0          0 IPC ISSU Version

  88   0          0          0       7228          0          0 IPC ISSU Version

  89   0          0          0       7228          0          0 IPC ISSU Version

  90   0       4728          0      11956          0          0 IPC ISSU Version

  91   0          0          0       7228          0          0 IPC ISSU Version

  92   0          0          0       7228          0          0 IPC ISSU Version

  93   0          0          0       7228          0          0 IPC ISSU Version

  94   0          0          0       7228          0          0 IPC ISSU Version

  95   0          0          0       7228          0          0 IPC ISSU Version

  96   0          0          0       7228          0          0 IPC ISSU Version

  97   0          0          0       7228          0          0 IPC ISSU Version

  98   0          0          0       7228          0          0 IPC ISSU Version

  99   0          0          0       7228          0          0 IPC ISSU Version

100   0          0          0       7228          0          0 IPC ISSU Version

101   0          0          0       7228          0          0 IPC ISSU Version

102   0          0          0       7228          0          0 IPC ISSU Version

103   0          0          0       7228          0          0 IPC ISSU Version

104   0          0          0       7228          0          0 IPC ISSU Version

105   0          0          0       7228          0          0 IPC ISSU Version

106   0          0          0       7228          0          0 IPC ISSU Version

107   0          0          0       7228          0          0 IPC ISSU Version

108   0          0          0       7228          0          0 IPC ISSU Version

109   0          0          0       7228          0          0 IPC ISSU Version

110   0          0          0       7228          0          0 IPC ISSU Version

111   0          0          0       7228          0          0 IPC ISSU Version

112   0          0          0       7228          0          0 IPC ISSU Version

113   0          0          0       7228          0          0 IPC ISSU Version

114   0        944        944       7228         39         39 RPC Sync

115   0        472        604       7228          0          0 RPC rpc-slave-15

116   0          0        148       7228          0          0 SCP Multicast

117   0        224          0       7452          0          0 simba

118   0          0          0       7228          0          0 MWAM CONFIG ON S

119   0          0          0       7228          0          0 SAMI CONFIG ON S

120   0          0          0       7228          0          0 cpf_msg_holdq_pr

121   0          0          0       7228          0          0 cpf_msg_rcvq_pro

122   0          0          0       7228          0          0 CHKPT Test clien

123   0          0          0       7228          0          0 DHCPC HA

124   0          0          0       7228          0          0 DHCPD HA

125   0          0          0       7228          0          0 DHCPv6 Relay HA

126   0          0          0       7228          0          0 Network-rf Notif

127   0          0          0       7228          0          0 ARP HA

128   0          0          0       7228          0          0 MDR SM Proxy Sla

129   0          0          0       7228          0          0 RF CWAN HA Proce

130   0          0          0       7228          0          0 CWAN HA VLAN SYN

131   0          0          0       7228          0          0 CWAN IF EVENT HA

132   0          0          0       7228          0          0 CWAN APS HA Proc

133   0          0          0       7228          0          0 CWAN SRP HA Proc

134   0          0          0       4228          0          0 Inode Table Dest

135   0        800        208       7228      76018      76018 Heartbeat Proces

136   0         76          0       7228          0          0 Const ICC reques

137   0          0          0       7228          0          0 Const ICC event

138   0          0          0       7228          0          0 Const ICC except

139   0          0          0       7228          0          0 ICC Local Reques

140   0          0       1168       7228          0          0 ICC Async mcast

141   0          0    1710624       7228          0          0 ICC Async true m

142   0       1064        864       7428          1          1 ICC Mcast IPC Ev

143   0          0          0       7228          0          0 ICC_REQ Debug Pr

144   0          0          0       7228          0          0 ICC FC Process

145   0          0          0       7228          0          0 ICC FC Test Send

146   0          0          0       7228          0          0 ICC FC Test Low

147   0          0          0       7228          0          0 PF Sync process

148   0          0          0       7228          0          0 QoS HA CHKPT

149   0          0          0       7228          0          0 CWRP SPA TSM HA

150   0        236        236      10228          0          0 Switch IP Host T

151   0          0          0       7156          0          0 EM Action CNS

152   0       4728          0      11956          0          0 IKE HA Mgr

153   0          0          0       7228          0          0 IPSEC HA Mgr

154   0     154076       9256      94388         20         20 PF_Init Process

155   0          0          0       7228          0          0 Transport Port A

156   0       3052       2648       7380        605        605 RTTYS Process

157   0          0          0      13228          0          0 RFS server proce

158   0          0          0       7228          0          0 BACK CHECK

159   0        236        236       7228          0          0 Compute load avg

160   0       4968       1196      12196          0          0 Per-minute Jobs

161   0       6396       5972      10652         16         16 Per-Second Jobs

162   0          0          0       7228          0          0 cpf_process_tpQ

163   0          0          0       7228          0          0 IPC Apps Task

164   2    1628868    1627492      14508        438        429 Virtual Exec

165   0          0          0       7228          0          0 AggMgr Process

166   0        236        236       7228          0          0 CWAN ICC

167   0          0          0       7228          0          0 SFF8472

168   0        236        236       7228          0          0 Service Group PC

169   0          0          0       7228          0          0 LSM Vlan alloc p

170   0     145440          0       7228      21154      21154 Remote Console P

171   0          0          0       7228          0          0 CCB SEND RETRY A

172   0          0          0       7228          0          0 POS APS Event Pr

173   0      88384      31164      59528       1036       1036 CWAN OIR Handler

174   0        236        236       7228          0          0 CWAN Egress SACL

175   0          0          0       7228          0          0 DSX3MIB ll handl

176   0          0          0       7228          0          0 EI RP

177   0       3592       3592       7228          8          8 FPD Management P

178   0          0          0       7228          0          0 FPD Action Proce

179   0        236        236       7228          0          0 const FPD ICC pr

180   0          0          0       7228          0          0 dwdm RPC RP proc

181   0        236        236       7228          0          0 netclk_process

182   0          0          0       7228          0          0 netclk_ha_proces

183   0        472        472       7228          7          8 SCP async: LCP#5

184   0        236        236       7228          0          0 CWAN ACD tasks

185   0       1884        472       8640          1          1 ACE_REQUESTS han

186   0      16668          0      41896          0          0 ACE Tunnel Task

187   0          0          0      25228          0          0 ACE VRF Task

188   0        472        472       7228          2          2 CMFI RP process

189   0          0          0       7228          0          0 CWAN CHOCX PROCE

190   0          0          0       7228          0          0 HAL Mailbox

191   0          0          0       7228          0          0 CWAN CHDSX PROCE

192   0          0          0       7228          0          0 CT3 Mailbox

193   0          0          0       7228          0          0 CE3 Mailbox

194   0        236        236       7228          0          0 MIP Mailbox

195   0          0          0       7228          0          0 Earl NDE Task

196   0    1873932    1873932       7228      15216      15216 esw_vlan_stat_pr

197   0        388        236       7380          0          0 Gemini L3 forwar

198   0          0          0       7228          0          0 CWRP SPA ATM OIR

199   0          0          0       7228          0          0 ACE event

200   0          0          0       7228          0          0 ACE HAPI

201   0          0          0       7228          0          0 ACE policy loade

202   0        236          0       4464          0          0 Async write proc

203   0        236          0       4464          0          0 Async write proc

204   0        236          0       4464          0          0 Async write proc

205   0        236          0       4464          0          0 Async write proc

206   0          0          0       4228          0          0 DFS flush period

207   0        236        236       7228          0          0 AAA Server

208   0          0          0       7228          0          0 AAA ACCT Proc

209   0      49384          0      56612          0          0 ACCT Periodic Pr

210   0          0          0       7228          0          0 AAA System Acct

211   0          0          0       7228          0          0 Auth-proxy AAA B

212   0          0          0       7228          0          0 IP Admin SM Proc

213   0       6576      13112      13452          0          0 Collection proce

214   0        224          0      13452          0          0 CEF switching ba

215   0        224          0       4452          0          0 ADJ NSF process

216   0     940704     940476      13456       8546       8546 ADJ resolve proc

217   0          0          0       7228          0          0 PPCP RP Stats Ba

218   0        236        236       7228          0          0 AAA Dictionary R

219   0          0          0       7228          0          0 DHCP Snooping cl

220   0          0          0       7228          0          0 DHCP Snooping

221   0      18372       1424      21124          1          6 IP ARP Adjacency

222   0      49384          0      56612          0          0 IP ARP Retry Age

223   0    1997100    1934388      36440     759858     759857 IP Input

224   0          0          0       7228          0          0 ICMP event handl

225   0        236        236       7228          0          0 PIM register asy

226   0          0          0       7228          0          0 IPv6 ping proces

227   0        236        236       7228        360        360 MOP Protocols

228   0        236        236       7228          0          0 PPP SIP

229   0        236        236       7228          0          0 PPP Bind

230   0        236        236      13228          0          0 PPP IP Route

231   0        584        584       7740        229        229 IFCOM Msg Hdlr

232   0          0          0       7228          0          0 HDLC HA

233   0      68868        236      81860          0          0 SSM connection m

234   0        472        236       7464      54699      54699 Spanning Tree

235   0      68100       2512      10228          6          6 EAPoUDP Process

236   0          0          0      13228          0          0 SSS Manager

237   0          0          0      13228          0          0 SSS Policy Manag

238   0          0          0       7228          0          0 SSS Feature Mana

239   0          0          0       7228          0          0 SSS Feature Time

240   0          0          0       7228          0          0 SG DPM

241   0          0          0       7228          0          0 Timer handler fo

242   0          0          0       7228          0          0 Prepaid response

243   0          0          0       7228          0          0 Timed Policy act

244   0          0          0       7228          0          0 AAA response han

245   0          0          0       7228          0          0 AAA EPD HANDLER

246   0          0          0       7228          0          0 PM EPD API

247   0        236        236      13228          0          0 KRB5 AAA

248   0        236        236       7228          0          0 cv6 per prefix a

249   0        236        236       7228          0          0 Const MGRE Proce

250   0          0          0       7228          0          0 CCM

251   0          0          0       7228          0          0 CCM IPC flow con

252   0          0          0      13228          0          0 DMLP Counter

253   0        236        236      10228          0          0 802.1x Accountin

254   0        236        236       7228          0          0 Dot1x Mgr Proces

255   0          0          0       7228          0          0 EAP Framework

256   0          0          0       7228          0          0 EAP Test

257   0        236        236      13228          0          0 Ethernet CFM

258   0        236        236      13228          0          0 Ethernet Timer C

259   0        236        236      13228          0          0 Ethernet Msec Ti

260   0        236        236      13228          0          0 Ethernet LMI

261   0       3052          0      16280          0          0 IMA PROC

262   0        236        236       7228          0          0 IPAM/ODAP Events

263   0      49384          0      62612          0          0 IPAM Manager

264   0        236        236      13228          0          0 IPAM Events

265   0     307568      44204     270592          0          0 EEM Server

266   0       8060       6848       8968         52         52 SW VLAN RP

267   0       1820          0       9048          0          0 const ipfib per-

268   0        472        236       7464          0          0 CFMPAL-RP Proces

269   0        472        236       7464          0          0 ECFMPAL RPC Proc

270   0        236        236       7228          0          0 ACR platform pro

271   0       9780        472       7136          2          2 OBFL Cfg Dispatc

272   0        236        236      10228          0          0 Ethernet OAM Pro

273   0      10224          0      11428          0          0 CEF RP IPC Backg

274   0      26656      13524      75076        106        106 HWIF QoS Process

275   0      33644      33996       9228          0          0 crypto sw pk pro

276   0       2624          0       9852          0          0 MLS FIB RP

277   0          0          0       7228          0          0 mcast_rp_icc Pro

278   0      86112        236      93104          0          0 MIRP-RP Process

279   0       1580        236       8572          0          0 const-mfib-rp Pr

280   0        236        236       7228          0          0 QOS Stats Export

281   0        236        236       7228          0          0 QOS Stats Gather

282   0          0          0       7228          0          0 Switch IP Admiss

283   0        236        236      10228          0          0 Switch IP Host T

284   0          0          0       7228          0          0 TCAM API Process

285   0        236        236       7228          0          0 REP Topology cha

286   0          0          0      13228          0          0 TC SIP

287   0        236        236       7228          0          0 IP PORTBUNDLE

288   0          0          0       7228          0          0 SG CMD HANDLER

289   0        236        236      13228          0          0 IP Static Sessio

290   0          0          0       7228          0          0 RADIUS Proxy

291   0          0          0       7228          0          0 vaccess_lock_pro

292   0          0          0       7228          0          0 va_lock_ha_proce

293   0       4728          0      11956          0          0 AC Switch

294   0        236        236