cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
24594
Views
0
Helpful
27
Replies

High CPU Utilization on 7606

sabuz_banik
Level 1
Level 1

Dear concern,

Our Router  7606 is in a problem with high CPU Utilization. We are using one EBGP  and one IBGP session. We are also running OSPF with area 0 between two  IBGP router to exchange few local networks. When bandwidth cross 100 MB  than only CPU use become 100%. when bandwidth consumption is 80MB CPU  use become 80% end so on.

Pls find the bellow out put and assist us.

----------------------------------------------------------------------------------------------------------------------

Out put is when bandwidth consumption is 34MB

----------------------------------------------------------------------------------------------------------------------

CPU utilization for five seconds: 44%/43%; one minute: 42%; five minutes: 30%

PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process

  78          44        9625          4  0.39%  0.03%  0.00%   2 Virtual Exec

  13          40         375        106  0.07%  0.00%  0.00%   0 ARP Input

  30         116        1098        105  0.07%  0.01%  0.00%   0 IPC Seat Manager

164         524       42660         12  0.07%  0.04%  0.05%   1 Virtual Exec

186           0       22945          0  0.07%  0.03%  0.02%   0 ACE Tunnel Task

259          12       89401          0  0.07%  0.10%  0.08%   0 Ethernet Msec Ti

   1           0           8          0  0.00%  0.00%  0.00%   0 Chunk Manager

   2           4         156         25  0.00%  0.01%  0.00%   0 Load Meter

   4           0          47          0  0.00%  0.00%  0.00%   0 SCP async: Draco

   5           0         173          0  0.00%  0.00%  0.00%   0 Retransmission o

   6           0           3          0  0.00%  0.00%  0.00%   0 IPC ISSU Dispatc

   7           0           1          0  0.00%  0.00%  0.00%   0 PF Redun ICC Req

   8        8684         466      18635  0.00%  0.86%  0.81%   0 Check heaps

   9           0           1          0  0.00%  0.00%  0.00%   0 Pool Manager

  10           0           1          0  0.00%  0.00%  0.00%   0 DiscardQ Backgro

  11           0           2          0  0.00%  0.00%  0.00%   0 Timers

  12           4         226         17  0.00%  0.00%  0.00%   0 WATCH_AFS

  14           8         853          9  0.00%  0.00%  0.00%   0 ARP Background

  15           0           2          0  0.00%  0.00%  0.00%   0 ATM Idle Timer

  16           0           1          0  0.00%  0.00%  0.00%   0 ATM ASYNC PROC

  End = e   Freeze = f

Enter Command:

-------------------------------------------------------------------------------------------------------------------------------

RTLSGR1#sh memory statistics

                Head    Total(b)     Used(b)     Free(b)   Lowest(b)  Largest(b)

Processor   1322CA94   1692218732   522133152   1170085580   1168835904   1169364236

      I/O   78000000   134217728    50319772    83897956    83897956    83896252

---------------------------------------------------------------------------------------------------------------------------------------

RTLSGR1#sh cef drop

% Command accepted but obsolete, see 'show (ip|ipv6) cef switching statistics [feature]'

IPv4 CEF Drop Statistics

Slot  Encap_fail  Unresolved Unsupported    No_route      No_adj  ChkSum_Err

RP             0           0         422          10           4           0

5/0            0           0           0           0           0           0

3/0            0           0           0           0           0           0

RTLSGR1#

-------------------------------------------------------------------------------------------------------------

SUP

RSP720-3C-GE

--------------------------------------------------------------------------------------------------------------------

IOS

c7600rsp72043_rp-ADVENTERPRISEK9-M

---------------------------------------------------------------------------------------------------------------------

Regards

27 Replies 27

Here is the issue

"***** FIB is in exception state for IPv4 unicast *****"

You FIB has gone into exception state. This could be due to some flood of prefixes in the past.

To check the current FIB TCAM usage, you can try the command " show platform hardware capacity forwarding"

To recover from this, we need to Reload the router.

Afte reload check the status using " show mls cef exception status"

I thoougt it was an IOS bug and replaced the IOS and restarted once but found same problem. Do i need to go with hard reboot?

RTLSGR1#sh platform hardware capacity forwarding

L2 Forwarding Resources

           MAC Table usage:   Module  Collisions  Total       Used       %Used

                              5                0  98304          6          1%

             VPN CAM usage:                       Total       Used       %Used

                                                    512          0          0%

L3 Forwarding Resources

Module              FIB TCAM usage:                     Total        Used     %Used

   5                     72 bits (IPv4, MPLS, EoM)      196608      196194    100%

                        144 bits (IP mcast, IPv6)       32768            9      1%

                     detail:      Protocol                    Used       %Used

                                  IPv4                      194146         99%

                                  MPLS                        2048          1%

                                  EoM                            0          0%

                                  IPv6                           2          1%

                                  IPv4 mcast                     4          1%

                                  IPv6 mcast                     3          1%

            Adjacency usage:                     Total        Used       %Used

                                               1048576         217          1%

     Forwarding engine load:

                     Module       pps   peak-pps                     peak-time

                     5         340819     757369  16:31:57 UTC Sun Sep 30 2012

RTLSGR1#

It is not a Bug

Module              FIB TCAM usage:                     Total        Used     %Used

   5                     72 bits (IPv4, MPLS, EoM)      196608      196194    100%

You FIB TCAM is 100% utilized

For non-XL mode systems:

IPv4 unicast and MPLS—192,000 routes 

IPv6 unicast and IPv4 multicast—32,000 routes

reference link:

http://www.cisco.com/en/US/docs/ios/ipswitch/command/reference/isw_i1.html#wp1087660

one option is to reduce the number of ipv4 prefixes you receive

Another option is to reduce the FIB TCAM used for ipv6 and multicast if you are not using them so that ipv4 FIB TCAM gets more TCAM space

example commands

mls cef maximum-routes ip-multicast 2

mls cef maximum-routes ipv6 2

This will release 28Kb of TCAM which can be used for IPv4.

This requires reload of the router. Also please keep only one boot statement when you boot

Check show ip route summary to see the number of ipv4 prefixes you have

OK, Let me check. I will update you shortly.

Shall i do the same for MPLS aswell ? As i am not running any MPLS network.

MPLS is using only 2k. You may reduce it to 1k

How is your show ip route summary looking? D owe have more than 200K routes?

OK I will make it 1K. I am unable to reboot now as device is running in a production.Watch my IP route Sum. I am unable to filter BGP table as source of carrier can be any real ip.

RTLSGR1#sh ip route summary

IP routing table name is default (0x0)

IP routing table maximum-paths is 32

Route Source    Networks    Subnets     Replicates  Overhead    Memory (bytes)

static          3           2           0           300         860

connected       0           6           0           400         1032

ospf 1          1           2           0           240         528

  Intra-area: 3 Inter-area: 0 External-1: 0 External-2: 0

  NSSA External-1: 0 NSSA External-2: 0

bgp 58636       152873      268794      0           25300020    72526724

  External: 421660 Internal: 7 Local: 0

internal        4818                                            17317728

Total           157695      268804      0           25300960    89846872

RTLSGR1#

Hi

You have more than 260K routes. 

Your Total FIB TCAM is 252 K inluding ipv6, mpls,ipv6 and multicast. So reducing the TCAM for IPV6 and multicast is not going to help

You need to redue the routes you are receving on this router.

Hi Rajs,

Is there any way to increase Tcam Size ?

How shall i reduce route/prefixes as source ip is not fixed?

Shall i replace the SUP?

If i connect with ASR1002-10G router will it be able to handle this much route ?

Any way to stick with this router with tuning guideline ?

Hi Sabuz,

If your requirement is to receive complete BGP routes, RSP 720-3C is not the right SUP for you

You can't increase the TCAM size. if you change the SUP to RSP 720-3CXL you will be able to handle upto 1M routes

Here is the comparison

http://www.cisco.com/en/US/prod/collateral/routers/ps368/product_data_sheet0900aecd8057f3b6.html

ASR1K will be able to handle it

http://www.cisco.com/en/US/prod/collateral/routers/ps9343/data_sheet_c78-441072.html

If you want to stick with the same router, either you need to reduce the routes or change the SUP to 3CXL

Hi Raj,

It was nice having views from you. Thanks a lot for your kind help.I hope your extened help will continue in future aswell.

Have a very nice time.

Thanks again.

Hi Sabuz,

Sure. You are welcome. Wish you nice time

Please don't forget to mark the query as answered if you are satisfied with the answer

Review Cisco Networking for a $25 gift card