ā09-29-2012 10:47 AM - edited ā03-04-2019 05:42 PM
Dear concern,
Our Router 7606 is in a problem with high CPU Utilization. We are using one EBGP and one IBGP session. We are also running OSPF with area 0 between two IBGP router to exchange few local networks. When bandwidth cross 100 MB than only CPU use become 100%. when bandwidth consumption is 80MB CPU use become 80% end so on.
Pls find the bellow out put and assist us.
----------------------------------------------------------------------------------------------------------------------
Out put is when bandwidth consumption is 34MB
----------------------------------------------------------------------------------------------------------------------
CPU utilization for five seconds: 44%/43%; one minute: 42%; five minutes: 30%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
78 44 9625 4 0.39% 0.03% 0.00% 2 Virtual Exec
13 40 375 106 0.07% 0.00% 0.00% 0 ARP Input
30 116 1098 105 0.07% 0.01% 0.00% 0 IPC Seat Manager
164 524 42660 12 0.07% 0.04% 0.05% 1 Virtual Exec
186 0 22945 0 0.07% 0.03% 0.02% 0 ACE Tunnel Task
259 12 89401 0 0.07% 0.10% 0.08% 0 Ethernet Msec Ti
1 0 8 0 0.00% 0.00% 0.00% 0 Chunk Manager
2 4 156 25 0.00% 0.01% 0.00% 0 Load Meter
4 0 47 0 0.00% 0.00% 0.00% 0 SCP async: Draco
5 0 173 0 0.00% 0.00% 0.00% 0 Retransmission o
6 0 3 0 0.00% 0.00% 0.00% 0 IPC ISSU Dispatc
7 0 1 0 0.00% 0.00% 0.00% 0 PF Redun ICC Req
8 8684 466 18635 0.00% 0.86% 0.81% 0 Check heaps
9 0 1 0 0.00% 0.00% 0.00% 0 Pool Manager
10 0 1 0 0.00% 0.00% 0.00% 0 DiscardQ Backgro
11 0 2 0 0.00% 0.00% 0.00% 0 Timers
12 4 226 17 0.00% 0.00% 0.00% 0 WATCH_AFS
14 8 853 9 0.00% 0.00% 0.00% 0 ARP Background
15 0 2 0 0.00% 0.00% 0.00% 0 ATM Idle Timer
16 0 1 0 0.00% 0.00% 0.00% 0 ATM ASYNC PROC
End = e Freeze = f
Enter Command:
-------------------------------------------------------------------------------------------------------------------------------
RTLSGR1#sh memory statistics
Head Total(b) Used(b) Free(b) Lowest(b) Largest(b)
Processor 1322CA94 1692218732 522133152 1170085580 1168835904 1169364236
I/O 78000000 134217728 50319772 83897956 83897956 83896252
---------------------------------------------------------------------------------------------------------------------------------------
RTLSGR1#sh cef drop
% Command accepted but obsolete, see 'show (ip|ipv6) cef switching statistics [feature]'
IPv4 CEF Drop Statistics
Slot Encap_fail Unresolved Unsupported No_route No_adj ChkSum_Err
RP 0 0 422 10 4 0
5/0 0 0 0 0 0 0
3/0 0 0 0 0 0 0
RTLSGR1#
-------------------------------------------------------------------------------------------------------------
SUP
RSP720-3C-GE
--------------------------------------------------------------------------------------------------------------------
IOS
c7600rsp72043_rp-ADVENTERPRISEK9-M
---------------------------------------------------------------------------------------------------------------------
Regards
Solved! Go to Solution.
ā10-01-2012 06:30 PM
Here is the issue
"***** FIB is in exception state for IPv4 unicast *****"
You FIB has gone into exception state. This could be due to some flood of prefixes in the past.
To check the current FIB TCAM usage, you can try the command " show platform hardware capacity forwarding"
To recover from this, we need to Reload the router.
Afte reload check the status using " show mls cef exception status"
ā10-01-2012 10:41 PM
I thoougt it was an IOS bug and replaced the IOS and restarted once but found same problem. Do i need to go with hard reboot?
RTLSGR1#sh platform hardware capacity forwarding
L2 Forwarding Resources
MAC Table usage: Module Collisions Total Used %Used
5 0 98304 6 1%
VPN CAM usage: Total Used %Used
512 0 0%
L3 Forwarding Resources
Module FIB TCAM usage: Total Used %Used
5 72 bits (IPv4, MPLS, EoM) 196608 196194 100%
144 bits (IP mcast, IPv6) 32768 9 1%
detail: Protocol Used %Used
IPv4 194146 99%
MPLS 2048 1%
EoM 0 0%
IPv6 2 1%
IPv4 mcast 4 1%
IPv6 mcast 3 1%
Adjacency usage: Total Used %Used
1048576 217 1%
Forwarding engine load:
Module pps peak-pps peak-time
5 340819 757369 16:31:57 UTC Sun Sep 30 2012
RTLSGR1#
ā10-01-2012 11:31 PM
It is not a Bug
Module FIB TCAM usage: Total Used %Used
5 72 bits (IPv4, MPLS, EoM) 196608 196194 100%
You FIB TCAM is 100% utilized
For non-XL mode systems:
āIPv4 unicast and MPLSā192,000 routes
āIPv6 unicast and IPv4 multicastā32,000 routes
reference link:
http://www.cisco.com/en/US/docs/ios/ipswitch/command/reference/isw_i1.html#wp1087660
one option is to reduce the number of ipv4 prefixes you receive
Another option is to reduce the FIB TCAM used for ipv6 and multicast if you are not using them so that ipv4 FIB TCAM gets more TCAM space
example commands
mls cef maximum-routes ip-multicast 2
mls cef maximum-routes ipv6 2
This will release 28Kb of TCAM which can be used for IPv4.
This requires reload of the router. Also please keep only one boot statement when you boot
Check show ip route summary to see the number of ipv4 prefixes you have
ā10-02-2012 12:19 AM
OK, Let me check. I will update you shortly.
ā10-02-2012 12:33 AM
Shall i do the same for MPLS aswell ? As i am not running any MPLS network.
ā10-02-2012 12:48 AM
MPLS is using only 2k. You may reduce it to 1k
How is your show ip route summary looking? D owe have more than 200K routes?
ā10-02-2012 12:58 AM
OK I will make it 1K. I am unable to reboot now as device is running in a production.Watch my IP route Sum. I am unable to filter BGP table as source of carrier can be any real ip.
RTLSGR1#sh ip route summary
IP routing table name is default (0x0)
IP routing table maximum-paths is 32
Route Source Networks Subnets Replicates Overhead Memory (bytes)
static 3 2 0 300 860
connected 0 6 0 400 1032
ospf 1 1 2 0 240 528
Intra-area: 3 Inter-area: 0 External-1: 0 External-2: 0
NSSA External-1: 0 NSSA External-2: 0
bgp 58636 152873 268794 0 25300020 72526724
External: 421660 Internal: 7 Local: 0
internal 4818 17317728
Total 157695 268804 0 25300960 89846872
RTLSGR1#
ā10-02-2012 01:33 AM
Hi
You have more than 260K routes.
Your Total FIB TCAM is 252 K inluding ipv6, mpls,ipv6 and multicast. So reducing the TCAM for IPV6 and multicast is not going to help
You need to redue the routes you are receving on this router.
ā10-02-2012 01:47 AM
Hi Rajs,
Is there any way to increase Tcam Size ?
How shall i reduce route/prefixes as source ip is not fixed?
Shall i replace the SUP?
If i connect with ASR1002-10G router will it be able to handle this much route ?
Any way to stick with this router with tuning guideline ?
ā10-02-2012 02:08 AM
Hi Sabuz,
If your requirement is to receive complete BGP routes, RSP 720-3C is not the right SUP for you
You can't increase the TCAM size. if you change the SUP to RSP 720-3CXL you will be able to handle upto 1M routes
Here is the comparison
http://www.cisco.com/en/US/prod/collateral/routers/ps368/product_data_sheet0900aecd8057f3b6.html
ASR1K will be able to handle it
http://www.cisco.com/en/US/prod/collateral/routers/ps9343/data_sheet_c78-441072.html
If you want to stick with the same router, either you need to reduce the routes or change the SUP to 3CXL
ā10-02-2012 02:29 AM
Hi Raj,
It was nice having views from you. Thanks a lot for your kind help.I hope your extened help will continue in future aswell.
Have a very nice time.
Thanks again.
ā10-02-2012 02:36 AM
Hi Sabuz,
Sure. You are welcome. Wish you nice time
ā10-02-2012 05:06 AM
Please don't forget to mark the query as answered if you are satisfied with the answer
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide