Since the early 1990s, we’ve watched as the number of entries on the Internet routing table has steadily grown. It wasn’t that long ago (2008) that the table reached 256k routes, triggering action by network administrators to ensure the continued growth of the Internet. Now that the table has passed 500,000 routes, it’s time to start preparing for another significant milestone – the 512k mark.
Looking Ahead to 512k
As an industry, we’ve known for some time that the Internet routing table growth could cause Ternary Content Addressable Memory (TCAM) resource exhaustion for some networking products. TCAM is a very important component of certain network switches and routers that stores routing tables. It is much faster than ordinary RAM (random access memory) and allows for rapid table lookups.
Networking Product Implications
No matter who provides your networking equipment, it needs to be able to manage the ongoing growth of the Internet routing table. We recommend confirming and addressing any possible impacts for all devices in your network, not just those provided by Cisco. The products that could be affected include those with a default configuration supporting 512k routes. From Cisco’s perspective, this includes:
- Cisco Catalyst 6500 Switches
- Cisco 7600 Series Routers
- Cisco ASR 9000 Series Aggregation Services Routers configured with Trident-based line cards (typhoon-based line cards are not affected)
- Cisco ASR 1000 Series Aggregation Services Routers with 4GB (devices with 8GB or RAM or higher can scale to up to 1,000,000 routes)
The Good News – Workarounds Are Available!
Cisco has published information on several workarounds that can be applied by our customers, including changing the default configuration for affected devices. In some cases this may require a reload of the device or line card. See below for the links to this customer information.
Cisco Catalyst 6500/Cisco 7600 Series Supervisor Engine 720
The following document describes how to customize the forwarding information base (FIB) ternary content addressable memory (TCAM) on Catalyst 6500 switches that run the Supervisor Engine 720:
http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-switches/116132-problem-catalyst6500-00.html
This guidance is specific to the Supervisor Engine models SUP720-3BXL and SUP720-3CXL. The “non-XL” versions do not support more than 256,000 IPv4 routes.
The following documents include additional information about the Cisco Catalyst 6500 and the Cisco 7600 Series Supervisor Engine 720 capabilities, respectively.
http://www.cisco.com/c/en/us/products/collateral/switches/catalyst-6500-series-switches/product_data_sheet09186a0080159856.pdf
http://www.cisco.com/c/en/us/products/collateral/routers/7600-series-routers/product_data_sheet0900aecd8057f3b6.html
Cisco ASR 9000 Series Aggregation Services Routers
The following document describes workarounds available for the Cisco ASR 9000 Series Aggregation Services Routers. When a Trident-based line card reaches its prefix limit, the message %ROUTING-FIB-4-RSRC_LOW occurs, causing potential traffic loss on the line cards:
http://www.cisco.com/c/en/us/support/docs/routers/asr-9000-series-aggregation-services-routers/116999-problem-line-card-00.html
Cisco ASR 1000 Series Aggregation Services Routers
Cisco ASR 1000 Series Aggregation Services Routers with 4GB can scale to up to 500,000 IPv4 or IPv6 routes. Cisco ASR 1000 Series Aggregation Services Routers with 8GB of RAM or higher can scale to up to 1,000,000 routes. The following document provides an overview of the number of supported routes:
http://www.cisco.com/c/en/us/products/collateral/routers/asr-1000-series-aggregation-services-routers/data_sheet_c78-441072.html
Additional Workarounds
Route filtering and the use of a default route can also be used to decrease the number of routes in an affected device. Prefix lists can be used as an alternative to access lists in many BGP route-filtering commands. The use of prefix lists provides significant performance improvements when loading and performing route lookup of large routing tables. Additional information about BGP best practices and configuring prefix lists is available at:
http://www.cisco.com/web/about/security/intelligence/protecting_bgp.html#8
Security Considerations
The possibility of TCAM resource exhaustion at 512k routes is a known issue that we all know has been coming for some time. There is no related security vulnerability, and it cannot be easily triggered by a remote, untrusted user.
The following website is a great resource that provides the current state of the Internet routing table. This could help Cisco customers when configuring route filtering:
Implementing the recommended workarounds ahead of time will help your network avoid any performance degradation, routing instability, or impact to availability. Having just passed the 500,000 route milestone, now is the right time to ensure your network is prepared to manage a 512k entry internet routing table.
