Maximum routes in VRF vs Command sh mls cef vrf

Plinio Brandao · ‎12-21-2012

Hi Everybody,

I'd like to take some doubts about the maximum route command under vrf configuration and about the total of routes in table and in the specific vrf.

I have a 7600 configured with a VRF called INTERNET and the following config at this device:

ip vrf INTERNET

rd 1:1

route-target export 1:1

route-target import 1:1

maximum routes 170000 85

router bgp 65041

no synchronization

no bgp enforce-first-as

bgp log-neighbor-changes

no auto-summary

!

address-family ipv4 vrf INTERNET

no synchronization

network Y.Y.Y.Y mask 255.255.240.0

neighbor X.X.X.X remote-as 65023

neighbor X.X.X.X maximum-prefix 170000 85 warning-only

According to the documentation, the maximum-prefix command in bgp configuration is clear for me. I receive more that 170k and the traps are generates:

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd

X.X.X.X 4 65023 4664116 73547 24175815 0 0 6w4d 172552

The maximum route command in the VRF configuration is also clear for me. But I don't know how to confirm that these numbers are being respected. In a personal lab, I simulated the scenario with just 10 routes and no more routes are included in the routing table, as should be.

In the real scenario, I use the following commands to check the total of routes:

ROUTER#sh mls cef vrf INTERNET summary

Protocol     : IPv4
VPN mask     : 0xFFF
VPN value    : 0x101
TOS mask     : 0x0

Total Routes : 170390

What this total of routes means? How can I check the total of routes installed in the routing table using the command: sh ip route vrf INTERNET?

ROUTER#sh ip route vrf INTERNET summary

IP routing table name is INTERNET (0x5)

IP routing table maximum-paths is 32

Route Source Networks Subnets Replicates Overhead Memory (bytes)

connected 0 9 0 468 1548

static 1 5 0 312 1032

bgp 65041 60690 109291 0 8839012 29236732

External: 163528 Internal: 6453 Local: 0 (Is this the total installed?)

internal 2753 7200860

Total 63444 109305 0 8839792 36440172

What's the difference between sh mls cef vrf INTERNET summary and sh ip route vrf INTERNET summary?

How can I get the correct total number of routes installed?

Thank you in advanced.

Plinio Monteiro

Mark Combs · ‎12-22-2012

Plinio,

I think there is a confusion between prefix and subnets. Your allowing 170k prefixes which could include a ny number of subnets or maybe I'm confused Your output shows your total prefix count of 60690 and there is approxmiately 109,291 subnets associated with these prefixes. So your way under your prefix limit by almost 100k.

The "sh mls cef vrf INTERNET summary" is just your l3 switching table for all protocols in use for your vrf INTERNET. The 7600 are called routers but i think they are nothing more than a fancy upgrade to the 6500 platform with a gray color thrown on them to make them stand out.

I see your output has connected, static, and BGP. Throw OSPF into the vrf and you will see that show up also if there are routes populating the cef table. The total routes shown in your "sh mls cef vrf INTERNET summary" command will be your complete total number of routes in your vrf INTERNET for everything.

The "sh ip route vrf INTERNET summary" is your routing table or "RIB" in this case. I'm going to make up a new word and call it the VRIB. Basically this is the difference between what your receiving and what your allowing into your actual routing table. Example would be if you had some type of filtering going on one of your routing protocols. You would see it in your "VCEF" but not your "VRIB". If your not doing any inbound filtering these should be pretty close if you add everything up. Routes are constantly coming in or leaving with routing tables this large so you have to be quick to run both commands if the math is to add up. I'm receiving the full routing table on my 7600 and the numbers came out correct when I added them up. Almost 500k routes

Full explanation:

The Subnets "6453" your asking about is NOT the number of subnets in your routing table. This is just saying that out of your total amount that 6,453 was internal and 163,528 was external.. The subnets section is the number of subnets and the network section is the number of prefixes in your routing table. The other number 2753 (internal) number is the number of bgp routes marked with an "i" for internal but for all of your ipv4 and vpnv4 address families that are in this VRF. I do not believe you can do a | count on the 7600 platform. But on a router I was able to perform the following:

This counted the number of lines marked with >i for all my address families. This matched up 100% to what was showing in my summary commands.

sh ip bgp all | count >i

Number of lines which match regexp = 5sh ip bgp all | count >i
Number of lines which match regexp = 5

Let's break out the math in your example for subnets:

Connected Subnets: 9

Static 5

BGP routes 109291

------------------------------------------------

109305 total subnets in your routing table (this doesnt include prefixes)

Now for Prefixes:

Connected Subnets: 0

Static 1

BGP prefixes 60690

internal prefixes 2753 Total internal bgp routes between all vpnv4 and ipv4 address families.

------------------------------------------------

63444 Total prefixes

I hope this helps.

-Mark Combs

Mark Combs · ‎12-29-2012

Plinio,

Just following up to see if this explanation helped or not? You kind of fell of the radar after your question.

- Mark

Plinio Brandao · ‎12-31-2012

Hi Mark,

Sorry for my delay. I was in a troble with an ACE issue.

I understood your explanation. The only thing that I can't understand is if I have a limit set in my vrf configuration of 170k, why the output of sh mls cef vrf INTERNET summary shows more the 170k.

ip vrf INTERNET

rd 1:1

route-target export 1:1

route-target import 1:1

maximum routes 170000 85

I simulated using GNS3 and I generated using BGP, static and connected routes. The result was if the neighbor advertises a new route and I already had reached the limit, I'll not learning the new route/prefix.

Another question is:

This is the total of routers supported by 7600 plataform right? So, when the value exceeds 170k I'd be hanging by a thread to have an overflow of my TCAM, right? I'm telling you this because I already have a burst of routes on this equipment.

ROUTER#sh mls cef maximum-routes

FIB TCAM maximum routes :

=======================

Current :-

-------

IPv4 + MPLS - 192k (default)

IPv6 + IP Multicast - 32k (default)

Mark Combs · ‎12-31-2012

Plinio,

Is is possible to provide the output of a "show module" command on your 7600? or just tell me what type of supervisor do you have in your 7600 and what are all the DFCs in use?

If you have a mix match of DFCs in your chassis the 7600 will take the lowest common demoninator. This will and probably is dramatically decreasing your performance of your 7600. You should be able to support 512k (default) routes or you can change this to support up to 1,000,000 routes using the mls cef maximum routes command, well at least my my version of card you can.

There is a previous post on this from some time ago. I included the links to the post as well as the Cisco doc that talks about the max number of routes for the various SUP engines. I have a sup720 3CXL-GE with a DFC3CXL. Below is my output of the show mls cef.

Cisco documentation:

http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/product_data_sheet09186a0080159856_ps4835_Products_Data_Sheet.html

My Output:

hostname_removed#sh mls cef maximum-routes

FIB TCAM maximum routes :

=======================

Current :-

-------

IPv4 + MPLS - 512k (default)

IPv6 + IP Multicast - 256k (default)

hostname removed#sh mls cef summary

Total routes:                     435795
    IPv4 unicast routes:          433740
    IPv4 Multicast routes:        4
    MPLS routes:                  2049
    IPv6 unicast routes:          2
    IPv6 multicast routes:        3
    EoM routes:                   0

hostname_removed#sh mls cef vrf INTERNET summary

Protocol     : IPv4
VPN mask     : 0xFFF
VPN value    : 0x100
TOS mask     : 0x0

Total Routes : 433711

Plinio Brandao · ‎01-03-2013

Hi Mark,

I agree with you. I already worked with 7600 and the default was 512k. In this actual scenario, I have a limit of 192 and sometimes I reach 172k routes, even if I use the limit route int vrf configuration. And like I told you, we already had a FIB overflow crashing the 7600.

Se my show module output:

ROUTER#sh module
Mod Ports Card Type                              Model              Serial No.
--- ----- -------------------------------------- ------------------ -----------
1   48 CEF720 48 port 10/100/1000mb Ethernet WS-X6748-GE-TX
2   24 CEF720 24 port 1000mb SFP              WS-X6724-SFP
4    8 CEF720 8 port 10GE with DFC            WS-X6708-10GE
5    2 Route Switch Processor 720 (Active)    RSP720-3C-GE
6    2 Route Switch Processor 720 (Hot)       RSP720-3C-GE
9    6 Firewall Module                        WS-SVC-FWM-1

Mod MAC addresses                       Hw    Fw           Sw           Status
--- ---------------------------------- ------ ------------ ------------ -------
1 0025.455f.37e4 to 0025.455f.3813   3.0   12.2(18r)S1 12.2(33)SRD6 Ok
2 0025.4558.0e38 to 0025.4558.0e4f   4.1   12.2(18r)S1 12.2(33)SRD6 Ok
4 0025.4575.ceb4 to 0025.4575.cebb   1.6   12.2(18r)S1 12.2(33)SRD6 Ok
5 001c.584c.f2a8 to 001c.584c.f2ab   5.9   12.2(33r)SRD 12.2(33)SRD6 Ok
6 001c.584c.f32c to 001c.584c.f32f   5.9   12.2(33r)SRD 12.2(33)SRD6 Ok
9 001f.ca7b.9de2 to 001f.ca7b.9de9   4.5   7.2(1)       4.0(7)       Ok

Mod Sub-Module                  Model              Serial       Hw     Status
---- --------------------------- ------------------ ----------- ------- -------
1 Distributed Forwarding Card WS-F6700-DFC3C     1.2    Ok
2 Distributed Forwarding Card WS-F6700-DFC3C     1.2    Ok
4 Distributed Forwarding Card WS-F6700-DFC3C     1.2    Ok
5 Policy Feature Card 3       7600-PFC3C         1.2    Ok
5 C7600 MSFC4 Daughterboard   7600-MSFC4         1.4    Ok
6 Policy Feature Card 3       7600-PFC3C         1.2    Ok
6 C7600 MSFC4 Daughterboard   7600-MSFC4         1.4    Ok

Mod Online Diag Status
---- -------------------
1 Pass
2 Pass
4 Pass
5 Pass
6 Pass
9 Pass
ROUTER#

ROUTER#sh mls cef summary

Total routes:                     141556
    IPv4 unicast routes:          141550
    IPv4 Multicast routes:        3
    MPLS routes:                  1
    IPv6 unicast routes:          2
    IPv6 multicast routes:        3
    EoM routes:                   0
ROUTER#

ROUTER#sh mls cef vrf INTERNET summary

Protocol     : IPv4
VPN mask     : 0xFFF
VPN value    : 0x101
TOS mask     : 0x0

Total Routes : 137032
ROUTER#

Thank you in advance.