Upgrade from PFC3C to PFC3CXL

[Zeeeee]

Hi,

I am seeking kind advises from experts before upgrading my 4 x sup720 engines from VS-S720-10G-3C to VS-S720-10G-3CXL. The 2 routers that I have are on VSS too.

Below is the existing setup.

Router A:

1 CEF720 48 port 10/100/1000mb Ethernet WS-X6748-GE-TX 
2 CEF720 48 port 10/100/1000mb Ethernet WS-X6748-GE-TX
5 Supervisor Engine 720 10GE (Active) VS-S720-10G 
6 Supervisor Engine 720 10GE (CSSO) VS-S720-10G 
8 CEF720 16 port 10GE WS-X6716-10GE
9 CEF720 16 port 10GE WS-X6716-10GE

1 Centralized Forwarding Card WS-F6700-CFC
2 Centralized Forwarding Card WS-F6700-CFC
5 Policy Feature Card 3 VS-F6K-PFC3C
5 MSFC3 Daughterboard VS-F6K-MSFC3
6 Policy Feature Card 3 VS-F6K-PFC3C
6 MSFC3 Daughterboard VS-F6K-MSFC3
8 Distributed Forwarding Card WS-F6700-DFC3C
9 Distributed Forwarding Card WS-F6700-DFC3C

Router B:

1 48 CEF720 48 port 10/100/1000mb Ethernet WS-X6748-GE-TX
5 5 Supervisor Engine 720 10GE (CSSO) VS-S720-10G
6 5 Supervisor Engine 720 10GE (Hot) VS-S720-10G
9 16 CEF720 16 port 10GE WS-X6716-10GE

1 Centralized Forwarding Card WS-F6700-CFC 
5 Policy Feature Card 3 VS-F6K-PFC3C 
5 MSFC3 Daughterboard VS-F6K-MSFC3 
6 Policy Feature Card 3 VS-F6K-PFC3C 
6 MSFC3 Daughterboard VS-F6K-MSFC3
9 Distributed Forwarding Card WS-F6700-DFC3C

Here are the questions.

1. From Cisco datasheet, it was mentioned that VS-S720-10G-3C can hold up to 256,000 (ipv4) routes. I was wondering why my existing setup can only hold 192,000 (ipv4) routes (I got 192,000 when I did show mls cef maximum-routes). The reason for asking is, if I change my sup720 engines to 3CXL, will I be able to hold 1,000,000 (ipv4) routes as stated in the datasheet? Or will it be lesser like how I experience right now with 3C?

2. I understand that I need to also change Distributed Forwarding Card WS-F6700-DFC3C modules to 3CXL to enjoy 3CXL features. How about CEF720 48 port 10/100/1000mb Ethernet modules? It seems like there is nothing I can change with these modules since it will not be impacted.

Looking forward to your advises! Happy new year to everyone in this forum.

Thank you,

Aizat

[educruz · Accepted solution]

Good day,

For question #1, in reality, the default FIB TCAM resources permit 192,000 IPv4 unicast AND MPLS routes; and 32,000 IPv6 unicast and IPv4 multicast routes. This space can be changed though with the command "mls cef maximum-routes", but a reload is required.

For question #2, when switching from non-XL to XL, there will be more space to hold IPv4 routes. For more information, you could review this document.

https://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-switches/117712-problemsolution-cat6500-00.html

As far as I know, you do not need to change your CFC cards. I have seen a mixture of XL cards and CFC cards work in a C7600, which shares a very similar hardware and software architecture of the Cat6500 Sup720. 

But like you said, you do need to change your DFC cards to XL, because the system as a whole (chassis) check all the PFC/DFCs and have them operate to the capacity of the lowest DFC card.

Switch# show module
Mod Ports Card Type                              Model 
Serial No.
--- ----- -------------------------------------- ------------------ -----------
   1    4  CEF720 4 port 10-Gigabit Ethernet      WS-X6704-10GE 
   2    8  CEF720 8 port 10GE with DFC            WS-X6708-10GE 
   3    4  CEF720 4 port 10-Gigabit Ethernet      WS-X6704-10GE 
   4    4  CEF720 4 port 10-Gigabit Ethernet      WS-X6704-10GE 
   5   48  CEF720 48 port 10/100/1000mb Ethernet  WS-X6748-GE-TX 
   6    2  Route Switch Processor 720 (Active)    RSP720-3CXL-GE 


Mod  Sub-Module                  Model              Serial       Hw 
Status
---- --------------------------- ------------------ ----------- ------- 
-------
   1  Distributed Forwarding Card WS-F6700-DFC3CXL   1.0    Ok
   2  Distributed Forwarding Card WS-F6700-DFC3CXL   1.1    Ok
   3  Distributed Forwarding Card WS-F6700-DFC3CXL   1.6    Ok
   4  Centralized Forwarding Card WS-F6700-CFC       2.0    Ok
   5  Centralized Forwarding Card WS-F6700-CFC       4.1    Ok
   6  Policy Feature Card 3       7600-PFC3CXL       1.0    Ok
   6  C7600 MSFC4 Daughterboard   7600-MSFC4         1.1    Ok

Switch#show platform hardware pfc mode
PFC operating mode : PFC3CXL

Hope this helps.

Eduardo.

[Zeeeee]

Hi Eduardo,

Your answer for Question 2 is well understood. Thank you.

For Question 1, correct me if I am wrong please, it means that in reality, it can hold 256,000 of ipv4 routes if I adjust the others (MPLS, IPv6 and IP Multicast) to 0.

So for 3CXL, 1,000,000 routes is a mixture of IPv4, MPLS, IPv6 and IP Multicast, and I have to adjust accordingly to achieve the number of IPv4 routes that I require. Also, I need to take note that IPv6 routes consume 2 TCAM entries.

Thank you.

Aizat

[educruz]

Good day,

For question #1, I believe the maximum amount of routes will be 239,000 and never 256,000. I assume this is either some scaling limitation of the platform, or the "difference" between 239k and 256k must be reserved for some other use, and that cannot be modified. 

However; yes, you can configure your switch to save space for more IPv4 routes, and as a side effect it reduces the space for IPv6 or multicast routes. Please be aware because you may not use IPv6 in your environment, but you might do for multicast.

On the other hand, yes, an IPv6 route will use two entries in TCAM.

Hope this helps,

Eduardo.

[Joseph W. Doherty]

"How about CEF720 48 port 10/100/1000mb Ethernet modules? It seems like there is nothing I can change with these modules since it will not be impacted."

Correct because they have the CFC modules.

BTW, I recall you need to power cycle the whole chassis to take full advantage of the non-XL to XL switch.

[Zeeeee]

Hi Joseph,

Got it. Thank you for the tip!

Aizat