Ask the Expert: Catalyst 6500 Switch Architecture

ciscomoderator · ‎07-17-2012

With Akshay Balaganur

Welcome to the Cisco Support Community Ask the Expert conversation. This is an opportunity to learn from Cisco expert Akshay Balaganur about Cisco Catalyst 6500, one of the most widely deployed switches in the world. The "Swiss Army knife of network", can do routing, switching, security, wireless and almost everything that you would want your core switch to do.

Remember to use the rating system to let Akshay know if you have received an adequate response.

Akshay might not be able to answer each question due to the volume expected during this event. Remember that you can continue the conversation on the Network Infrastructure sub-community discussion forum shortly after the event. This event lasts through July 27, 2012. Visit this forum often to view responses to your questions and the questions of other community members.

sr1482613 · ‎07-18-2012

Hi ! Akshay !

This is Hyun-Goo Hank Kim.

Nice to meet you.

Actually, This is NOT about Catalyst 6500 Switch Architecture. It's about PFC.

When I was looking through a documentation about understanding of MSFC/PFC/DFC on Catalyst 6500 Switch, the documentation mentioned about EARL7 on PFC.

I just wanna know what EARL 7 is exactly.

Thanks.

Best Regards.

Hank.

Akshay Balaganur · ‎07-19-2012

Hi Hank,

EARL stands for Enhanced Address Resolution Logic. It is gives the 6500, the aility to do packet forwarding in hardware. EARL is nothing but a set of ASICs or CHIPs that does forwarding lookups at hardware level. The same job that a CPU or Proccesor does. By by doing forwarding lookups in hardware and not involving the CPU, helps us to achieve the Millions Packet/Sec rarte.

Let me give you some history about it. EARL was originally introduced in Catalyst 5000. Then came Catalyst 5500 , Catalyst 6000 and eventually Catalyst 6500. EARL has evolved in each generation. The SUp720 came woth PFC3, which hosts the EARL version 7. Now the latest SUpervisor SUP2T has PFC4 which is EARL8.

Cheers,

Akshay

harizmthaha · ‎07-22-2012

Hi

Akshay,

our 6513 switch doesn't supprts SSH. is this anything with the IOS versions?

Here's the show version result.

can u tell me whats the action should i take to resolve this..??

------------------ show version ------------------

Cisco IOS Software, s72033_rp Software (s72033_rp-IPSERVICES_WAN-M), Version 12.2(33)SXH2a, RELEASE SOFTWARE (fc2)

ROM: System Bootstrap, Version 12.2(17r)S4, RELEASE SOFTWARE (fc1)

System image file is "bootdisk:s72033-ipservices_wan-mz.122-33.SXH2a.bin"

cisco WS-C6513 (R7000) processor (revision 1.1) with 458720K/65536K bytes of memory.

Processor board ID SAL1223ST4U

SR71000 CPU at 600Mhz, Implementation 0x504, Rev 1.2, 512KB L2 Cache

Last reset from s/w reset

26 Virtual Ethernet interfaces

90 Gigabit Ethernet interfaces

1917K bytes of non-volatile configuration memory.

8192K bytes of packet buffer memory.

65536K bytes of Flash internal SIMM (Sector size 512K).

Configuration register is 0x2102

regards

Hariz

Leo Laohoo · ‎07-22-2012

our 6513 switch doesn't supprts SSH.

Your IOS doesn't support SSH because the IOS downloaded doesn't support "crypto".

Download the IOS with "crypto" support and SSH will be configurable.

Akshay Balaganur · ‎07-23-2012

Hi Hariz,

Yes your IOS is non-crypto image. You would need a crypto enabled image. The one that ends with "k9".

Example:

s72033-ipservicesk9_wan-mz.122-33.SXH2a.bin

harizmthaha · ‎07-23-2012

Hi Akshay & leolaohoo

thanks for the informations.

regards

Hariz

srprasaad_nj · ‎07-23-2012

Hi Akshay,

I am going through a source about 6509 and have got few doubts:

> Where does the IOS reside on 6509 , is it on SUP-Engine (32 or 720) with integrated MSFC and PFC cards?

> What is DataPlane and Control plane and how SUP engine controls both?

> Where CEF engine and CEF Forwarding table is located ?

What is functional difference between CFC, CEF720 crad, dCEF256/720 cards ? Ar

Aslo, see below output when i login to module 5, it says SP and shows only a basic config, so does this mean that I am on PFC card of mod 5 .

6509-CORE-1#sh mod
Mod Ports Card Type                              Model              Serial No.
--- ----- -------------------------------------- ------------------ -----------
1    8 CEF720 8 port 10GE with DFC            WS-X6708-10GE      SAL1532M7MD
3   24 CEF720 24 port 1000mb SFP              WS-X6724-SFP       SAL09444A1J
5    2 Supervisor Engine 720 (Active)         WS-SUP720-BASE     SAD072800PT
7   48 48-port 10/100/1000 RJ45 EtherModule   WS-X6148A-GE-TX    SAD094900WD

Mod MAC addresses                       Hw    Fw           Sw           Status
--- ---------------------------------- ------ ------------ ------------ -------
1 44d3.ca96.fcc0 to 44d3.ca96.fcc7   2.3   12.2(18r)S1 12.2(33)SXJ1 Ok
3 0015.629b.9fa0 to 0015.629b.9fb7   2.3   12.2(14r)S5 12.2(33)SXJ1 Ok
5 000d.290f.c1d0 to 000d.290f.c1d3   2.1   7.7(1)       12.2(33)SXJ1 Ok
7 0013.c4c2.c0c0 to 0013.c4c2.c0ef   1.2   8.4(1)       12.2(33)SXJ1 Ok

Mod Sub-Module                  Model              Serial       Hw     Status
---- --------------------------- ------------------ ----------- ------- -------
1 Distributed Forwarding Card WS-F6700-DFC3C     SAL1532LUDB 1.4    Ok
3 Centralized Forwarding Card WS-F6700-CFC       SAL09528ZT6 2.0    Ok
5 Policy Feature Card 3       WS-F6K-PFC3A       SAD072704UL 1.1    Ok
5 MSFC3 Daughterboard         WS-SUP720          SAD072801SC 1.2    Ok

Mod Online Diag Status
---- -------------------
1 Pass
3 Pass
5 Pass
7 Pass

.

6509-CORE-1#remote login module 5
Trying Switch ...
Entering CONSOLE for Switch
Type "^C^C^C" to end this session

6509-CORE-1-sp#

Akshay Balaganur · ‎07-23-2012

1. Where does the IOS reside on 6509 , is it on SUP-Engine (32 or 720) with integrated MSFC and PFC cards?

Yes the IOS is mostly stored in the bootflash located on the MSFC of the Supervisor.

It can also be stored in an external compact flash.

2. What is DataPlane and Control plane and how SUP engine controls both?

Lets look at how 6500 is designed. And tehn you will get your answer !

Like any other router the 6500 builds its Routing and CAM table. But unlike routers, 6500 pushes this information into hardware chips called TCAMs( PFC/DFC). Now the PFC/DFC based on this information have the capability to forward the packets independently without engaging the CPU. NOw this forms the Data Plane.

CPU is now not bothered about the data forwarding, since it is being taken care by PFC/DFCs. So CPU can continue handling the management tasks like maintaining the protocols (routing, switching, cdp, vtp etc..), monitoring the Chassis and the heardware. This forms the Control Plane.

Since we have separeated the Control plane and Data Plane, it enables us to have a high availability feature when using the 6500.

3. Where CEF engine and CEF Forwarding table is located ?

Let us revisit our previous answer with a little more details. :-)

The CPU first constructs the routing table and ARP tables. These are compressed again made as FIB and Adjaceny tables in CEF (Software CEF). Next FIB and Adj tables are downloaded from the processor and programmed as HW FIB and HW adjacency in the HW TCAMs. HW FIB and HW Adj are also known as MLS CEF (Multi Layer Switching CEF).

So the question is a tricky one. We have CEF in both software and hardware. But you can safley say that CEF is programmed in hardware, when referencing to 6500. Because you use MLS CEF the most in 6500.

4. What is functional difference between CFC, CEF720 card, dCEF256/720 cards ?

I would suggest you to go through the following White paper which clearly explains the architecture of Different categories of line cards.

http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/prod_white_paper0900aecd80673385.html

If I have to explain in short, it would be like this.

Classic Line cards- Utilize the 32 Gbps shared bus.

Cef256 - Utilize the 8Gbps dedicated fabric channel for data communication.

Cef720 - Utilize the 20Gbps dedicated fabric channel for data commanucation.

dcef256- Cef256 Line card equipped with a DFC daughter card.

dcef720- Cef720 Line card equipped with a DFC daughter card.

CFC is a daughtercard that provides centralized forwarding for the 67xx linecards(Cef720). The CFC is the base requirement for 67xx linecard operation and is a zero cost option. The daughtercard houses two ASICs that function only as a bus interface. As the name implies, the CFC is only used for centralized forwarding.

Akshay Balaganur · ‎07-24-2012

Hi ,

I missed to notice your last question. When you do a remote login module 5, you are actaully logging into the Switch processor. And hence you see "SP" at the prompt. It stands for the Switch Processor.

Cheers,

Akshay

srprasaad_nj · ‎07-24-2012

Hi Akshay,

Thanks.. for your reply.

Please correct me if i am wrong...does SP and RP reside on MSFC or PFC cards

And is there anyway to login to RP as well ?

What config's we will see in SP and RP ?

Akshay Balaganur · ‎07-25-2012

Hi Sitarama,

Yes both the RP and SP reside on the MSFC.

When you are running in Native mode , you login to RP by default. If you wish to go to SP , you can use the "remote switch login" command. In Native mode you do all your configurations on RP only. You would go to SP only check few show command outputs.

If you are running in hybrid mode ( IOS on RP and CatOS on SP), then it is the otherway round. You by default login to SP. To login to RP you can use a "session 15" command.

In hybrid mode , you will do the layer 2 configurations like spanning-tree , vlans , trunking etc on SP. And you will do all the layer 3 stuff on RP.

It gets little confusing with Native and Hybrid modes. Feel free to post if you have any doubts with this.

Hope it helps. Please rate if you find it usefull.

-Akshay

ajay chauhan · ‎07-24-2012

Hi,

How do we calculate vlan-port instances on 6500 ? what it has to do with STP instances? Also max limit depends upon what factors.

Thanks

Ajay

Akshay Balaganur · ‎07-24-2012

Hi Ajay ,

  I think it's best to start with defining what exactly counts as
"vlan-port"
 
  The sum of all logical interfaces (vlan-ports) = [Number of trunks on
switch x Number of active Vlans on trinks] + Number of non trunking
interfaces on the switch 
 
  Also, VLANs can be counted more than once; if VLAN 1 is carried on ten
interfaces, it will count as ten VLAN ports
 
  Now with that explained, let's look into the number that you are
getting. The number is a software/hardware limitation and is mainly
dependant on the kind of spanning tree that you are running in your
network. Pasted below are the different possibilities, the first number
is the total vlan-ports on the whole chassis, the second one is per
module, which is what you are hitting
 
         PVST+ 13,000 total 1,800/module
         RPVST+ 10,000 total 1,800/module
         MST 50,000 total 6,000/module

Could you please provide more information about your device like the Supervisor you are using and the IOS image being run.

1. show vlan virtual-port  
( This command is what you are looking for ! )

2. show spanning-tree summary totals
( Command displays the number of logical interfaces in the STP Active column.)

Cheers,
Akshay

huangedmc · ‎07-24-2012

hi Akshay,

When we cable uplinks between switches/routers, we try to fan out the redundant ports, so that they're on different ASIC's. We were able to find the port to ASIC mapping's via Networkers slide decks for the Nexus 7K's, but the ones for Cat 6K's are too generalized.

I already checked out the Cat 6500 Architecture Whitepaper, and it too didn't show the exact ASIC to port layout for the CEF720 cards. (most of our line cards are 6748's or 6724's)

Could you please provide URL to reference that shows the ASIC to port mapping?

thanks,

Kevin