cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
6792
Views
10
Helpful
21
Replies

Ask the Expert: Scaling Data Center Networks with Cisco FabricPath

ciscomoderator
Community Manager
Community Manager

With Hatim Badr and Iqbal Syed

Read the bioRead the bio

Welcome to the Cisco Support Community Ask the Expert conversation. This is an opportunity to learn and ask questions about the Cisco FabricPath with Cisco technical support experts Hatim Badr and Iqbal Syed. Cisco FabricPath is a Cisco NX-OS Software innovation combining the plug-and-play simplicity of Ethernet with the reliability and scalability of Layer 3 routing. Cisco FabricPath uses many of the best characteristics of traditional Layer 2 and Layer 3 technologies, combining them into a new control-plane and data-plane implementation that combines the immediately operational "plug-and-play" deployment model of a bridged spanning-tree environment with the stability, re-convergence characteristics, and ability to use multiple parallel paths typical of a Layer 3 routed environment. The result is a scalable, flexible, and highly available Ethernet fabric suitable for even the most demanding data center environments. Using FabricPath, you can build highly scalable Layer 2 multipath networks without the Spanning Tree Protocol. Such networks are particularly suitable for large virtualization deployments, private clouds, and high-performance computing (HPC) environments.

This event will focus on technical support questions related to the benefits of Cisco FabricPath over STP or VPC based architectures, design options with FabricPath, migration to FabricPath from STP/VPC based networks and FabricPath design and implementation best practices.

Hatim Badr is a Solutions Architect for Cisco Advanced Services in Toronto, where he supports Cisco customers across Canada as a specialist in Data Center architecture, design, and optimization projects. He has more than 12 years of experience in the networking industry. He holds CCIE (#14847) in Routing & Switching, CCDP and Cisco Data Center certifications.

Iqbal Syed is a Technical Marketing Engineer for the Cisco Nexus 7000 Series of switches. He is responsible for product road-mapping and marketing the Nexus 7000 line of products with a focus on L2 technologies such as VPC & Cisco FabricPath and also helps customers with DC design and training. He also focuses on SP customers worldwide and helps promote N7K business within different SP segments. Syed has been with Cisco for more than 10 years, which includes experience in Cisco Advanced Services and the Cisco Technical Assistance Center. His experience ranges from reactive technical support to proactive engineering, design, and optimization. He holds CCIE (#24192) in Routing & Switching, CCDP, Cisco Data Center, and TOGAF (v9) certifications.

Remember to use the rating system to let Hatim and Iqbal know if you have received an adequate response.  

They might not be able to answer each question due to the volume expected during this event. Remember that you can continue the conversation on the Data Center sub-community Unified Computing discussion forum shortly after the event. This event lasts through Dec 7, 2012.. Visit this support forum often to view responses to your questions and the questions of other Cisco Support Community members.

21 Replies 21

Sarah Staker
Level 1
Level 1

Hello,

I would like to better understand FabricPath. Can you tell me for FabricPath differs from VPC?, what about the benefits of FabricPath in comparison with STP?

Thank you,

Sarah

Hi Sarah,

Thank you for your question.

Spanning Tree Protocol is used to build a loop-free topology. Although Spanning Tree Protocol serves a critical function in these Layer 2 networks, it is also frequently the cause of a variety of problems, both operational and architectural.

One important aspect of Spanning Tree Protocol behavior is its inability to use parallel forwarding paths. Spanning Tree Protocol forms a forwarding tree, rooted at a single device, along which all data-plane traffic must flow. The addition of parallel paths serves as a redundancy mechanism, but adding more than one such path has little benefit because Spanning Tree Protocol blocks any additional paths

In addition, rooting the forwarding path at a single device results in suboptimal forwarding paths, as shown below, Although a direct connection may exist, it cannot be used because only one active forwarding path is allowed.

Virtual PortChannel (vPC) technology partially mitigates the limitations of Spanning Tree Protocol. vPC allows a single Ethernet device to connect simultaneously to two discrete Cisco Nexus switches while treating these parallel connections as a single logical PortChannel interface. The result is active-active forwarding paths and the removal of Spanning Tree Protocol blocked links, delivering an effective way to use two parallel paths in the typical Layer 2 topologies used with Spanning Tree Protocol.

vPC provides several benefits over a standard Spanning Tree Protocol such as elimination of blocker ports and both vPC switches can behave as active default gateway for first-hop redundancy protocols such as Hot Standby Router Protocol (HSRP): that is, traffic can be routed by either vPC peer switch.

At the same time, however, many of the overall design constraints of a Spanning Tree Protocol network remain even when you deploy vPC such as

1.     Although vPC provides active-active forwarding, only two active parallel paths are possible.

2.     vPC offers no means by which VLANs can be extended, a critical limitation of traditional Spanning Tree Protocol designs.

With Cisco FabricPath, you can create a flexible Ethernet fabric that eliminates many of the constraints of Spanning Tree Protocol. At the control plane, Cisco FabricPath uses a Shortest-Path First (SPF) routing protocol to determine reachability and selects the best path or paths to any given destination in the Cisco FabricPath domain. In addition, the Cisco FabricPath data plane introduces capabilities that help ensure that the network remains stable, and it provides scalable, hardware-based learning and forwarding capabilities not bound by software or CPU capacity.

Benefits of deploying an Ethernet fabric based on Cisco FabricPath include:

Simplicity, reducing operating expenses

– Cisco FabricPath is extremely simple to configure. In fact, the only necessary configuration consists of distinguishing the core ports, which link the switches, from the edge ports, where end devices are attached. There is no need to tune any parameter to get an optimal configuration, and switch addresses are assigned automatically.

– A single control protocol is used for unicast forwarding, multicast forwarding, and VLAN pruning. The Cisco FabricPath solution requires less combined configuration than an equivalent Spanning Tree Protocol-based network, further reducing the overall management cost.

– A device that does not support Cisco FabricPath can be attached redundantly to two separate Cisco FabricPath bridges with enhanced virtual PortChannel (vPC+) technology, providing an easy migration path. Just like vPC, vPC+ relies on PortChannel technology to provide multipathing and redundancy without resorting to Spanning Tree Protocol.

Scalability based on proven technology

– Cisco FabricPath uses a control protocol built on top of the powerful Intermediate System-to-Intermediate System (IS-IS) routing protocol, an industry standard that provides fast convergence and that has been proven to scale up to the largest service provider environments. Nevertheless, no specific knowledge of IS-IS is required in order to operate a Cisco FabricPath network.

– Loop prevention and mitigation is available in the data plane, helping ensure safe forwarding that cannot be matched by any transparent bridging technology. The Cisco FabricPath frames include a time-to-live (TTL) field similar to the one used in IP, and a Reverse Path Forwarding (RPF) check is also applied.

Efficiency and high performance

– Because equal-cost multipath (ECMP) can be used the data plane, the network can use all the links available between any two devices. The first-generation hardware supporting Cisco FabricPath can perform 16-way ECMP, which, when combined with 16-port 10-Gbps port channels, represents a potential bandwidth of 2.56 terabits per second (Tbps) between switches.

– Frames are forwarded along the shortest path to their destination, reducing the latency of the exchanges between end stations compared to a spanning tree-based solution.

    – MAC addresses are learned selectively at the edge, allowing to scale the network beyond the limits of the MAC addr

Hi Hatim,

I wasn't familiar with Cisco FabricPath and would first like to thank you for explaining what it is. I'm already using vPC and seems like I should migrate to Cisco FabricPath. Before I start checking config docs, can you please clarify is there any reason why I should still stick with vPC? Seems like this new feature is Cisco proprietary but it also seems it will work with 3rd party devices that support vPC only, correct?

Also, I noticed you are using term "vPC+": can you please write a sentence or two about this one too?

Thanks,

Tenaro

Hi Tenaro,

Thanks for your question ...As Hatim mentioned in the reply above - there are lot of reasons to migrate to Cisco Fabricpath from either a STP based or a VPC based architecture..I am summarizing some of them below :

- Scalability :Cisco Fabricpath allows you to scale to much larger extent compared to a STP or VPC based design mainly because FP is not using STP at all ( unlike VPC which is still using VPC in the background ) , so because you are not using STP you dont have to worry about blocking vs forwarding ports as well as things like STP logical ports which is huge consideration for scale when designing any networks using STP.

- Growth : Even if you have two spines ( like in VPC designs ) - its still a good idea to consider FP solution because FP will allow you to easily grow in future without making significant changes in the design

- L2 Domain segmentation : Despite all the benefits VPC offers, you still cant avoid L2 domain segmentation where your vlans are typically segmented to a particular POD unlike in FP all of your vlans are available everywhere in the FP network.

- Flexibility : With VPC you are still looking at a single design which is supported by VPC , unlike with FP you can design the network the way you want to with complete flexibility .

- Bandwidth : the bandwidth increase with VPC compared to a STP design is typically double however with FP you get huge improvement over bandwidth availability as detailed by Hatim above.

- MAC address scale : the typical STP networks we are used ( same with VPC ) has drawbacks when it comes to scaling of mac addresses essentially because we are used to concept that every device will learn every mac address in a transit network.FP offers conversational learning which is a new and innovative way of handling mac address learning .With Convesational learning - the devices which are either connected to source or destination hosts will learn the mac address , all the other devices in transit will not learn that mac address thereby providing major scalability benefits to mac address scale.

Hope that helps.

Finally to answer your questions about Cisco properietary - yes FP is Cisco properietary feature but having said that we are also going to support TRILL which is standard based feature very similar to FP ( mostly based upon FP ) .However FP offers a lot more benefits over TRILL with features such as convesational learning , VPC+ etc which are not available with TRILL.

Regarding VPC+ ....VPC+ is FP variant of VPC ...so its pretty much similar to VPC but has a different name VPC+ when its used in FP environment.This is a way for hosts connecting to FP network to connect in an active-active fashion.

Hope that answers all of your questions .

Let us know if you have any more questions.

Regards,

Iqbal

joe-vieira
Level 1
Level 1

Hi there. I am wondering about using FP when connecting legacy switches such as 6509s running HSRP to Nexus running FP. What design considerations are important and what't the best way to integrate the two.Can the 2 be run in parallel during transition and migration?

Thanks

Hi Joe 

Just to understand you better you want to connect your fabric path network to you existing 6500 HSRP for L3 connectivity till you migrate your 6500 Switches to N7K.

Yes, you can connect to your Legacy 6500 HSRP switches and extend your L2 to 6500 HSRP switches. Please make sure you do not have any L2 loop outside Fabric path. HSRP hello will pass through Fabric Path cloud.

This will provide a L2 transit path for the vlans within FabricPath infrastructure to the respective SVIs (which are still configured in Catalyst 6500 switches). This layer 2 link will be configured as ‘Classical Ethernet (CE) ports on the Nexus FabricPath switches.

Few design considerations need to be made while extending the layer 2 link between the existing network and FabricPath domain. By default, the following STP-FabricPath interaction happens, when a STP domain is extended;

  • The Fabric Path switch appears to be a single bridge to outside devices. Hence it sends the same STP information on all FabricPath edge/leaf ports.
  • The FabricPath switches should be configured as the STP root for all vlans extended from the STP domain. If for any reason, a better BPDU is received on FabricPath edge port, the edge port will get blocked due to the inherent ‘Root-Guard’ protection.

For any layer 3 traffic or inter-vlan traffic, the traffic flow path will be forwarded towards the corresponding VLAN SVI that is configured on the Catalyst 6500 switches.

When you are ready with Fabric path all the vlan interfaces (SVI)  and any Layer 3 related routing configuration should be migrated to the Nexus Fabric path switches. All layer 3 decisions will be made in Nexus Fabric Path switches.

Thx

Hatim Badr

shivamishra85
Level 1
Level 1

Dear Sir ,

           Dec 2012 my ccnp certification is going to expiry . so please sugges me which certification I should

           clear to extend my certification validity.

Hi Shiva Nandan,

You posted the question in the worng forum , please contact Cisco Sertification support at the url below to get an answer to your question :

http://ciscocert.force.com/english

Regards,

Iqbal

Andras Dosztal
Level 3
Level 3

Hi,

Does the following migration scenario work? If not, could you recommend a method, which minimizes downtime?

Thank you.

Hi

My Understanding that you are planning to use vPC+ on N5K Switches as shown in diagram.

There are other ways to have minimal disruption based on your design and timeline. Here I’m suggesting one of the ways to go consider based on the information provided.

If you are currently running vPC then the first thing is to migrate to vPC+ in N5K ( this is disruptive when changing the peer link to vPC+.

Then keep continue using both 6500 switches and use Classical Ethernet connectivity between N5K and 6500 and use them as you HSRP routers. Ensure you do not have any L2 loop outside fabric path. HSRP can flow over the Fabric Path cloud.

While connecting N5K to Legacy 6500 switches for L3 connectivity, start building N7K in parallel as Fabricpath spine switches with vPC+ and connect them to N5K as FabricPath links.

After that migrate 6500 link from N5K to N7K (Introducing another layer for interim). Where N5K will be connected only N7K and N7K is connect to 6500. 6500 Switches are still your L3 boundary router.

After you are sure Fabric Path is working fine between N5K and N7K. I will start migrating SVIs in a phased approach to N7K and connect it directly to L3 domain. After migrating all SVIs you can safely remove N7K to 6500 links and decommission 6500 switches.

Thx

Hatim Badr

canero
Level 1
Level 1

Hi,

I would like to ask about VPC+ also, I understand that VPC+is used for connecting the hosts to FP domain. Is there a limitation for the VPC+ to connect to 2 Nexus Switches, or the Lacp link for the host or switch may extend to more than 2 (for example 4 different Chassis?) switches across the FP Cloud?  If possible then a mechanism such as peer-link should be shared between 4 nexus switches. It is like Host A connects to s1,s2, s3 and s4?

isyed
Cisco Employee
Cisco Employee

Hi,

VPC+ is very similar to VPC in terms of how it operates , so your hosts can connect to two switches which are VPC+ peers.You can have two VPC+ peers in a domain.

Regards,

Iqbal

bgandhi
Level 1
Level 1

Hi,

As I understand Fabric Path is available only with Nexus series. has it been introduced in other product lines also.

Regards,

Bhavesh

isyed
Cisco Employee
Cisco Employee

Hi Bhavesh,

Cisco FabricPath is currently only available on Nexus series , i heard about plans of introducing it on other Cisco platforms but cannot comment on when that will happen.

Regards,

Iqbal

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: