Fabric Path/TRILL in a 2-post core (?)

[visitor68]

Hi - wondering what the benefit is of running FP in a 2-post core - in other words, not in a folded CLOS that requires more than 2 active L2 paths.

Sure, you either totally remove STP or at least mitigate the STP domain, but you are also replacing STP with another protocol, which in itself is rather complex in the primitives and semantics in which it engages, is largely untested in the enterprise (nevr ran into a client who uses it) and may present network operations with more complexity when troubleshooting. Then you have the added cost of licenses....is it really worth it??

Anyone else have thoughts on this?

Would love to hear from the heavy hitters on here - you know hwo you are. :-)

[visitor68]

Where are all the brainiacs on here? 

I have to say this board has changed over th elast few years. It's become so tactical..."how do I configure this" and "whats the command line syntax" to do that...? We used to be able to have higher-level strategic discussions on here about technology, designs and architecture...

Remember having some great conversations with Jon Marshall, Edison Ortiz, and Giuseppe to name but a few....

[Peter Paluch]

Hello,

Well, it's weekend and a time of summer holidays - I would say that it is understandable for answers to more high-level questions to arrive with a certain delay. Also, I find the remark about "this forum changing" to be targeted inappropriately - whether we discuss design or principial issues, or simple how-to topics, that depends on the original question. I could retort that it is you who come in asking questions who define the topics to be discussed, and if the questions are predominantly how-to based then the forum shifts in that direction - but it is not us who answer your questions to be blamed for this. Please do not get disenchanted and vocal about your dissatisfaction too soon just because you have not received an answer right away - that tends to spoil the good mood in your thread.

Regarding your question, I am somewhat familiar with TRILL and FabricPath and I wanted to join the discussion but I was confused by the "2-post" and "folded CLOS" terms. I do not know what these are. Perhaps if you explain them more people will join.

Best regards,

Peter

[visitor68]

Peter, asking how-to questions is to a great extent what this board is about. So, theres nothing wrong with asking them and its very kind of people like you to answer them. As for me, I dont ask how-to/CLI questions about Cisco because I dont manage any Cisco environments. I do ask higher-level architecture and design questions that may or may not include Cisco - and you have answered a couple of them, so I thank you for that.

Anyway, I commented accordingly because this is not the first time I ask such questions and get no answer...this has become the norm, actually. It seems that if I dont ask something that is very Cisco-specififc and low-level (meaning tactical - configurations), I get blank stares.

A 2-post core is what we design everyday....2 switches in the distro/core...so in those cases you use vPC for L2MP. A folded CLOS is a fully meshed design in which the network scales out horizontally, not vertically, so you end up with what is called a leaf and spine architecture in which each leaf (access/ToR switch) is multihomed to each spine (core) switch, and there can be anywhere between 4 and a dozen or so spine switches. Speaking of Cisco, their relatively new "MSDC" design guide covers such an architecture.

As for Fabric Path, its a Cisco proprietary technology (open standard would be TRILL), which provides L2MP across a CLOS. So, that removes the need to deploy STP between the leaf and spine.

Thanks

[Peter Paluch]

Hi,

Okay, I get your point.

Sure, you either totally remove STP or at least mitigate the STP domain,  but you are also replacing STP with another protocol, which in itself  is rather complex in the primitives and semantics in which it engages

If you mean IS-IS here then I do not see any significantly added complexity when comparing TRILL/FP to a routed access layer approach. If you have a routed access layer then you usually run whatever routing protocol you want, including IS-IS, OSPF (which is overoptimized so much it hampers its scalability) or EIGRP (still a black box for many people). Having IS-IS/OSPF/EIGRP in a routed access layer, and having IS-IS in a switched TRILL/FP domain is not that different. Certainly, when compared with STP, the TRILL/FP is more complex, but it should not be labeled as the only "more complex" approach as the routed access layer with a similar complexity has been here even longer than TRILL/FP.

Yes, there are another added complexities of the additional encapsulation, distribution trees for unknown unicast/multicast/broadcast, different flooding rules... however, they are not that complicated once you understand why they are performed and how they are done.

Running FP even in a 2-post core can have its advantages, for example:

• You can achieve load sharing even within a single VLAN, something STP would never allow you to do
• The inter-switch connections actually do not use VLANs at all, thus isolating any user traffic from the network infrastructure and saving the VLAN space
• With vPC, if I am not mistaken, you can actually use FHRP protocols like HSRP in Active-Active scenario, achieving load balancing without resorting to mechanisms like GLBP
• TRILL/FP practically eliminates the need for a distinct distribution layer

In any case, in my opinion, the full potential of TRILL/FP can be exploited in data centers where the leaf-and-spine topology is prevalent. Using TRILL/FP in a 2-post core is certainly possible though the full power of this technology probably won't be unleashed (just like running OSPF on a few routers in a single area - it works nicely but you just have no use for the full extent of OSPF capabilities).

Still, if someone offered me an option of building a formerly classic access/distribution layer network with TRILL/FP, I would go for it. Currently, the problem is that it is not supported on typical enterprise switches, only on Nexuses 5500 and higher. An idea of Catalyst 2960/3560 running TRILL/FP brings a smile to my face - but I hope for it some day!

Whether it is worth the money... well, that's a totally different question for which I do not consider myself competent at all

So my two cents to make this discussion lift off a little...

Best regards,

Peter

[visitor68]

Peter, good stuff...agree for the most part...TRILL/FP's benefits are more pronounced in a CLOS...but a 2-post core, not so much - and that is my question. With STP and vPC, you have active/active links, so thats not a differentiator. Flattening the network? Not really, if you have no need to expand past 2 posts. LB within a VLAN? vPC gives you that.

As for running a routed acces layer and FP - yes, some similarities, but some differences, too...not quite the same. With FP, its a MAC-in-MAC protocol and its a 'tunneling' mechanism whose ultimate goal is to proivide L2 adjaceny, not L3 isolation....

All in all, I dont think FP in a 2-post core is something worth the money, time to learn and manage and complexity...removing STP is not that big a deal if you can mitigate its most important limitation - blocked links. And you can do that with vPC.

[Peter Paluch]

Hello,

Thank you.

Yes, absolutely, the TRILL/FP's advantages are most visible in CLOS. And you are right, many things can be achieved by vPC, or - with those of us who cannot afford such fancy boxes - using stacked switches and cross-stack EtherChannels

Anyway, you may be inclined to think: If I have a Nexus supporting the vPC, or a 4K/6K Catalyst capable of VSS, why should I use TRILL/FP if I already have other technologies providing me with similar features? My take on it would be: imagine that at some point in the future, TRILL/FP support would be available on even the low-end non-stackable Catalysts that are way cheaper than Nexus boxes. Now my question would be - would you consider running TRILL/FP in such a network? Let's try for a moment to put the financial aspect away. I know that in real world, you can not abstract from that, but for now, I want to make us all think whether we're avoiding TRILL/FP because of its supposed technical complexity or because currently (which does not mean forever!), the price-to-performance ratio isn't that good.

In any case, the 2-post topology is very simple. And quite right, the TRILL/FP is capable of so much more that running it in this kind of topology strongly limits its capabilities. The only additional advantage that comes to my mind is that you can actually interconnect the access layer switches together if your VLANs span several access switches and there is a significant client-to-client traffic in these VLANs, with TRILL/FP allowing you to reuse those links, as STP would have blocked them and vPC/VSS/etc. would not be of help here.

Best regards,

Peter

[visitor68]

Hey, Pete! I think we are almost in total agreement. Putting price aside, would I go with FP in a 2-post core? Eh, I dont know. I am not very familiar with FP from an operational perspective. Honestly, it seems somewhat complicated, but again, this is just from reading white papers and not seeing it in action. I never ran into a client who has it deployed or who has a CLOS arcitecture in the first place.

Moreover, I never spoke to anyone who really saw a big advantage in removing STP from an environment in a 2-post architecture. STPs biggest drawback is blocked links, and if you can avoid it with vPC, i think youre fine.