Spine and Leaf Loops

[xklima6]

How is it with loops (STP) in spine and leaf architecture ? Can someone please explain. 

[MHM Cisco World]

There is no need STP in vxlan since it depends on underlying L3 to learn MAC.

MHM

[Joseph W. Doherty]

If you're referring to an actual leaf and spine fabric architecture, they have their own mechanisms to deal with multiple L2 paths.

If you're referring to classical L2 switch hardware, that happens to use a leaf and spline like architecture, it would need a STP to break L2 loops and could not leverage the advantages of an actual leaf and spine fabric architecture.

For example, in the Cisco world an actual leaf and spine fabric architecture might be provided by ACI.

[Giuseppe Larosa]

Hello @xklima6 ,

all spine and leaf topologies use L3 links that can be either : numbered with specific IP subnet for each point to point link or in some cases unnumbered using the loopback address as a reference.

So there is no need of STP in the underlay because it is a pure L3 architecture using OSPF or IS-IS and ECMP equal cost multi path.

The overlay can emulate L2 topologies but it does not use STP because the control plane take care of MAC address learning using standard MAC address learning on user facing L2 ports and using other methods for remote learning mainly flood and learn over multicast underlay or using MP BGP EVPN to advertise locally learned MAC addresses and IP addresses ( endpoints )

Hope to help

Giuseppe

[Joseph W. Doherty]

@Giuseppe Larosa I wonder about "all spine and leaf topologies use L3 links . . ." and "So there is no need of STP in the underlay because it is a pure L3 architecture using OSPF or IS-IS and ECMP equal cost multi path." because of my reading, leaf devices are often L2 technologies, like SPB (801.1aq - NB does use IS-IS, but not IP, I believe) or TRILL support L2 ECMP without L3 and STP.

I.e. my understanding, as I'm definitely not very knowledgeable in this subject, I'm under the impression leaf and spine may not all be pure L3 or even require L3 to support L2 running on them.

Could you point me to any literature to further clarify leaf and spine architectures?

Thanks.

[Joseph W. Doherty]

Just a follow-up, possibly the different understanding @Giuseppe Larosa and I am might have has more to due with semantics.(?)

I.e. if you "route" frames and not packets are we L2 or L3?  Especially if there's  an IP L3 routing layer above the leaf-and-spine architecture, i.e. for communication with the rest of your L3 topology beyond the leaf-and-spine component.

From Wiki IS-IS:

Other uses

[edit]

IS-IS is also used as the control plane for IEEE 802.1aq Shortest Path Bridging (SPB). SPB allows for shortest-path forwarding in an Ethernet mesh network context utilizing multiple equal cost paths. This permits SPB to support large Layer 2 topologies, with fast convergence, and improved use of the mesh topology.^[5] Combined with this is single point provisioning for logical connectivity membership. IS-IS is therefore augmented with a small number of TLVs and sub-TLVs, and supports two Ethernet encapsulating data paths, 802.1ad Provider Bridges and 802.1ah Provider Backbone Bridges. SPB requires no state machine or other substantive changes to IS-IS, and simply requires a new Network Layer Protocol Identifier (NLPID) and set of TLVs. This extension to IS-IS is defined in the IETF proposed standard RFC 6329.

[Giuseppe Larosa]

Hello @Joseph W. Doherty ,

>>  I wonder about "all spine and leaf topologies use L3 links . . ." and "So there is no need of STP in the underlay because it is a pure L3 architecture using OSPF or IS-IS and ECMP equal cost multi path."

Ok for current Cisco solutions using spine and leaf topologies in data center   either ACI  or EVPN with VXLAN and MP BGP EVPN the links are L3.

All the standards about PBB or SPB actually use MAC in MAC encapsulation in the data plane but if you use IS-IS with new TLVs and sub TLVs we can say they are routing frames.

The SPB or PBB allows service provider switches to avoid to have to learn MAC addresses in the customer space providing scalability and this can be seen as a natural evolution of 802.1Q tunneling  Q in Q where we hide the customer used VLANs but not the customer used MAC addresses.

So SPB or PBB core switches like P nodes in MPLS services don't need to know customer specific / service specific MAC addresses ( routes in VRFs in the case of P nodes)

VXLAN and Fabric Path use MAC in UDP encapsulation.

Cisco has chosen to implement VXLAN and for example some modern Nexus platforms in EVPN multi site can perform in hardware VXLAN header rewrite at border gateways  (devices that are the edge of each site ) because VXLAN encapsulation is stateless this can be done.

Hope to help

Giuseppe

[Joseph W. Doherty]

@Giuseppe Larosa thank you.

With your reply, and rereading other material, what we have, at the underlay, is routing of Ethernet frames (the reason STP is unnecessary), just within the leaf and spine architecture.  On top of that, we can have routing of packets.

Your examples of Q in Q and MPLS are good, possibly something like a L2 tunnel, running over an IP network might be too.

Possibly, what might cause confusion is different layers of the OSI being used within layers.

For example, in a leaf and spine architecture, hosts see both "normal" L2 Ethernet and L3 IP services.

What hosts often don't see is how their L2 or L3 service is supported.  For example, one host sending an Ethernet frame to another host doesn't know, or usually care, whether the two hosts are connected via a back to back Ethernet cable, hub, bridge, switch, etc.  Ditto leaf and spine.

What threw me with your earlier replies, I took your leaf and spine L3 architecture encompassing much more than what I now believe you intended to mean.