How ACI Spine syncronizes with each other

[Nitesh_A]

Hi Community,

Could someone please help me understand this.

Q1. In Cisco ACI when the Leaf learns MAC/IP from front panel port, does it send that information to both Spines ?
I know spine uses COOP protocol but how does spine syncronizes with each other as there is no direct link between two?

Q2. How root switch election is done in FTAG tree ? I mean how ISIS extension helps here ?

Thankyou

[M02@rt37]

Hello @Nitesh_A,

In ACI, when a Leaf switch learns MAC/IP information from a front panel port (like a host connected to the leaf), this information is indeed sent to both Spine switches. The Spines use COOP to synchronize this information. The Spine switches maintain a COOP database, which contains the learned MAC/IP information from all the leaf switches. While there's no direct link between Spine switches, the COOP protocol ensures that MAC/IP information is consistently shared and synchronized between the Spines.

As concerned FTAG, the root switch election in the FTAG tree is done to ensure efficient tree construction for optimal forwarding.

The election process involves determining a root switch (Spine) for each FTAG tree. This is usually based on factors like spine switch identifier, system ID, or other predefined metrics.
The elected root switch becomes the root of the FTAG tree and is responsible for computing the forwarding paths.

ACI leverages the ISIS protocol for control plane communication and information dissemination within the fabric. The ISIS extension in ACI allows for additional capabilities and information specific to ACI requirements, including FTAG tree construction and maintenance. ISIS extensions aid in the efficient computation of forwarding paths and ensuring that the traffic flows optimally within the ACI fabric. By utilizing ISIS extensions and root switch election in FTAG trees, ACI optimizes forwarding, load balancing, and resilience within the fabric, contributing to a highly efficient and scalable networking solution.

[RedNectar]

Hi @Nitesh_A ,

Just to add a small detail to M02@rt37 's excellent answer to you Q1:

When a leaf switch learns a new MAC on an access port, it sends that information to ONE of the spine switches chosen based on a has of that MAC address (there's a bit more to it than that, but it conveys the idea) using the Council Of Oracles Protocol (COOP)

The Spine switch that gets this information then re-distributes this information to all other spines using COOP also

The process is the same for a new IP address learned by a leaf switch.

So the MAC and IP information is always learned by both (in fact ALL) Spine switches.  

[Nitesh_A]

@RedNectar 

The Spine switch that gets this information then re-distributes this information to all other spines using COOP ---->

So spine forwards that learned IP/MAC through leaf(from where it originally learned) to other spine OR (as per M02@rt37 )leaf creates two copies of that packet and sends that to both spine at a time ?

[RedNectar]

Hi @Nitesh_A ,

I don't think M02@rt37 ACTUALLY SAID that two copies of the packet are created by the leaf - but I could see that his answer could lead to that interpretation- which is why I went to the trouble of clarifying.

And you can have many more spines than two, so let's forget this idea of "two copies".  Every spine, whether there is 2 or 10 gets the information.

But the LEAF sends it to ONE Spine. That spine distributes that information - I'm not sure if it uses a multicast packet to do that (I suspect it does) or multiple packets. I've never dug that deep.

Here's a slide form https://www.ciscolive.com/c/dam/r/ciscolive/emea/docs/2020/pdf/BRKACI-3545.pdf that may help

BTW - you may find this ancient post useful https://community.cisco.com/t5/application-centric-infrastructure/mp-bgp-is-is-coop/m-p/3063789/highlight/true#M3075

[M02@rt37]

Hello @RedNectar,

Thanks fo that clarification !

[Nitesh_A]

@RedNectar I have gone through your article on https://rednectar.net/2018/03/01/isis_coop_bgp_and_mp-bgp_in_cisco_aci/ . But there is no where clearly mentioned how spine synchronizes coop database with other spine.

It just says leaf reports information to one of the spine switches (chosen at random) using the Council Of Oracles Protocol (COOP).  The spine switch (oracle) that was chosen then relays this information to all the other spines (oracles) so that every spine (oracle) has a complete record of every end point in the system

So may be i have to dig more on how coop works in aci 

[RedNectar]

Hi @Nitesh_A ,

I have to ask WHY you need to dig more to see how COOP works?  If you are working for CIsco TAC or the development team, you may need to know more than what's already been covered. It is VERY UNLIKELY that you'll need to troubleshoot COOP issues, or any issues to do with the spines.

Now, if your reason for wanting to know is simply because you just like to know - then I get that - you and I have something in common.  And I guess I've just never had the desire to dig any deeper. However, I can dig our a bit more information and it will take a bit of time - I'm travelling at the moment (holiday YEAH) so may take some time to dig it out.

[Nitesh_A]

@RedNectar yes i was very curious to know how this works, also i am preparing myself to answer all question that comes up in my mind. This is of no urgent. Please enjoy your vacation.

Happy Journey

[RedNectar]

Hi @Nitesh_A ,

I sent yo a PM - did you get it?

[Nitesh_A]

@RedNectar yes i got it. Thankyou so much for the help