ACI Leaf Forwarding

[visitor68]

Hello. 

Once a leaf switch has added the VxLan header and crafted the frame, how does it know which interface to send it out of? Assume 4 spine switches, how does it know which spine to send it to? I know they should all be equal cost, but what decision mechanism does it use?

Thanks

[omz]

i think .. thats where is-is and coop protocols come in .. from what I can remember watching some videos :)

[omz]

found this - 

As traffic enters the fabric, ACI encapsulates and applies policy to it, forwards it as needed across the fabric through a spine switch (maximum two-hops), and de-encapsulates it upon exiting the fabric. Within the fabric, ACI uses Intermediate System-to-Intermediate System Protocol (IS-IS) and Council of Oracle Protocol (COOP) for all forwarding of endpoint to endpoint communications. This enables all ACI links to be active, equal cost multipath (ECMP) forwarding in the fabric, and fast-reconverging. For propagating routing information between software defined networks within the fabric and routers external to the fabric, ACI uses the Multiprotocol Border Gateway Protocol (MP-BGP).

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/2-x/L3_config/b_Cisco_APIC_Layer_3_Configuration_Guide/b_Cisco_APIC_Layer_3_Configuration_Guide_chapter_01.html

[Gaurav Gambhir · Accepted solution]

VM1----Leaf1------SPINE(s)------Leaf2----VM2

If source is VM1, that makes leaf1 ingress leaf and leaf2 egress leaf.

now if the ingress leaf(leaf1) has VM2 ip learned already, meaning it will be pointing towards VXlan tunnel towards the TEP address of leaf2. Now for the packet to be encapsulated towards VM2, leaf1 will look for route information for leaf2 TEP. This will be in overlay-1 vrf from ISIS. Depending upon number of links Leaf1 has towards spines Leaf1 will have ECMP path in Routing table. It will use the 5-tuple load-balance hash to pick the uplink towards Spines.

Static hash load balancing is the traditional load balancing mechanism used in networks where each flow is allocated to an uplink based on a hash of its 5-tuple.

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/aci-fundamentals/b_ACI-Fundamentals/b_ACI-Fundamentals_chapter_010010.html

hope that answers your question

[visitor68]

Outstanding! That answers my questions and satisfies my curiosity. Would you believe I was in a Cisco DCCOR class with a reputable Cisco training partner and the instructor couldn't even answer this basic question? What you explained just now is what I suspected, but he threw me off when he said "there is no routing" and the leaf does not have any routing table information. I figured I was missing some nuanced information, so I asked him for clarification. I asked him the exact same question that I posted here and he was lost. Hard to believe that class cost $4,600!