ACI Fabric - Underlying protocols

[LakshmiPrabu]

Please point to the best reference to understand the underlying protocols in ACI Fabric communication like IS-IS, Multicast, MP-BGP

[Nik Noltenius]

It's probably not "the best reference" but I personally found the Multi-Pod guide quite useful for understanding the concepts:

http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/white-paper-c11-737855.pdf

It starts on page 25 with "Multi-Pod ‘Overlay’ Control and Data Planes". 

This is also quite useful to understand the concepts:

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/pf/configuration/guide/b-pf-configuration/Introducing-Cisco-Programmable-Fabric-VXLAN-EVPN.html

It's for VXLAN EVPN fabrics but as far as I understand it the principles are the same.

Hope that helps

Nik

[micgarc2]

ACI uses MP-BGP as the routing protocol we run internal to the fabric. It allows border leafs to redistribute external routes inside the fabric. MP-BGP will be used to distribute these external routes to the other leaf switches.  By default, this route redistribution actually is not enabled. To do this we have to make the spines route reflectors and assign a BGP AS number. Once this is configured the fabric will use these route reflectors to do the actual distribution of routes from each of the leafs to the rest of the fabric. With traditional BGP you actually have to peer with every device so ACI used the route reflector concept so only the spines have to act as the peers for all the leafs in the fabric.

IS-IS is a routing protocol that is used between the leaf and spine switches. We use IS-IS as the routing protocol for reachability between the TEP IPs (VTEPs). It is what is actually building the routing table between the VTEPS. All the convergence happens independently of the APIC. IS-IS, along with COOP are kind of the brains behind endpoint to endpoint communication. 

As far as muliticast I am not sure what you are referring to: Layer 3 multicast, how we use use mulitcast routing in the BD? When we initially set up an APIC, we configure an address pool for BD multicast addresses, also known as the GiPo. Each BD we create is assigned one of these addresses in the range we allocate. 

This is explained more here:

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/aci-fundamentals/b_ACI-Fundamentals/b_ACI-Fundamentals_chapter_010010.html

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/2-x/multicast/b_Using_Layer3_Multicast/b_Using_Layer3_Multicast_chapter_00.html

http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/white-paper-c11-737909.html

[sumitdhyani1]

Hi All,

If anybody can clear my doubt in Cisco ACI MPBGP is considered underlay or overlay. 

[Sergiu.Daniluk]

Hi @sumitdhyani1 

Although there are different perspective of how you can look at it, ACI is a software defined network so the nature of the layers changed a bit. 

I would suggest to start with the overlay first , which is the tenant with it's constructs (VRFs, BDs, L3Outs, EPs). This is where user traffic is being contained. What is providing the forwarding for this layer? It's VXLAN. But how is the EP information exchanged inside the VXLAN fabric, you might ask? Through the use of COOP. So far so good, right?

Good, continuing the discussion, we have the underlay: this is the infra network, with all it's TEPs, IS-IS, MP-BGP and all protocols used to manage ACI switches (example NTP). Why is MP-BGP here? Because is the control plane protocol which provides information to overlay to run: external routing (VPNv4) information from the Border Leafs, as well as EP (EVPN) information between pods/sites spines COOP.

There is also another layer in my opinion, and that is the security layer which is just another abstraction layer on top of the overlay. This is composed of EPGs, Contracts etc.

I hope this helps.

Stay safe,

Sergiu

[sumitdhyani1]

Hi Sergiu,

Thank you so much for the reply . Overall what I understood is MPBGP will be considered underlay protocol with ISIS. ISIS for Vtep and MPBG for external routes. I was making a small architecture and was getting littile confused. When one VRF is getting connected to friewall via leaf the connectivity will be considered overlay from vrf prospective or it is connecting to underlay. What should I consider that As they will share the routes in overlay. Attaching a diagram. Thank you so much again

[Sergiu.Daniluk]

---

@sumitdhyani1 wrote:

When one VRF is getting connected to friewall via leaf the connectivity will be considered overlay from vrf prospective

Yes, from tenant/user VRF, this is overlay.

Regards,

Sergiu

[sumitdhyani1]

Hi Sergiu,

Thanks for the reply So in this case routes are again external so can we consider MPBGP overlay here. 

[sumitdhyani1]

Hi Sergiu,

Thank you so much for the reply really appreciated.

If we connect any firewall to leaf is it necessary that leaf should be border leaf or any leaf can be connected to firewall.

Regards,

Sumit