cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
15867
Views
156
Helpful
14
Replies
Highlighted
Beginner

MP-BGP , IS-IS, COOP

Hello,

Can someone explain to me what are doing those 3 technologies ?

Thanks

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted
Collaborator

The roles of IS-IS, COOP, BGP and MP-BGP in Cisco ACI

Hi mgual57@gmail.com

 

Let me start with a visual picture.  Imagine a simple 2leaf/2spine topology with HostA attached to to Leaf1 and with HostB attached to to Leaf2

Leaf1 has a VTEP address of 10.0.1.101

Leaf2 has a VTEP address of 10.0.1.102

Spine1 has a VTEP address of 10.0.1.201

Spine2 has a VTEP address of 10.0.1.202

 

HostA has a MAC address of A and an IP address of 192.168.1.1 and is attached to port 1/5 on Leaf1

HostB has a MAC address of B and an IP address of 192.168.1.2 and is attached to port 1/6 on Leaf2

 

 

Enter IS-IS

The leaves and spines will exchange IS-IS routing updates with each other so that Leaf1 sees that it has two equally good paths to reach Leaf2, and Leaf2 sees that it has two equally good paths to reach Leaf1

For now, that's all we need to know about IS-IS - IS-IS is the routing protocol used by the VTEPs to learn how to reach the other VTEPs.

Now think about the hosts.

This is where COOP comes in.

When Leaf1 learns about HostA because say HostA sent an ARP request seeking the MAC address of 192.168.1.2 (which you know is HostB, but that's not relevant at the moment)

Leaf1 looks at that ARP request, and just like a normal switch, learns that MAC A is present on port 1/5.  But the leaf is a bit more clever than that, and looks INSIDE the payload of the ARP packet and learns that Host1 also has an IP address of 192.168.1.1 and records all this information in its Local Station Table

AND THEN reports this information to one of the spine switches (chosen at random) using the Council Of Oracles Protocol (COOP).  The spine switch that was chosen then relays this information to all the other spines so that every spine has a complete record of every end point in the system.

The spines record the information learned via the COOP in the Global Proxy Table, and this information is used to resolve unknown destination MAC/IP addresses when traffic is sent to the Proxy address.

Note that all of this happens without anything to do with BGP.

But to round off the COOP story, we would assume that at some stage Leaf2 will also learn HostB's MAC and IP and also inform one of the spines at random of this information using the COOP.

So COOP is used solely for the purpose of distributing endpoint information to Spine switches. As far as I know, Spine switches never use COOP to distribute end host information to leaf switches.

So where does BGP fit in?

BGP is not needed until an external router is connected.  So now imagine that Leaf2 has had a router connected and has learned some routes from that external router for a particular VRF for a particular Tenant. 

How can Leaf2 pass this information on to Leaf1 where HostA is trying to send packets to one of these external networks?  For Leaf2 to be able to pass routing information on to Leaf1 and keep that information exclusive to the same VRF, we need a routing protocol that is capable of exchanging routing information for multiple VRFs across an underlay network.

Which is exactly what MP-BGP was invented for - to carry routing information across MPLS underlay networks.  In the case of ACI, BGP is configured by choosing an Autonomous System number and nominating one of the spine switches to be a route reflector.  MP-BGP is self configuring, you don't need to do anything to make it work!

Hope this helps.

Reference: A Version of this post with illustrations and examples can be found here.

 

 

 

RedNectar
aka Chris Welsh


Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem

View solution in original post

Everyone's tags (5)
14 REPLIES 14
Highlighted
Enthusiast

Hi @mgual57@gmail.com  

Hi [@mgual57@gmail.com]  

MP-BGP, when enabled by configuring an ASN and enabling your spines as route reflectors,  handles distributing the (host and external) routing table to the devices in the fabric (i.e leafs) and to external devices via L3outs.

Think of IS-IS as the IGP within the fabric that is building the routing tables.

Think of COOP (Council of Oracle Protocol) as the protocol that that helps to build the tables that identify endpoints...what helps build the "arp table" across the entire fabric, not just on a sigle device.

This is how I like to think of them...there is alot more "under the hood" but one of the nice things about ACI is that it does alot of that for you!

Highlighted
Beginner

Thanks you !So if I

Thanks you !

So if I understand well , IS-IS is used for routing between leaves and spines and VTEp , MP-BGP is used for routing to external devices but leaves learn Endpoint's mac and BGP send it to spines. BGP also store information about the IP address of VTEP. And finally COOP is used for mac discovery of the host?

Highlighted
Beginner

One more question : When is

One more question : When is COOP used exactly , cause MP-BGP can learn host mac address so I dont really understand the utility of COOP

Highlighted
Collaborator

The roles of IS-IS, COOP, BGP and MP-BGP in Cisco ACI

Hi mgual57@gmail.com

 

Let me start with a visual picture.  Imagine a simple 2leaf/2spine topology with HostA attached to to Leaf1 and with HostB attached to to Leaf2

Leaf1 has a VTEP address of 10.0.1.101

Leaf2 has a VTEP address of 10.0.1.102

Spine1 has a VTEP address of 10.0.1.201

Spine2 has a VTEP address of 10.0.1.202

 

HostA has a MAC address of A and an IP address of 192.168.1.1 and is attached to port 1/5 on Leaf1

HostB has a MAC address of B and an IP address of 192.168.1.2 and is attached to port 1/6 on Leaf2

 

 

Enter IS-IS

The leaves and spines will exchange IS-IS routing updates with each other so that Leaf1 sees that it has two equally good paths to reach Leaf2, and Leaf2 sees that it has two equally good paths to reach Leaf1

For now, that's all we need to know about IS-IS - IS-IS is the routing protocol used by the VTEPs to learn how to reach the other VTEPs.

Now think about the hosts.

This is where COOP comes in.

When Leaf1 learns about HostA because say HostA sent an ARP request seeking the MAC address of 192.168.1.2 (which you know is HostB, but that's not relevant at the moment)

Leaf1 looks at that ARP request, and just like a normal switch, learns that MAC A is present on port 1/5.  But the leaf is a bit more clever than that, and looks INSIDE the payload of the ARP packet and learns that Host1 also has an IP address of 192.168.1.1 and records all this information in its Local Station Table

AND THEN reports this information to one of the spine switches (chosen at random) using the Council Of Oracles Protocol (COOP).  The spine switch that was chosen then relays this information to all the other spines so that every spine has a complete record of every end point in the system.

The spines record the information learned via the COOP in the Global Proxy Table, and this information is used to resolve unknown destination MAC/IP addresses when traffic is sent to the Proxy address.

Note that all of this happens without anything to do with BGP.

But to round off the COOP story, we would assume that at some stage Leaf2 will also learn HostB's MAC and IP and also inform one of the spines at random of this information using the COOP.

So COOP is used solely for the purpose of distributing endpoint information to Spine switches. As far as I know, Spine switches never use COOP to distribute end host information to leaf switches.

So where does BGP fit in?

BGP is not needed until an external router is connected.  So now imagine that Leaf2 has had a router connected and has learned some routes from that external router for a particular VRF for a particular Tenant. 

How can Leaf2 pass this information on to Leaf1 where HostA is trying to send packets to one of these external networks?  For Leaf2 to be able to pass routing information on to Leaf1 and keep that information exclusive to the same VRF, we need a routing protocol that is capable of exchanging routing information for multiple VRFs across an underlay network.

Which is exactly what MP-BGP was invented for - to carry routing information across MPLS underlay networks.  In the case of ACI, BGP is configured by choosing an Autonomous System number and nominating one of the spine switches to be a route reflector.  MP-BGP is self configuring, you don't need to do anything to make it work!

Hope this helps.

Reference: A Version of this post with illustrations and examples can be found here.

 

 

 

RedNectar
aka Chris Welsh


Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem

View solution in original post

Everyone's tags (5)
Highlighted
Beginner

Just one last question , I

Just one last question , I thought MP-BGP can be used to learn host mac address but it seems like I was wrong or I misunderstood your explaination?

Highlighted
Collaborator

Ahh - now you are getting

Ahh - now you are getting into the realm of MP-BGP EVPN (what a mouthful) which is used between pods in a multi-pod environment and between sites in a multi-site environment, but I don't have those details in my head, I'd have to read http://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/guide-c07-734107.html as a starting point

So for now I'll leave this - perhaps you should ask this as a new question to attract other respondents who by know are skipping this thread.

CW

RedNectar
aka Chris Welsh


Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem

Highlighted
Cisco Employee

Re: Just one last question , I

Indeed your understanding is correct and this can be done using BGP as well but not in ACI

Highlighted
Contributor

Re: Hi mgual57@gmail.com

This is a fantastic, clear, and super concise write up of the ACI fabric in a nutshell. Thank you for writing this.
Highlighted
VIP Collaborator

Re: The roles of IS-IS, COOP, BGP and MP-BGP in Cisco ACI

As a teacher, I appreciate the clear, concise story you told to explain these concepts. If you are not a teacher, please consider it. You will improve the lives of many.

Highlighted
Beginner

Re: The roles of IS-IS, COOP, BGP and MP-BGP in Cisco ACI

Dear Chris
Thanks a lot but where does Multicast role is here

Highlighted
Beginner

Re: The roles of IS-IS, COOP, BGP and MP-BGP in Cisco ACI

Just one minor detail. I'm trying to understand the use of IS-IS in this particular scenario. In my understand, IS-IS is mandatory because it runs over layer 2, in opposite to OSPF or even IGRP that run over layer 3. Am I right, or there is no reason to use IS-IS in particular? Thanks!

Highlighted
Collaborator

Re: The roles of IS-IS, COOP, BGP and MP-BGP in Cisco ACI

Hi @JoaoCadavez ,

I hope I can settle your concerns.

IS-IS is a routing protocol

Just one minor detail. I'm trying to understand the use of IS-IS in this particular scenario. 


Although IS-IS was originally defined to run on CLNS and does run at the Data Link Layer, it still carries routing information.  In 1990, extensions to the IS-IS protocol were defined in RFC 1195 to allow it to carry IP routing information as well as CLNS routes.

Why IS-IS is Mandatory

In my understand, IS-IS is mandatory because it runs over layer 2, in opposite to OSPF or even IGRP that run over layer 3. 


The fact that IS-IS runs at Layer 2 dies not make it mandatory.

To build a Leaf-Spine underlay toploogy, whether it be ACI or any other variation, an underlying routing protocol is required to maintain the mapping of the Leaf-Spine topology.  Some vendors chose BGP as the underlying protcol. But BGP lacks the super-fast convergence that can be achieved with a link-state routing protocol. When Insieme developed ACI's Leaf-Spine topology, they chose IS-IS with its IP extensions as the underlying protocol.

If you use ACI, you get IS-IS as the underlying protocol, but that it totally transparent to the ACI administrator. I have been working extensively with ACI since version 1.01 and have NEVER had to look at any of the workings of IS-IS other than to stisfy my curiosity.

There's no reason to use IS-IS - in fact you can't!

Am I right, or there is no reason to use IS-IS in particular? Thanks!


As far as the end-user is concerned, you will NEVER have to do any IS-IS configuration. The only routing protocol you'll need to configure is between ACI and the outside world. As of 2020, your choices are limited to using OSPF, BGP or EIGRP.  You don't even have IS-IS as a choice!

The advantage IS-IS gives ACI

ACI leaf-spine structure is almost as transprent to the end user as the backplane of say a Nexus 7000 or 9000 chasis.  You as the user will never have to worry about the underlying leaf-spine topolgy.  There are chaper SDN solutions that allow you to build your own leaf-spince topolgy, where you get to manage all that leaf-spine infrastructure and routing yourself. In fact you can even buy the same Nexus 9000 switches running in NXOS mode to do just that. But I wouldn't advise doing that when you can buy a ready made ACI solution.

 

 

RedNectar
aka Chris Welsh


Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem

Everyone's tags (3)
Highlighted
Beginner

Re: The roles of IS-IS, COOP, BGP and MP-BGP in Cisco ACI

Thank you so much!

Highlighted
Beginner

Ok i got this , thanks you so

Ok i got this , thanks you so much. Very clear explaination