Build a multisite POC

[AlexNastas74909]

Hello Experts, 

We are looking into building an ACI fabric for a POC. The POC is purely functional, therefore cheaper devices are required. 

We want to have 1 spine + 2 leafs / per site (2 spines and 4 leafs total)

Several questions: 

- We've chosen N9K-C9332C as spine and N9K-C93180YC-FX-24 as LEAF. Is there anything cheaper / more suitable for a low-cost test environment ? 

- We need to build the IPN network. I've learned that we need a device for this. Having a "wire" between the spines to emulate IPN will not work. Is this right ? 

- Can we use only ONE device for IPN ? Or does it have to be 2 ? All the demos in the internet show 2 devices. 

- If we can use only one device - is PIM BiDir still a requirement ? I don't really see the point for only one IPN device. Simple IGMP snooping should work, no ? 

- Any recommandation of a cheap Cisco IPN device that would do the trick ? 

Thanks 

Alex

[Sergiu.Daniluk · Accepted solution]

Hello again @AlexNastas74909 ,

- We've chosen N9K-C9332C as spine and N9K-C93180YC-FX-24 as LEAF. Is there anything cheaper / more suitable for a low-cost test environment ?

You have the list of supported h/w for multi-site:

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/aci_multi-site/sw/2x/hardware-requirements/Cisco-ACI-Multi-Site-Hardware-Requirements-Guide-201.html 

Not entirely sure what are the prices, but I believe the -EX is cheaper then -FX, and First generation of h/w (non -EX/-FX) is even more cheaper. However, if you need more and newer features to be supported, better go for at least -EX.

- We need to build the IPN network. I've learned that we need a device for this. Having a "wire" between the spines to emulate IPN will not work. Is this right ?

Actually you can connect the Multi-Site spines back-to-back.

Check ACI Multi-Site spines back-to-back connectivity from multi-site whitepaper: https://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/white-paper-c11-739609.html 

- Can we use only ONE device for IPN ? Or does it have to be 2 ? All the demos in the internet show 2 devices.

Just to confirm, you plan on building a Multi-Site or Multi-Pod?

• Multi-Pod = a single APIC cluster with multiple leaf-and-spine networks (pods) interconnected through IPN (inter-pod network)
• Multi-Site = is the architecture interconnecting multiple APIC cluster domains with their associated pods. The Sites are interconnected directly or using ISN (inter-site network).

To answer your question: yes, you can build the IPN (to interconnect ACI Pods) using a single device.

- If we can use only one device - is PIM BiDir still a requirement ? I don't really see the point for only one IPN device. Simple IGMP snooping should work, no ?

For Multi-Site, multicast in underlay is no longer a requirement. For Multi-Pod, your IPN needs to support multicast forwarding. PIM BiDir is kinda strongly suggested for it's scalability.

- Any recommandation of a cheap Cisco IPN device that would do the trick ?

Again, I am not aware of the prices, but any N9K would do.

Hope it helps,

Sergiu

[AlexNastas74909]

Sergiu, 

Thanks. As always your answers are straight to the point. 

I have made a mistake however. We are planning a Multi-POD setup, not Multi-Site. Sorry about that. 

So a couple of follow-up questions to make sure my understanding is correct: 

- Can the SPINE devices in a multi-pod network be connected back-to-back ? 

I assume no ? 

- you mentioned that I can use only one device. I must be misunderstanding something, but PIM is a multicast routing protocol. But if it needs to be activated only on the IPN device (one device) then what will it route ? 

Thanks

Alex

[Sergiu.Daniluk]

Hi,

No problem. Always a pleasure ^_^

- Can the SPINE devices in a multi-pod network be connected back-to-back ? I assume no ? 

That is a good assumption. There is no support for back-to-back Pods.

- you mentioned that I can use only one device. I must be misunderstanding something, but PIM is a multicast routing protocol. But if it needs to be activated only on the IPN device (one device) then what will it route ? 

That is correct, IPN can be composed of a minimum one router, but you have different subnets for each spine interface facing the IPN. So topology is like:  Spine_Pod1 ---- subnet1 --- subinterface .4 IPN subinerface.4 --- subnet2 --- Spine_Pod2 . You also need to bring up OSPF adjacencies with the IPN router. 

Let me know if you any other question.

Cheers,

Sergiu