09-15-2021 04:20 PM
Hi guys,
I'm working on an HLD for a customer that asked for a DC network architecture for:
The idea is to propose ACI because it meets most of the customer requirements, the supposed solution is a Multi-Site architecture made up of:
I have few doubts:
Thanks a lot in advance!
Solved! Go to Solution.
09-24-2021 12:27 PM
Here's a sample topology that would be similar to yours. You can ignore the WAN connection on the bottom. The IxN devices at each Pod/Site need to run OSPF between them and the Spine switches, but within the Inter-Site/Pod-Network you can run anything. We only need IP reachability to the other devices. However you want to interconnect the IxN devices is up to you.
Yes you're correct that onPrem CSRs are needed regardless. There's work being done with ExpressRoute/DX in the future that may remove this requirement, but for now they're required. They also serve to encrypt your ExpressRoute traffic which wouldn't be otherwise.
Robert
09-24-2021 03:20 AM
Any help?
09-24-2021 07:12 AM - edited 09-24-2021 07:13 AM
I'll offer insight where I can. Answers inline below.
[FYI - Multisite Orchestrator (previously MSO) is known now as NDO (Nexus Dashboard Orchestrator) going forward]
Regards,
Robert
Considering this as a greenfield, is it right to start with the combination of Multi-Pod and Multi-Site, instead of using only Multi-Site?
[Robert] So this comes down to cost. Running Multipod saves you from having an extra APIC cluster to manage. Multisite is going to be needed regardless if/when you go to Azure though, and its also extremely helpful when you want to deploy consistent policies across multiple fabrics (like the Active fabric & DR fabrics and/or Cloud sites). To use these fabrics with multisite you're already going to need the Advantage level license for every leaf at every site, so the question of using Mpod+Msite vs. Msite only comes down to cost vs. benefit:
Is it better to use the same IP network for the IPN and ISN or keep them separated?
[Robert] Most customers dual-task their physical ISN/IPN devices.
I read on the Multi-Site white paper that the uplink interfaces for the IPN and ISN should be the same on spine nodes. Does this mean that I can't use back-to-back connectivity with dark fibers for the Multi-Pod environment and MPLS for the ISN?
[Robert] Correct. It's not supported to mix back-to-back connections with ISN configuration like MPLS. These deployments are mutually exclusive.
Could I connect the fabrics to Azure using the IPN/ISN deploying IPsec devices on that network, through ExpressRoute links?
[Robert] The point of ExpressRoute is that it gives you a secure link directly into the cloud. Typically customers will use IPSec for ISN to Cloud connections that transit the Internet.
What if I would like to use 1 Express Route terminated on the IPN/ISN and 1 internet link in any fabric?
[Robert] This is possible. You use the ExpressRoute as your primary cloud connection, and use IPSec as backup connection for the same site. Routing could be configured to prefer one or the other, but ExpressRoute will offer a far better SLA. Another option to cut costs a bit would be to use ExpressRoute or IPSec in one site and should there be a failure reaching the cloud from an OnPrem site, it can leverage the DCI to the other ISN devices and egress towards the cloud from there - which could be ExpressRoute or IPsec. No issues mixing & matching, but you don't want to overcomplicate your design too much. Colos (such as Equinox) are another cost-effective options for ExpressRoute/Direct Connectivity to the public cloud providers.
Is it better to deploy the Multi-Cloud environment with IPsec tunnels between CSRv routers in the infra Vnet and ANG objects in the user Vnets, or use Vnet peerings?
[Robert] CSRs must be used for Cloud to Cloud (ie. AWS to Azure), or Cloud to OnPrem without ExpressRoute. For anything internal to Azure vNet Peering is the way to go when lower latency & Higher bandwidth is required. For Regions that may not support vNet peering you can still use VPN/IPsec. Keep in mind there will be a cost premium for vNet peering vs. VPN GWs. Regardless of your choice, cloud APIC/Msite will handle the management and deployment of these intra & Inter-region peerings. There's a good compare/contrast between these two Vnet connectivity options here https://azure.microsoft.com/en-us/blog/vnet-peering-and-vpn-gateways/
09-24-2021 09:47 AM
Hi Robert and thanks for your support.
I still have few doubts:
[Robert] Most customers dual-task their physical ISN/IPN devices.
Could you provide an example diagram of a dual-stack IPN/ISN network?
I was looking for mixed IPN/ISN with two possible designs:
[Robert] The point of ExpressRoute is that it gives you a secure link directly into the cloud. Typically customers will use IPSec for ISN to Cloud connections that transit the Internet.
I've understood that Multi-Cloud requires IPsec routers to terminate IPsec tunnels with CSR1000v regardless of the type of transport you have, because the Cloud APIC automatically configure an infra Vnet with 1 or more CSR1000v devices and IPsec connections to IPsec devices on prem.
So my idea is to connect a couple of routers to the IPN/ISN network in the same fashion described before (as the IPN/ISN devices described before), and those routers will terminate IPsec VPNs and route the traffic originate by the CSR1000v routers to the IPN/ISN.
Many thanks!
09-24-2021 12:27 PM
Here's a sample topology that would be similar to yours. You can ignore the WAN connection on the bottom. The IxN devices at each Pod/Site need to run OSPF between them and the Spine switches, but within the Inter-Site/Pod-Network you can run anything. We only need IP reachability to the other devices. However you want to interconnect the IxN devices is up to you.
Yes you're correct that onPrem CSRs are needed regardless. There's work being done with ExpressRoute/DX in the future that may remove this requirement, but for now they're required. They also serve to encrypt your ExpressRoute traffic which wouldn't be otherwise.
Robert
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: