ACI MPOD with host route advertisement

[tuanquangnguyen]

Hi folks,

With host route advertisement capability in BD (along with dynamic routing L3Out), we can control ingress to which pod.

What if, I have two pods in two locations that also interconnect outside fabric via Leafs (L3Out-Pod1 and L3Out-Pod2). These L3Outs also connect to different zones in each location (non-ACI) via respective firewall pairs. Imagine a topology where ACI also acts as Core network. The outer zones are not interconnecting to each other (except for links coming towards ACI L3Outs mentioned above)

1. Can Pod 2 Border Leaf learn host route from Pod 1 Border Leaf, and vice versa?
2. How does forwarding take place when traffic, say, ingress Pod 1, but the endpoint is in Pod 2 when the respective BD/EPG is spanned in both pods? Does it forward via endpoint learning and IPN, or via host route learned on L3Out-Pod1 and ingress again in L3Out-Pod2?

For point 2, my speculation is that traffic would stay within the fabric (mapped to EPG, then get IRB'ed and traverse the IPN to reach the endpoint in Pod 2). But then, if that is the case, return traffic would egress local L3out-Pod2 which may incur asymmetric routing.

A picture speaks a thousand words

[ADP-89 · Accepted solution]

Hello,

Trying to answer inline:

---

@tuanquangnguyen wrote:

Can Pod 2 Border Leaf learn host route from Pod 1 Border Leaf, and vice versa?

<ADP> It won't happen. There is mechanism in action that will prevent routes originated from ACI to be learned back on a different L3OUT. ACI tags the OSPF/EIGRP routes with an ID based on the VRF. If it receives a route with that tag it will drop it. You can read more on this on the L3OUT White Paper - VRF tag and Transit Routing

How does forwarding take place when traffic, say, ingress Pod 1, but the endpoint is in Pod 2 when the respective BD/EPG is spanned in both pods? Does it forward via endpoint learning and IPN, or via host route learned on L3Out-Pod1 and ingress again in L3Out-Pod2?

<ADP> Based on the above answer POD2 will only know reachability of POD1 EPs via coop. So all traffic will remain inside the fabric/IPN(vrf MPOD).

Hope that helps,

ADP

[tuanquangnguyen]

Hi @ADP-89,

Thank you for your input. The reason I am asking it because this is gonna replace a single pod setup - where currently ACI is the border gateway between server farm, external zones and between locations (with the N7K as the Core Network for the other location) as per below:

I am aware of the L3Out loop prevention mechanism, however, from my understanding it can be overridden by setting different tags from 4294967295. In the MPOD setup that I described, I need such transit routing to happen else external zones in different locations cannot reach each other (due to no external interconnect).

Unless, I have to migrated the original L3Out DCI into a design where ACI MPOD as a whole become a transit (L3Out-Pod1 and L3Out-Pod2 communication happens via Spine-IPN-Spine). Which, to be fair, I have yet thought of. I might as well look into that option.

[ADP-89]

Hi @tuanquangnguyen ,

Thank you for the update.

Not sure if I follow them as I might be missing some infos on the overlay side. Are the EPGs/BDs in each POD on a different VRFs?

If yes, then changing the default VRF tag in one of them would solve the issue. In this case inter-VRF traffic must pass through an external device that merges the two VRFs unless you leak subnets with contracts. 

If not, changing the tag won't have any effect as this is per VRF, and not per L3OUT. If the picture lists all the possible external connectivity block I would definitely think about using ACI as transit router between pods. Possibly moving from OSPF to BGP would simplify your life too in order to control which subnet has to be exported on each side.

Cheers!

[tuanquangnguyen]

Hi @ADP-89,

While there are some can be in different VRFs, there are other cases where the BDs of the EPGs are in the same VRFs. I overlooked the  part where route tagging is per VRF.

I might have to consider other options.

Thank you very much for your support