Multipod routing preference L3 Outs

[jachbr]

In a multipod deployment where 2 of the 3 pods have L3 outs for internet/external connectivity, is it possible to influence routing to have the 1 pod that doesn't have the L3 out to prefer use of one of the other pods for internet traffic over the other?  

For example, Pod 1 and Pod 2 have L3 outs for internet connectivity.  Pod 3 doesn't have an L3 out but we would prefer that it uses Pod 1 for it's internet connectivity.  I understand the each pod would prefer the local L3 out for it's external connectivity but am looking to see if we can influence Pod 3 to use Pod 1's L3 out rather than have it hashed between Pod 1 and Pod 2.  

A single VRF encompasses all three pods and L3 outs.

[Manuel Velasco · Accepted solution]

Hi Jacobs,

what routing protocol are you running on the L3outs in pod 1 and pod 2?

ACI uses mp-bgp to distribute external routes within the fabric, if you need to have routes with higher preference between your L3outs you can manipulate BGP metrics to accomplish this.  

Now in ACI if your L3outs are running OSPF you can use the interleak policy to manipulate the routes from OSPF to BGP.  See link below

Cisco ACI and Interleak of External Routes

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/kb/b_KB_Configuring_Interleak_of_External_Routes.html

[jachbr]

Thanks for the response.

L3 outs are running OSPF in pods 1 and 2.  

Through the use of the interleak of external routes, will the manipulation of the default route at pod2 affect pod2 preferring it's own L3 out for external/internet connectivity?  The goal is still to have Pod 1 and Pod 3 use Pod 1's L3 out and Pod 2 to use Pod 2's L3 out.

[Jonathan Schultz]

Did you ever figure this out? I have a similar scenario I need to figure out quickly.

[jachbr]

I wasn't able to test but I had a colleague suggest setting a static default route on Pod 3's MPOD L3Out pointing to Pod 1's IPN connection.  

[Jonathan Schultz]

When you say out Pod 1's IPN connection, are you referring to a next hop of the overlay address? I've found it won't use a next-hop of another L3 out.

[jachbr]

Yes, the suggestion was to use the overlay address of Pod 1 but you have proved that isn't an option.  Another option may be outbound route filtering on Pod 2 for the default route.   

[richmond · Accepted solution]

The inter-leak profile approach will work when you have separate L3Outs in each of the other pods.

If you have a single L3Out with borders in Pod1 and Pod2 then you can't use the inter-leak profile to influence the routing as this will set the attributes of the routes being redistributed on the L3Out into MP-BGP on both pods at the same time. I believe there is a feature enhancement coming that will allow more granular control, but I'm not sure which release (hopefully 3.2!).

The other way to do it is to change the metrics in the IPN. Pod3 load shares between Pod 1 and Pod 2 because the IGP cost to the next hop is the same for both pods and BGP is doing multi-path as the routes are essentially the same (take a look at the MP-BGP routing table on the leaves in Pod3 and you will see what BGP is doing). If you fiddle with the IPN metrics so that Pod2 has a higher cost than Pod1 then Pod3 will want to use routes learnt on the Pod1 L3Out over Pod2. Keep in mind this will influence more than just default though, everything will start to want to go that way!