ACI static route clarity

[JonathanC1]

Hi folks,

This is driving me a little crazy! And maybe this is easier than I'm thinking through. I need to add a static route for a network we have that is on ACI ( a BD l3 unicasting routing enabled.) We used to have a static route on IOS but now I'm confused to even if this is possible.

The BD on ACI is 10.10.20.1/24 and the IP the server has is 10.10.20.12 - we need to route the 10.2.199.0/24 subnet towards this IP (10.10.20.12.) There are other hosts on this BD/EPG.

Would this be a layer 3 out static route? The server in question is currently bound to a EPG statically for the VLAN it is on. If it is a l3out can it be both if that makes sense?

Kind regards

J

[Sergiu.Daniluk]

Hi @JonathanC1 

Funny enough, the scenario described by you is a recurring topic for discussion with some of my customers.

The problem in ACI when it comes to static routes is that you can only configure them in a L3Out and only pointing to an external next hop. Why? Because in the philosophy of ACI, L3Out is the construct which points you to prefixes outside of ACI fabric. From the same point of view, the EPGs (or BDs to be technically accurate) are the subnets inside (or behind) ACI fabric. In other words, you cannot have other prefixes (and static routes for those prefixes) behind an Endpoint from an EPG, because is a `bad design`.

What will happen if you point a static route in a L3Out to a next hop which is an EP? I didn't tested but I think the route will not be programmed. Or if it is programmed, then the policy enforcement will definitely not work.  So either way will not work.  At least in the current ACI images. Maybe in the future the future will be added, though I don't think it will happen.

What is the solution to your problem?

There are two solutions which I see:

Option1: if there are not a lot of IP addresses in the subnet, create host routes (/32) in EPG. YES, you can create /32 routes in an EPG pointing to an endpoint. It looks like this:

Info: you add a /32 host route, check "no default svi gateway", select "EP Reachability", and type the EP which is the next hop.

Downside of this is if you have a big subnet. You can automate it, but still it will look ugly

Option2: redesign. You need to move the subnet behind a L3Out, either by moving the endpoint which holds the prefix (IP readdressing is necessary) or you move just the prefix itself.

Hope it helps.

Sergiu

[weylin.piegorsch]

Hi @Sergiu.Daniluk ,

I want to thank you for this explanation.  I've been struggling with finding a sane explanation of static pervasive routes, this is the first I could find.

If I may ask a follow-up?  We have an L3 FW with a PBR policy, so that it won't participate in OSPF.  We want it to do NAT for the occassional subnet, and I therefor need to static route a pool to it that doesn't exist anywhere else in the fabric.  How would I accomplish this?  The static route in the L3out was working ok for internal traffic, but it doesn't advertise the NAT pool to OSPF peers.

---

[Sergiu.Daniluk]

The solution to your problem is to move the L3FW behind a L3Out. This way you will have the capability to have additional static routes to it, independent of PBR. The route can then be advertised to other L3Outs.

And of course, the PBR can still be used, with slight adjustments: https://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/white-paper-c11-739971.html#PBRdestinationinanL3Out

Stay safe,

Sergiu

[weylin.piegorsch]

Thank you @Sergiu.Daniluk. I was finally able to build a suitable lab environment to do the testing, and it worked perfectly.  I wish I could mark this as a second solution.

[RedNectar]

---

@weylin.piegorsch wrote:

Thank you @Sergiu.Daniluk. I was finally able to build a suitable lab environment to do the testing, and it worked perfectly.  I wish I could mark this as a second solution.

---

That's why it's important to ask new question instead of resurrecting old ones! Time Cisco implemented my suggestion

[iores]

@Sergiu.Daniluk 

Hi,

when you say "you need to move the subnet behind a L3Out" what exactly do you mean? Which subnet? For example, if hosts are in ACI behind 10.10.10.0/24, and if I need to configure static route for 192.168.100.0/24 pointing to 10.10.10.10 (host in ACI), how will this work if static route needs to point out only to external next-hop? 

[weylin.piegorsch]

@iores,

From what I was able to research, static routes can be provisioned in two ways:

1. Within the L3out
2. On the EPG

Static routes program on the L3out are only used by the L3out (I think), even if they can be advertised by a routing protocol operating on the L3out (which seems *really* weird to me, allows for inserting a routing loop on the IGP).

If 10.10.10.10.10 is the next-hop address for which you want to direct 192.168.100.0/24, and if that endpoint is in an EPG (where the BD is assigned 10.10.10.0/24), then you can program 192.168.100.0/24 into the EPG, pointing to that address.  The kicker?  You can't program the /24 on the EPG - only /32's.  So, you would program 256 (254?) individual /32's that way.

If you have dynamic routing on the L3out, and if you want the /24 to be advertised.... well, I think you can redistribute the set of 250 /32's.  There might be a way to summarize them into the /24, but I haven't yet got that far in my research.

weylin

[Sergiu.Daniluk]

Hi,

By “move the subnet behind L3out” I mean to move the entire subnet, including the subnet’s gateway, behind an external router. 
this way, from ACI’s perspective, you will have the subnet behind L3out. Then routing is simplified. 
sure, you have the “ugly” second alternative of 254 host routes, as described by @weylin.piegorsch  in it’s second half’s reply. 

Anyway, if you are still unsure what to do, follow the simple rule of thumb: endpoints should belong to directly connected subnets (BDs), external prefixes should point to L3outs next hops. Do not mix these two.

Stay safe!

Sergiu

PS: @weylin.piegorsch  “Static routes program on the L3out are only used by the L3out” - not sure what you mean by this, but just to clarify. Static routes configured in L3outs are redistributed into the fabric, as per the L3out’s consumed/provided contracts, so that all the endpoints from consumer/provider EPGs can use those routes. Cheers.