L3Out design question

[suneq]

Hi experts,
We have a very basic legacy network which works like this (please see the diagram below for simplicity).
All the servers and a LB belong to the same VLAN 100. We have a SVI 100 10.0.100.254/24 configured on the switch, it is used as default gateway of the servers and the LB.
On the switch, we have also a static route 192.168.100.0/24 (VIP subnet of LB) with next-hop 10.0.100.253 (LB).

Now we want to move the servers and LB to ACI.
What we did:
- we configured 10.0.100.254/24 as the subnet of the BD_100
- we put all servers to EPG_100 (associated to BD_100)
Everything was OK until this point.

For the LB, we think we will configure a SVI L3Out to route statically the VIP subnet 192.168.100.0/24 (we choose SVI because we will to add a backup LB later).
However, it seems that we have to change the default gateway of the LB (because 10.0.100.254 is already used for the BD, can we still use it for SVI L3out interface?)
It's OK for us to change the default gateway of the LB for example to 10.0.100.252, we want just to make sure that it will work with the following IP addressing (servers can still join the LB 10.0.100.253)

BD subnets (for servers): 10.0.100.254/24
SVI L3out interface: 10.0.100.252
LB: 10.0.100.253

Many thanks for your help.

[YanL]

if you wish to keep the existing design I would bring the LB in the same L2 network as your VMs (same EPG/BD) and would create the L3OUT using the 192.168.100.0/24 network with a route to 10.0.100.253.

[richmond]

You can't have a BD subnet in a Layer 3 Out a subnet is either configured as a BD subnet or it is configured on a Layer 3 Out, never both at the same time. The fabric has different endpoint learning semantics for the different types of networks so it can't mix them.

If you don't use the load balancer IP address 10.0.100.253 for anything, the simplest solution would be to use a different subnet for transit between ACI and the load balancer and configure this as an SVI in the new Layer 3 Out. Route the VIP subnet via this L3 Out and you are done.

You also have the option to put in /32 routes for the VIPs in the EPG with the load balancer IP as the next-hop. This works only with /32 routes though.

[Hector Gustavo Serrano Gutierrez]

I agree with @richmond,

1.- L3Out pointing to a next-hop in a BD/EPG is not supported. The reason is because a physical or virtual host in a BD/EPG requires an encap VLAN assigned. The configuration of a L3Out also requires of an encap VLAN. Make both encap vlans match is not officially supported.

2.- Support for Static Routes in Bridge Domains was introduced in APIC Release 3.0(2). The specific use case is when the EndPoint has some Virtual IP outside of the BD Subnet range. You create this /32 static route to reach the VIP pointing to the EndPoint IP address within the BD Subnet range. This is not really a replacement for L3Outs.

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/4-x/L3-configuration/Cisco-APIC-Layer-3-Networking-Configuration-Guide-401/Cisco-APIC-Layer-3-Networking-Configuration-Guide-401_chapter_01010.html

3.- As @richmond suggest. The best approach would implement a new subnet for Leaf to Load Balancer communication and implement the L3Out as usual.