ACI L3Out BGP to NXOS switch pair

[romanrodichev@iementor.com]

Hello,

If you are building a BGP L3Out from a pair of border leafs to a pair of NXOS switches, would you choose to create two L3Outs, one per NXOS switch, each with a vPC SVI on the Leaf side, and a single SVI on the specific NXOS switch on the NXOS side? Then run iBGP between NXOS switches, each with two eBGP peers to two leafs, total four eBGP peers?

I'm curious if anyone would choose to try to consolidate two L3Outs into one, by creating a four port vPC with a single SVI on leaf side and a vPC SVI on the NXOS side? I'm aware of the historic limitations of L3 Routing over vPC VLANs, but wonder if they apply here. There's no HSRP on the NXOS side and no shared IP on the Leaf side.

Or would you choose four separate Routed interfaces with four BGP peers and a single L3Out? Or possibly even only two Routed interfaces with two BGP peers and a single L3Out?

Thanks!

[richmond · Accepted solution]

I would always use a single L3Out object if the external routing domain is the same for the peers. In your example if the BGP peers are all part of the same upstream network then it would be one L3Out. If they were different upstream networks (e.g. one is a partner organization and the other is your network core) then I would lean towards an L3Out each. This makes it easier to assign subnets to L3 EPGs have have traffic match the correct L3 EPG for policy enforcement.

How this is implemented on the leaf interfaces is a separate matter. Do you need to run routed multicast? If so then SVI is not an option. Generally having your routing protocol peers follow the physical cabling is simpler to troubleshoot. I would probably run routed sub-interfaces and have an eBGP peer per physical link, that way you get to take advantage of fast external fallover and ECMP.

[romanrodichev@iementor.com]

Thanks for getting back to me. Great answer!