Kubernetes integration with ACI

[brlehigh]

Greetings all,
I have a prod ACI fabric running 4.2(7s) in network-centric. Connected to this fabric is a VMware cluster consisting of around a dozen physical ESXi hosts. We're doing VMM integration, so there aren't any static path bindings for these servers.
I have been asked to setup a Kubernetes integration with Calico. The Kubernetes cluster is running on the VMware cluster.
I'm reading through Cisco Application Centric Infrastructure Calico Design White Paper and having a heck of a time wrapping my head around the L3Out piece, since I thought they needed to be mapped to a physical interface. This makes sense if the K8s cluster were running on bare metal, but since it's running on an existing VM environment, how do we provision the L3Out? 
The white-paper states that floating SVIs are available in v5.0 and higher, however, I do see a Floating SVI option in the L3Out wizard in my v4.2 environment. This looks like the best option but I'd like to confirm before proceeding. If this is the correct route, do I need to add node information for every leaf switch in my fabric that has an ESXi host attached?

If there are better resources available that explain the integration process, I would greatly appreciate them.

Thanks,

-Brian

[aleccham]

You are going down the correct path since your K8 nodes are VMs then Floating L3 out is the correct configuration. This is done so that if your K8 nodes move between vPC pairs then your L3 connectivity will still remain. 

Do I need to add node information for every leaf switch in my fabric that has an ESXi host attached?

yes

[jacalzad · Accepted solution]

Hi Brian,

The white-paper states that floating SVIs are available in v5.0 and higher, however, I do see a Floating SVI option in the L3Out wizard in my v4.2 environment.

The floating L3Out was available since 4.2.1 where it supported VMM domains. On 5.2 they added support for physical domains.

If this is the correct route, do I need to add node information for every leaf switch in my fabric that has an ESXi host attached?

You should only need to create an anchor node (or nodes) with a primary IP and floating IP when using a VMM domain. Once the virtual router moves to another leaf (non-anchor node), ACI will deploy the floating IP there to maintain connectivity.

The following link can help clarify:

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/kb/Cisco-ACI-Floating-L3Out.html

HTH,

Jaime