APIC inband and VMM connectivity issues

[petar.forai1]

Hi,

I've configured an inband IP policy (172.16.128.0/20 with an IP pool for all the fabric components)for spine, leaves and the APIC. So far I can reach all spines and leaves from the APIC (172.16.129.14/20) and the switches can reach the APIC as well (from VRF mgmnt:inb via iping).

I can also reach the deafult gateway(172.16.128.1/20) of the mgmtn subnet and I can reach the VMM/vcenter (172.16.128.14/20) and all other IPs from the mgmt subnet from the spine and the leaves, but I cannot connect to those IPs from APIC.

I can ping the same default gateway (172.16.128.1/20) from the APIC via it's bond0.10 inband interface, but nothing else in the mgmt EPG outside the fabric IPs. 

I've formed a management EPG that contains the physical domains where the vcenter is running on (bare metal host) and all other management EP (NetApp, UCS, ESX mgmnt vmk, etc. - all physical domains where the mgmt interface is encap vlan-350 and all static paths for those VPCs) but I cannot add the APIC's single port (leaf1 E1/46 with encap vlan-10). If I try to add that interface the fabric complains that it cannot add  encap on a port where the controller is connected to. 

So what is the EPG config to include the APIC and the vSphere from the mgmt tenant within one EPG to allow communication? 

Any hints appreciated!

TIA,

P

[petar.forai1]

I was missing filters on my contracts to allow any:any on all protos :)

[Blake Parker]

Hi Petar,

I had the exact same configuration and the exact same issue.  The issue is that the build-in mgmt tenant is a special purpose tenant and does not have all of the functionality of a normal tenant.  It is not best practice to have VMM access in the mgmt tenant as this should be reserved for fabric management.

What I ended up having to do to get around this limitation was the following:

1. Remove inband management

2. Create a shared L3 out in the common tenant to routers that are directly connected to the OOB managment network.  The easiest method is to enable OSPF routing.

3. Create a new "Management" tenant.  

4. Create a new "EPG" in that tenant and add your physical domain to that EPG.

5. Create a new bidirectional contract between the shared L3 out and the Management EPG you created so that vcenter can access the APIC and the APIC can access vcenter.

6. Add static bindings for the paths to the devices in the physical domain.

Hope this helps.  You can reach me on twitter @vbootstrap