Clarifying In-Band Management IP Assignment in ACI - Confusion

[willytech007]

I’m diving into in-band management communication within the ACI fabric. After setting up access policies, VLAN pools, interface groups, contracts, and other configurations, the next step is to assign static IPs for management. My question is:

Do these manually configured IPs replace the initial DHCP-assigned IPs (from fabric initialization) when using data interfaces (fabric ports) for in-band management?

The book mentions configuring an AAEP with "Enable Infrastructure VLAN" checked, but I'm unclear on:

• What this feature actually does in practice

• When it's mandatory vs optional

• How it interacts with in-band management

I’d appreciate:

1. Confirmation on whether DHCP-assigned IPs are overridden.

2. Any screenshots or CLI examples showing this process.

3. Best practices for avoiding conflicts.

Thank you so much

[RedNectar · Accepted solution]

Hi @willytech007 ,

I’m diving into in-band management communication within the ACI fabric. After setting up access policies, VLAN pools, interface groups, contracts, and other configurations, the next step is to assign static IPs for management.

I do hope you are following the steps explained in this blogpost or maybe this one (Disclosure - the links are to my blog)

My question is:

Do these manually configured IPs replace the initial DHCP-assigned IPs (from fabric initialization) when using data interfaces (fabric ports) for in-band management?

The manually configured IPs are assigned to a Subnet in the pre-defined inb Bridge Domain. At least that's in the way I described how to do it in the blog I mentioned.  I doubt the book you are referring do does it differently, but it MAY assign ALSO inband and/or out-of-band IP addresses to Leaf and Spine switches via the mgmt tenant - but again these inband IP addresses are linked to the special inband EPG  that you need to create (and link to the inb BD) - they have nothing to do with the infrastructure IP addresses that are assigned via DHCP during deployment which belong to overlay-1 VRF found in the infra tenant, not the mgmt tenant.

Here are some screendumps

And if you look via the CLI, you will see things like this on an APIC

admin@apic1:~> ifconfig | egrep -B1 "inet\ "
bond0.202: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500      #Recall that VLAN 202 is linked to inb_EPG
        inet 10.10.2.2  netmask 255.255.255.255  broadcast 0.0.0.0   #This is the IP for inb_EPG shown above for apic1
--
bond0.3961: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500     #VLAN 3961 is the infrastructure VLAN assigned at commission
        inet 10.1.0.1  netmask 255.255.255.255  broadcast 0.0.0.0.   #apic1 is ALWAYS the first IP on the infrastructure VTEP pool
--
docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500        #Always exists for docker. Ignore
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 0.0.0.0
--
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
--
oobmgmt: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500 #This is the out-of-band management IP, also seen 
        inet 172.16.11.2  netmask 255.255.255.0  broadcast 0.0.0.0.   #in the screendump above.  Note NO VLAN 

The book mentions configuring an AAEP with "Enable Infrastructure VLAN" checked, but I'm unclear on:

• What this feature actually does in practice

It is the equivalent of adding the infrastructure VLAN to a VLAN Pool that is linked to a VMM Domain that is linked to this AAEP.

• When it's mandatory vs optional

It is mandatory to use it when you use a virtual switch using VXLAN encapsulation for EPG grouping on the vSwitch, such as the Cisco Application Virtual Engine (AVE). It allows the infrastructure VLAN to extend to the vSwitch so the vSwitch can get a VTEP address on the overlay-1 VRF

At all other times it is mandatory to NOT use it

It is only optional if you are trying to create problems for someone, such as in a troubleshooting lab.

• How it interacts with in-band management

Easy. It NEVER interacts with in-band management

I’d appreciate:

1. Confirmation on whether DHCP-assigned IPs are overridden.

DHCP-assigned IPs are NEVER overridden. 

1. Any screenshots or CLI examples showing this process.

Check the blogposts I mentioned above

1. Best practices for avoiding conflicts.

You should never have either in-band or out-of-band IP addresses overlapping with the VTEP pool.  In fact, you should avoid using any VTEP pool addresses in your network - there's a blog-post about that too (disclaimer - it's another one of mine!)

[willytech007]

Sir can you give one example with the expresion "It is the equivalent of adding the infrastructure VLAN to a VLAN Pool that is linked to a VMM Domain that is linked to this AAEP" then all information is clear.

I was confused about IPs but seeing it well, It's a different tenant jeje, thanks

[RedNectar]

Hi @willytech007 ,

To begin, you need to be quite clear about the concept I call an Access Policy Chain

I'll start with a diagram that I've taken (with permission - it's my blog) from this blog

Now think about every interface on an ACI leaf.  For it to be functional in ACI, that leaf must be tied to an Interface Policy Group, which must be linked to an AAEP, which must be linked to at least one Domain and each Domain must be linked to a single VLAN Pool.

Now think about the interface. You can only have VLANs on that interface that are in a VLAN pool at the end of the chain. And, if you wanted to deploy a AVE virtual switch on one of those ports, you'd need to have the Infrastructure VLAN in one of those VLAN Pools. 

So I hope that explains my comment of "It is the equivalent of adding the infrastructure VLAN to a VLAN Pool that is linked to a VMM Domain that is linked to this AAEP"

But it's actually not quite that simple - the Infrastructure VLAN is special, and Cisco have decide that if you want to include the Infrastructure VLAN in the Access Policy Chain, you need to tick that option in the AAEP rather than add it to the VLAN Pool.

[willytech007]

Very clear! Thanks