Mgmt Within ACI

[Steven Williams]

How are others out there designing their Out-of-Band mgmt access for devices within ACI?

Separate tenant? Separate vrf? Separate BD?

Pros and cons of each?

[Claudia de Luna]

Hi @Steven Williams,  

ACI is set up for management out of the box.  Recall that it comes with a mgmt tenant as one of the three default tenants. So you already have tenant isolation (including vrf) and EPGs to use out of the box with out of band or in band management.    This is always where your ACI Fabric management is configured.

[Sergiu.Daniluk]

Sooooo many opened tabs there ^_^

[Claudia de Luna]

LOL...why does everyone always say that??...and I had just cleaned up! :D

[Sergiu.Daniluk]

haha, don't get me wrong. I also like the tabs ^‿^

[Claudia de Luna]

Maybe we need help :D ... tabs anonymous?  I had to clean up because I had so many tabs open I lost a page!

[Steven Williams]

What about things that are not ACI infrastructure? Like a physical appliance or a physical server that is being built inside ACI?

[Claudia de Luna]

I've configured "management" type EPGs and BDs for clients when they require it.   Normally those have been EPGs/BDs in the common tenant or in their "production" tenant.  It really will depend on your design.
The most common scenario for those (that I have seen) is to put them on the out of band management network for the fabric.     I don't think we pay enough attention to that.  That is practically the first thing I start talking about when first engaging with a client on ACI.     I always recommend an OOB Mgmt network with at least the following subnets/vlans (network, compute, infrastructure, security).   Network has the ACI OOB connections and any other network appliances, compute has all the ILO/DRACS if the design calls for that, infrastructure had things like UPSs, cameras, etc and security has OOB management links for security devices if the design calls for it.

[Steven Williams]

So the common practice is stick these ILO and other OOB connections in the mgmt tenant and create different BD for possible different types of OOB or maybe even different tenants that require them, You are also saying that these types of networks can exist in the tenant that requires them and have a BD in the prod network. 

[Claudia de Luna]

I would not say that.  I tend to leave the mgmt tenant alone.  I always recommend a management design like below.  The thinking here is that I want a network separate from my fabric to manage my fabric.  I want it secure (hence the firewall) and I want the layer 3 gateway off my fabric and as close to my wan/internet connections as possible to reduce the things that can fail between me and my fabric.  Also, I don't want to burn ACI fabric interfaces for management.  I'd rather use cheaper switches (with some redundancy) for that.  

If you want to use your fabric interfaces for management interfaces, of course that can be done, and at that point its really a matter of your design and what you are trying to do.  Don confuse the built in mgmt tenant with a general mgmt tenant.  When I have built general management constructs its been half in the common tenant and half in some other production-ish tenant..depending on what the client is trying to do and if that general management subnet would need to be shared across tenants.

[Sergiu.Daniluk]

Hello Steven,

Sorry I misinterpreted your question. You were asking about OOB mgmt design for devices/servers connected to ACI, and I understood mgmt for ACI nodes. 

In my opinion, regardless the type of environment your ACI is deployed in, or what services you offer over ACI, the Out-Of-Band management should always be over a dedicated Out-Of-Band network - exactly like how @Claudia de Luna highlighted in the drawing. I know most people would say, there are only corner cases where dedicated network for OOB would really make sense. But trust me, I've been 4 years in Cisco TAC and I've seen so many things go wrong.  A pure OOB management network can save you from a lot of troubles.

Regards,

Sergiu

[Sergiu.Daniluk]

Hi Steven,

There are not a lot of design options when it comes to OOB, as it is binded to the mgmt0 interface on switches and oob management interfaces on APIC.

---

Separate tenant?  Separate vrf? Separate BD?

---

No. You cannot go out of the realm of Mgmt Tenant and management VRF. This is basically the VRF management in NXOS.

Regarding EPGs, well, you can create more of them, but I do not see a scenario where you would need more then one.

The only instance for which I see multiple profiles is the External Management Network Instance Profile - in case you need to create specific contracts for specific external subnets, which will create as a consequence different entries in iptables.

Cheers,

Sergiu