I have deployed an ACI fabric. Currently its network centric with 1 VLAN = 1 BD = 1 EPG. I have static VLAN pool (1-999) that covers all the VLANs that the servers to be migarated to ACI live on in the legacy datacentre. The pool is associated with a "migartion" physical domain and I have a vPC connecting the fabric to the legacy datacentre. All the EPGs for the migartion VLANs are stically assigned to this vPC with the encap set as the correct external to ACI VLAN. All bridge domains are L2 and the gateway lives outside of ACI currently.
I have some test servers that also have some of these EPGs statically extended to them on 802.1q trunks attached to the fabric (I used the same physical domain) and they can talk to each other just fine and can talk to servers on any VLAN outside the fabric.
Now we want to deploy some UCS B series servers with VMware integration with ACI and migarte some VMs from the legacy DC into the fabric and this is where I am getting confused.
The documentation says that for VMware integration to use a dynamic VLAN pool. How does that work with migration ? A VM on the legacy network will be on say vlan 12 and when its migrated onto the UCS it will be expecting to still be connected to VLAN 12. How do we control which EPG is mapped to which dynamic VLAN in this case ?
When initially setting up the ESXi servers the vSwitch wont be under ACI control as it cant be until ESXi is up and running and has an ACI controlled DVS pushed to it so what goes on with VLAN to PEG mapping here ?
On the vPC from the fabric leaves to the UCS fabric interconnects do I extend the same physical domain and VLAN pool as to the DVS ?
Also is it possible to reuse VLANs in different physical domains ? For VMware servers and VMs for example we might need only VLANs 100 - 200 and for IBM AIX servers VLANs 100 - 150 can I create two VLAN pools with overlapping VLANs and use them with different physical domains and still have the devices talk to each other ?
I am sure I'm missing something very obvious here.
1. You are not forced to use a dynamic vlan pool for your vlan integration. You can configure your vlan pool with static vlans and the dvs will use whatever you define in your epg for that vlan. You are required to use the same vlan on your portgroup as you have in your legacy if you want a stretched network.
2. Not shure about your question here. But when you have your vcenter integration running you will be able to push your epgs down to vmware as portgroups.
3. If you have a physical server connected to the FI directly you need to add your phycDomain to your vPCs. If you are only running a vmmDomain towards your VMware you dont need to configure the epg with the physDomain. That you have already done in the vmmDomain profile.
4. You can create different vlan pools with the same vlans thats okay. But using them on the same interface och vPC I am not shure will work. There is a feature that allows you to run the same vlan as long as it is on different switches. Think its called "per port vlan".
Q: The documentation says that for VMware integration to use a dynamic VLAN pool. How does that work with migration ? A VM on the legacy network will be on say vlan 12 and when its migrated onto the UCS it will be expecting to still be connected to VLAN 12. How do we control which EPG is mapped to which dynamic VLAN in this case ?
A:You do not control which EPG is mapped to which VLAN, it's ACI controlled. Hence the key word "dynamic", although you can "pin" a VLAN statically even if it's a dynamic pool.
However, the key to remember is that VLAN Encapsulations between ACI and end hosts is completely irrelevant to your "Legacy VLAN" that are extended/trunked between ACI and your legacy switch. Within the same EPG, if you associate VMM domains, you only need to add a static path binding for VLAN-12 as tagged. This is the same case for any devices, not just VMM. You can have VLAN-4321 configured as 802.1P (don't use access) for a Dell rack server and then VLAN-12 tagged. Just need to make sure that you have all the correct domains associated. This is the part where "VLANs" does not matter anymore unless it's 802.1Q tag to the servers, where you would probably have to discuss this with your server admins about updating VLAN tagging.
Q: When initially setting up the ESXi servers the vSwitch wont be under ACI control as it cant be until ESXi is up and running and has an ACI controlled DVS pushed to it so what goes on with VLAN to PEG mapping here ?
A: If you are referring to ESXi management connection, it should be static path binding just like configuring any bare metal servers. The switch port should be access mode.
In fact, even when the DVS is pushed to the ESXi, it still doesn't have any mapping until you create an EPG, which will then be pushed down to the DVS as a port-group. If you are using UCS, make sure that the UCS uplinks have been correctly configured with the same VLAN pool configured in ACI
Q: On the vPC from the fabric leaves to the UCS fabric interconnects do I extend the same physical domain and VLAN pool as to the DVS ?
A: On the UCS, you do need to make sure that the uplinks have the same VLANs allowed as configured in the dynamic VLAN pool. You do not need to associate any physical domains, it should be just VMM domains. You need to make sure that LLDP or CDP (depends on your UCS version, pre 2.2(4b) you need to enable CDP, later versions support LLDP). For full detail guide between UCS and VMM, here is a good guide written by Joe Ristaino
Q: Also is it possible to reuse VLANs in different physical domains ? For VMware servers and VMs for example we might need only VLANs 100 - 200 and for IBM AIX servers VLANs 100 - 150 can I create two VLAN pools with overlapping VLANs and use them with different physical domains and still have the devices talk to each other ?
A: Yes, you can. However, generally it is not recommended unless you are using those VLANs for very specific design scenarios with dedicated hardware.
In your case, by default, VLAN overlapping cannot occur if the VLANs are to be assigned to different EPGs, but on the same leaf switch, unless you use "port-local VLAN" which has its own limitations and caveats.
If you have a pair of leaf switches dedicated for just VMs and another pair dedicated just for AIX, then this is not a concern at all, but make sure the AEP, VLANPools and Domains are created separately.
Again, generally speaking you do not need to worry about end-host VLANs unless tagging is required. If using VMM integration, then you just need to make sure the range is enough to cover all the VMs.
We delivered a partner enablement training session in September 2021 to share the ACI upgrade Best Practices.
The slide deck is enclosed here for wider audience in the community, it provides more details in terms of the best practices, tools and co...
What is Cisco ACI Anywhere?What are ACI connectivity options for managing Primary On-Prem DCs?What are ACI options for extending your Data center to secondary remote locations (Physical)?How ACI provides centralized network policy framework for workloads ...
Cloud Networking Community on Cisco Customer Connection
Join our community!!
As a valued Cisco Cloud Networking (former DCN) customer, you can be part of Cisco Customer Connection Program (CCP), Cisco’s global online community program. Connect ...
Join us for a live demo of Cisco Intersight Cloud Orchestrator to learn how you can simplify the orchestration and automation of your infrastructure and workloads across your hybrid cloud environment. We will take a closer look at Cisco Intersight Cloud O...
Hybrid Cloud Demo Series: Simplify Orchestration of Your Infrastructure and Workloads
Tuesday, September 7, 202110:00 am Pacific Time(San Francisco, GMT-08:00)Join us for a live demo of Cisco Intersight Cloud Orchestrator to learn how you can simpli...