Connecting Esxi Host with ACI Fabric

[Ahmed Sabanaa]

Dears ,

I have a connected Host to our ACI fabric with the below Properties :

1- the Vcenter Domain Has Been Created With VLANs 20,40-70, 80 PooL  at the same time that I have many Vm clients inside it with VLANs differs from the Vcenter Domain VLAN Pool , lets focus on One Of them VLAN 100 with a VM machine created in Side 10.10.100.10/24  .

2-We Have Switch Connected To the ACI Fabric through Trunk with Allowed VLANs (All VMs VLANs )and the physical Domains that the Connection Between the Switch and the leaf use is configured with the Same Vlans That Has been Allowed (All VMs VLANs ) on the Interface switch Port .

3 -there is EPG created to host the 10.10.100.10(VLAN10) Vm Machine , and has been connected to the both Vmm Domain (VLANs 20,40-70, 80 )and the Physical Domain that has been used on the SW- To -ACI Fabric Connection .

 My question is , how do the ACI Fabric know where to go to reach the 10.10.100.10 Vm since the connection between the Fabric and the host going through physical domain configured with VLANs 20,40-70, 80 PooL  and the VLAN 100 is not included ?

[Tomas de Leon]

Ahmed,

My apologies in advance for making any assumptions or incorrect assumptions. But I do not know or understand what you are trying to accomplish by reading your comments and looking at your attached topology picture.

From the Picture, there are two physical host connections to the ACI Fabrics.  This is either a VPC or two individual connections. First assumption, a VPC.  An each of these links are uplinks from a VDS(or DVS). Then there are two additional physical links to external switches.

It looks like 20,40-70,80 VLAN encaps are configured for VMware Networking links and the External Trunks.  And that VLAN100 is NOT configured on these links.  As a result, VLAN100 is secluded and will not communicate to the  20,40-70,80 VLAN encaps in or thru the ACI Fabric.

If you have a vswitch with uplinks to other external switches which do carry VLAN100 externally and have routes that will route\forward traffic thru an L30ut to the ACI then traffic can reach the 20,40-70,80 VLAN encaps in or thru the ACI Fabric.  You can also acomplish this with another physical connection that carries vlan100 to the ACI fabric as a L2 extension to one of those external switches, but there needs to be another HOST connection with VLAN100 outside the ACI Fabric.

Thank you for participating in the Cisco Support Forum for ACI! If you have other questions related to this post, please let us know. If this response answers your questions, please mark this post "answered" and assign a rating to the response(s) provided. This will help notify other viewers that your question(s) is answered and this helps us provide better responses for this and future questions.

Thank you!

T.

[Ahmed Sabanaa]

hi ,

Sorry its A DVS, And Individual Connections  , and the VLANS 20,40-70, 80 IS the VMM Domain VLAns , and the VLAN 100 is included within the allowed VLAN , I have A mistake on the Attached Image , Thank You >

[Robert Burns · Accepted solution]

To ensure you understand how VMM domains operate in ACI let me offer the following:

-Your VMM domain VLANs, should be a dedicated VLAN range.  They only need to be allowed between the fabric and the ESX Host only - no where else, including the uplinks to your other switches.  This range of VMM VLANs are specific only to ACI and do not have to exist outside the fabric.  This is because as you create EPGs and bind them to the VMM domain, a random VLAN is allocated from the VLAN pool associated to the VMM domain for each EPG.  ACI does not care which specific VLAN is used, since ACI manages the end-to-end communication between the fabric and Virtual Machines.   Everything above here references a vDS/AVS created/managed by ACI. 

-If you want to have your VMM domain endpoints (Virtual Machines) reach the external switches, then you need a static VLAN pool with the appropriate VLANs (usually known from your existing/legacy network ranges).  If you want one of your VMM managed endpoints to communicate to destinations attached externally to the fabric (such as connected to the uplinked switches) then you would need a physical domain binding for each EPG with the specific VLAN you want that VM traffic to use. 

I'll use my own example since I'm not sure which VLANs are your legacy production VLANs vs. ones you've defined specifically for VMM.

VLAN Pools:

"vmm-vlans" dynamic (range 300-400) - These VLANs will be reserved for VMM usage only and do NOT exist in the prod network.

"prod-vlans" static (range 100-200) - These VLANs will be used for traffic 'leaving' the fabric connected to the product network (SW1 & SW2).  These VLANs are your existing/legacy VLANs in use.

EPG_Web:

VMM domain binding

Physical domain binding, static path mapped to Leaf ports to connected to SW1 & SW2 configured with trunk encap vlan-100

In this example, when an EPG_Web is deployed to vCenter as an ACI-managed vDS Port Group, it will pull one of the VLANs from the dymanic VMM pool.  Here we'll say it allocates VLAN 300.   You do not need to specify this VLAN to the VMM domain binding manually - that's what the dynamic VLAN pool does for us.  For the static path bindings for the EPG you're going to manually assign the appropriate VLAN for your Web traffic.  In this example we'll assume your "web" VLAN is vlan-100. 

Let's look at some scenarios now. 

1. VM to VM communication within Fabric

If you have two 'web' virtual machines within the fabric that need to communicate, they would both be attached to the ACI managed vDS port group 'EPG_Web'.  This communication would occur over VLAN 300 within the fabric. 

2. VM to External endpoint connected to SW1/SW2

The VM's traffic would still use VLAN 300 within the fabric, but when it needs to exit the fabric it would re-tag the traffic on VLAN 100 using the interfaces defined in the static path binding.  On the return path, the external endpoint would respond on VLAN 100, that traffic would reach the Leaf Interfaces and be reclassified into VLAN 300 and continue to it's VM destination.

The whole purpose of these dynamic VLAN is to break the 1:1 mapping for ACI managed Virtual endpoints against specific VLANs which may or may not ever need to leave the fabric. 

Make Sense?

Robert

[dennislarge]

Hello Robert. Thanks for the info above, it helps my understanding. I do have a couple of questions, though. 

Your description helps me understand the usage of internal and external vlans. For traffic leaving the fabric, you say to define a physical domain in the EPG to the external switch. Would that not be an external L2 domain? At least, that's what I've got setup.

Also, for the purpose of migrating services into the fabric, for the VMM (or baremetal?) domain vlan pool, I would need to redefine the UCS vlans accordingly, correct? (to match what is in the new dynamic pool)

Thanks - dennis 

[eahmed007]

Hi Robert ,

I am going to implement ACI fabric and will integrate Firewall , Load balancer and UCS Server with VMWare .

Can you tell me how can I integrate VCenter first time as VCenter will be of one the VM  .Should i create ip address manually  in VCenter and create vSwitch in VMware .

I am bit confuse to connect UCS balde System with VCenter .

I am looking forward to your assistance and what would be best practice to connect UCS Blade Server with VCenter first time .

Your valuable comments and guideline is highly appreciated .

With regards

Erfan

[dennislarge]

I went thru something similar recently with another "chicken and egg" scenario, where I needed to define iSCSI storage to even allow for building VMs. The servers had no local storage. My suggestions may not be the best, but it works.

First, the AAEP for the UCS will have at least 2 domains associated, one for hardware, one for VMM.

Second, the UCS servers (via templates of course) should have separate nics available for standard networking and ACI VMM. Since vShere really likes redundacy, I would have 2 for mgmt (standard vkernel sw), 2 for vMotion (ditto), 2 for vm data (standard switch with a mgmt vlan portgroup) and finally 2 that will be put in the eventual VMM switch.

Build the vCenter VM, put it in that standard vm data switch, and you should be off to the races.

Hope that helps.

[Robert Burns]

What does your physical topology look like?  I'm interested in how UCS it connected to ACI and/or your legacy network, also where is vCenter hosted?

Provide more details and I'll explain how to set it up.

Robert

[eahmed007]

Hi Robert ,

I am going to share the basic LLD for your reference.Please assist me how i can connect vCenter and UCS blade system integration for the first time with ACI fabric .

I need initial steps by steps process to connect ACI fabric with vCenter .

I am looking forward to your reply soon.

With regards

Erfan

[eahmed007]

Hi Robert ,

I am waiting for your valuable comments as per my provided topology .

Please help me on this case .

With regards

Erfan

[Ahmed Sabanaa]

First Of All I would Like to thank you so much My Dear ,it was very  good explanation thank you so much .