ACI - UCS IC`s and VPC

[Jostein]

Hi All

We have for many years been running ACI with VPC to UCS IC`s . And it has been working fine.

We are in the progress of setting up a new ACI fabric, that will connect to the same UCS IC`s

Both ACI fabrics run with a physical Vlan Pool 1-500,900-1200, which is mapped to the VPC for IC`s.

When we set the new fabrics VPC`s to active, it will make a lot of our VMware guests not reachable. Shutting down the ports again and it works like a charm.

The existing VMware guest runs with a DVS vlan pool (500-899).

New ACI fabric has just basic configuration with no tenants, BD`s etc. The only thing we have attempted is to connect to IC`s.

Anyone have any ideas ? The are no errors showing in either of the ACI fabrics before/when this happens.

[Robert Burns]

There's lots of missing details we'll need to provide any sort of assistance.

1. Is VMM integration setup?

2. Do you have a single AEP in use with both VLAN Pools and Domains assigned to it? Is this AEP assigned to the VPC Policy Group for the UCS FI's? (fabric interconnects).

I hope you aware that if you're running VMware ESX hosts on UCS Servers (behind an FI), you can't use LACP.  The only option is to apply a vSwitch policy using MAC-Pinning.  For the VPC interfaces directly connected the leafs, you can use LACP VPC no problem.  This is because LACP implies upstream link aggregation, and UCS FIs are NOT aggregated switches (VPC Pairs).   See this post for further details: https://community.cisco.com/t5/application-centric-infrastructure/aci-integration-with-ucs/td-p/4436168

Let's start here and get your input on above, then we'll advise further.

Robert

[Jostein]

Thanks for your reply Robert.

1. No it`s not setup yet. But that is the plan.

2. There are 4 AEP`s for connecting the FI`s. FI1A_AEP , FI1B_AEP  and FI2A_AEP, FI2B_AEP.  
These assigned to same vlan-pool  and pyshical domain.
Have only tested the FI1A_AEP so far, wich caused the issues.
FI1A , portchannel group 2, Containg Port 1/39 and 40 , which we have at leaf1 port39 and leaf2 port39.
The leaf switches are VPC paired.
Only have 1 vlan pool and domain currently setup in new ACI fabric.
The vlan pool contains the same vlan`s as the old enviorment, currently connected to the FI`s.

As far as mac-pinning and VMware ESX host behind FI, this what we currently do in our old aci fabric.
And will have to look at when implementing VMM again.
Just waiting for the server guys to get vcenter up and running.

Let me know if anything is unclear, or some more information is needed.

Regards

[Robert Burns]

You really don't need 4x AEPs. An AEP is a way to represent a physical network/environment (Legacy Network, DMZ, etc).  In most cases a single AEP will suffice in ACI - especially for UCS - this is considered one domain/environment and therefore should use a single AEP.  Keep it simple.  The problem is that you can only bind a single AEP to a [VPC] Policy Group, and the VMM can only belong to one AEP.  So in short, single AEP for everything is the way to go.  

That single AEP can be linked to your Physical Domain (with single static VLAN Pool), and your VMM domain (with single dynamic VLAN Pool). Of course, always avoid overlapping VLANs across ANY pool.

We also have a special integration app for UCSM & ACI - External Switch App.  This App can automate the VLAN programming on both ACI & UCSM so you don't have to manually pre-create all VLANs and assign them to each host vNIC.  Instead it's offers an on-demand way to plumb ACI VLANs dynamically to UCSM & vNICs as ACI allocates them during EPG > VMM domain binding.  This app become especially helpful if/when you apply microsegmentation (or intra-EPG isolation) where you have to assign PVLAN pairs on any intermediate switch (like UCS FIs).  This task becomes automated with the App. Some details for this here: https://dcappcenter.cisco.com/externalswitch.html
First thing you should do is clean up and simplify your AEPs though.

Robert