-- Cisco UCS B200 M4 blade with Cisco UCS VIC 1340 (Adapter 1 / UCSB-MLOM-40G-03) and Cisco UCS VIC 1380 (Adapter 2 / UCSB-VIC-M83-8P) adapters. -- Firmware is UCS Manager 4.0(1b) but was an issue in 3.x firmware as well. -- Hosts run VMware ESXi, 6.5.0, 10175896, but issue existed in lesser builds as well.
-- MAC Pool configured, size 256 and assigned sequential
-- vNIC Templates consist of eight vNICs: -- two for management, one on FI-A and the other on FI-B (eth0 and eth1) -- two for vMotion, one on FI-A and the other on FI-B (eth2 and eth3) -- four for various VLANs, split round-robin between FI-A and FI-B (eth4; eth5; eth6; eth7)
-- LAN Connectivity Policy is configured with all eight vNICs
-- vNIC/vHBA Placement Policies have been created, one for Linear and another for Round Robin
In the Service Profile Template, I have played around with the vNIC/vHBA Placement trying to only use the Cisco VIC 1380 adapter, which is Adapter 2. My placement policy is set to Linear so according to Cisco documentation, vCon 1 and vCon 2 should be for Adapter 1 and vCon 3 and vCon 4 should be for Adapter 2 in a blade with two adapters.
Under vCon 3, I have the following:
So...in UCSM, when I go to Equipment > Chassis > Chassis 1 > Servers > Server# > Adapters > Adpater 2 > HBAs or NICs, they are all on Adapter 2! This seems exactly like what I am looking for.
However...in VMWare, click on Configure Management Network > Network Adapters, my network adapters are out of order (by MAC Address) and spanned across Mezzanine Slot 1 and Chassis slot 6! How? Why?
Ok, so this calls for drastic measures. I consulted the VMware article https://kb.vmware.com/s/article/2091560 which describes how VMware ESXi determines the order in which names are assigned to devices. So, rather than manually reorder the file and reboot, I decided to "nuke it" by using the 'Reset System Configuration'. The system came back on-line with the same problem of half the Network Adapaters on the Chassis and the other half on the Mezzanine.
1. Have you added extra vnic/vhba in a service-profile after OS install?
2. Was ESXi installed after creating SP with linear placement?
3. What is the actual/desire order?
4. Have you moved the SP from one blade to another where OS was already installed on target blade with diff SP?
Since you haven't mentioned, 3rd Gen VIC card have diff architecture, having 2 PCIe lanes internally (Call 'host port'). So you have to consider them assigning manually to each vnic's/vhba's
As an example, let's say we have configured 4 vNICs and 2 vHBAs in the service profile. With the automatic placement, the vNICs and vHBAs will be split between the host-ports:
host-port 1 : vnic0, vnic1, vhba0
host-port 2: vnic2, vnic3, vhba1
This load balancing will be seen in the Actual Order column of the vNIC/vHBAs.
Here we can see an obvious split between the vNICs and vHBA in terms of their "Actual Order". Using this method, we can confirm that vnic0, vnic1 and vhba1 are on host-port 1, and vnic2, vnic3 and vhba2 on host-port 2.
Unless you manually set the Admin Host Ports, load balancing may occur on every vnic add/delete which will change the host-port of existing or old vnic/vhba.
Thank you for replying. Just wanted to follow-up and say that I ended up just doing a manual re-ordering on one of my hosts using localcli. I am going to create a host profile and see if I can "fix" the ordering on other hosts as was suggested in a VMware KB article. I had updated UCSM to 4.0(1b) and also the Host Firmware. This whole thing was more or less an experiment to see if I could actually get the "desired order" to be the "actual order". I also needed to clean-up and rebuild most of the Service Profiles so I built new Service Profile Templates. The only thing I preserved was the WWNN Pool since we boot from SAN and I did not want to have to reconfigure zoning.
A few answers to your questions, though. This particular host is a B200 M4 with a Cisco VIC 1340 MLOM and Cisco VIC 1380. The reason I tried a 'Reset System Configuration' was that I wanted the reset VMware OS to enumerate the devices again and see what order they would be in. The reset did re-order the vNIC and vHBA, but still in a wonky order.
-- No extra vNIC or vHBA were added, at least this was not the root of the issue. I played around with changing the number of vNIC from eight to one and then back to eight, but then had non-contiguous MAC addresses. At one point, I didn't even have MAC addresses near the same range as one another. They were all over the place. I altered my MAC Pool and created two pools, one for each Fabric Interconnect and reflecting such in the MAC Pool. I also set the assignment order from default to sequential.
-- ESXi was originally not installed after creating Service Profile with linear placement, but doing the 'Reset System Configuration' essentially did make this so.
-- The Service Profile was not moved from one blade to another and these blades have only ever had VMware ESXi installed.
-- The desired order is basically to have 1:1:1 with the lowest to highest vmnic match the vNIC and the MAC address increase from low to high as well. See below for an example. This is now my "reference host".
SymptomsCisco Prime is unresponsive. Web GUI will not load. Device doesn't respond to ping.DiagnosisPrime chassis is off.SolutionSince the CIMC uses Flash you can no longer use your bowser. SSH to CIMC address instead. show chassis detailChassis:Powe...
Very first data cloud — Evolution of data lake to hybrid cloud
Today, in this digitized world, data is being generated at a very high rate and from many different sources, and it is growing exponentially in many use ca...
PXE (Preboot eXecution Environment) Server allows unattended or automated OS installation over the Network. The main benefit of pxe is that we don’t need any bootable drive to boot OS(Operating system) and we do not need to to burn any ISO file into DVD o...
New episode on the Cisco UCS X-Series Modular SystemListen: https://smarturl.it/CCRS8E23 Follow us: https://twitter.com/CiscoChampion The Cisco UCS X-Series Modular System simplifies your data center, adapting to the unpredictable needs of...
Join us for this #CiscoChat focused on how you can unlock the power of your hybrid cloud infrastructure. A panel of Cisco and industry experts will discuss Cisco’s hybrid cloud strategy and dive deep into the future-ready infrastructure behind it all.