Introduction
This document explains the scenario where the vNIC/vHBA order presented by UCSM is not the same as shown in ESXi. Although in most cases the placement of vNIC/vHBA is irrelevent and wont affect the working of the system, but in some cases the OS/server may look for vNIC/vHBA in a specific slot and if not found it may fail to boot or VM traffic may be affected.
Problem
The example service profile has been configured for 8 x vNICs and 2 x vHBA's. All of the vNICS and vHBA's are assigned to vCON1. A vNIC placement policy with virtual slot 1 is configured as "assigned only" and slots 2-4 left set to the default setting of "all". Note that the vCON is optional.
After changing/adding the vNICs or vHBAs to the service profile the ordering of vNIC(s) and/or vHBA(s) changes and no longer reflects the same order as that presented/configured in UCS.
Software Version
Tested on UCS Manager 2.1 (3a), however valid for all UCSM versions.
vNIC/vHBA Placement Options
Following are the techniques you can use to specify the placement options.
- Let System Perform Placement: Specifies that Cisco UCS Manager determines the vNIC/vHBA placement for the server associated with the service profile. The placement is determined by the order set in the PCI Order table. If you are configuring the service profile/template for iSCSI boot, choose this option..
- Specify Manually: Explicitly assign the vNICs and vHBAs associated with this service profile to a virtual network interface connection. Configure the types of vNICs and vHBAs that can be assigned to a vCon, either manually or through a vNIC/vHBA placement policy.
- vNIC/vHBA Placement Profiles: Assigns an existing vNIC/vHBA placement policy to the service profile. If you choose this option, Cisco UCS Manager displays the details of the policy.
Explanation
The ordering is done according to the PCI order. This has the effect, that if you add e.g. 2 additional vnic's to a existing config with e.g. 2 vhba's and 4 vnic's, the ordering will not work, as described.
Depending of the OS, numbering of interfaces in the Service Profile doesn't match the one the OS discovers. This is because once the OS detects a new hardware in the system, it does a PCI bus scan and can come up with a different order than what UCS server is showing.
Resolution
To fix this issue, first delete all interfaces except vmnic0, and start from scratch. It might mean that vhba's get another wppn, and this will affect you zoning config also. However, the vhba may pick up the same pwwn. (deleting vhba gives the pwwn back to the pool).
Adding or removing virtual interfaces is disruptive (reboot of the blade). Sometimes it is cleaner after a config change, to do a disassociate / reassociate the service profile.
The following VMware document also describes the issue and resolution.
ESXi/ESX host loses network connectivity after adding new NICs or an upgrade
In case the server is in production and reboot is not a viable option, then map the vmnics via mac addresses to UCS vnics and assign the vmnics to correct vswitch (ie. ignore the vmnic number).
A third option is to rename the vmnics, and is decribed in Cisco bug CSCuv34051. Note that the issue is not with UCS firmware or driver.
Related Information
Setting the vNIC/vHBA Placement
vHBA/vNIC desired order does not match actual order
