UCS Clustered ESX vHBA and vNICs best practice guides

terran0925 · ‎05-27-2010

Are there any guides for a 2-node clustered UCS configuration? I was able to find a guide like this: https://supportforums.cisco.com/docs/DOC-6158 as well as some other guides on the "Tips and Examples" section but I can't seem to find one that drills deeper in the failover policies for vNICs and vHBAs when setting up ESX on the blades.

Thanks.

Jeremy Waldrop · ‎05-28-2010

Are you referring to the vSwitch failover policies on the VMware side? What adapter do you have in your blades?

terran0925 · ‎05-28-2010

Hi Jeremy,

We're using the Menlo card.

What I would like to know is how to properly configure the vNIC properties within UCSM. I have 2 vNICs set up in UCSM where vNIC eth0 points to Fabric A and vNIC eth1 points to Fabric B.

What I'm not sure about is whether I should enable the "Enable Failover" option in UCSM.

I would also like to know how we're supposed to configure ESX's vSwitches with this setup.

I also noticed that although I have vNIC eth0 and vNIC eth1 set up in UCSM, in ESX, it only see's 2 vmnics and not 4. The set up I'm working with right now is a UCS Cluster setup with 2 x 6102s, 2 chassis. I remember that when I set up ESX with only 1 chassis and had only 1 vNIC eth0 setup, ESX see's 2 vmnics. This leads me to believe that with this clustered setup and the 2 vmnics shown, ESX is not seeing the second vNIC eth1.

Jeremy Waldrop · ‎05-28-2010

I would not enable failover on the UCS side because you may encounter unpredeictable MAC address cache issues on upstream switches. I would let the vSwitch teaming and failover handle that.

The hardware failover in UCSM is great for Windows or Linux servers that have only on vNIC configured in the service profile.

In the service profile you should configure 2 vNICs, 1 on fabric A and 1 on fabric B. ESX will then see 2 vmnics. The max vNICs you can present with the Menlo adpater is 2. These two vmnics should then both be added to a single vSwitch or dvSwitch.

terran0925 · ‎05-31-2010

Thanks Jeremy. I faintly remember that even when I set up a single fabric with 1 chassis and presented 1 vNIC from UCSM, I still saw 2 vmnics in ESX. I'll confirm that a bit later but will disable failover from UCSM for now. Thanks.

terran0925 · ‎05-31-2010

Hi Jeremy,

I just disabled the failover in UCSM and noticed that the service profiles that were derived from this template still has that option selected. Is it necessary to disassociate these dervied service profiles from each blade and recreate the service profiles from the updated template?

Jeremy Waldrop · ‎05-31-2010

You should use updating service profile templates instead of the default of initial. Or even better use updating vNIC templates and tie those to an updating service profile template so that when you make a change like the failover setting or adding a new vlan you only need to modify the template.

In your specific case you will have to modify each individual service profile or start over with an Updating service profile template/vNIC templates.