Connectivity between blades in Cisco UCS environment

[Lee Harris]

We're trying to setup a UCS environment for testing purposes, and seem to be unable to establish any connectivity between servers within the UCS. At the moment, the environment has no upstream LAN connectivity and therefore no uplinks defined on the fabric interconnects. We had wanted to run the test within the UCS keeping it isolated, and connect in via the management interface to UCS Manager. I suppose my question is, is it possible to get the blades to talk to each other without uplinks defined and connection to an external network?

Many Thanks - Lee

[jejoseph]

You can do it by

a. Changing the 6120 from end host mode to Switch mode

or

b. http://docs.google.com/View?id=dhfgpk58_100f5kcqcfc

Disadvantage: NIC failover will not work in both scenarios.

[simon.wright]

Lee,

Without an uplink network switch you will not have any connectivity between the blades, when I did this I connected it to a nexus 5K and a Nexus 2K.  both were not connected to the main network and I was able to play / test the kit.

The Fabric Interconnects are the endpoints for the nics and not the cna's in the blade. So without them being connected you will get no connection, even between blades in the same chassis.

SImon.

[cco-iamrodos]

Further to what has been mentioned, you can stay in end-host mode but change the behaviour for no uplinks, which is the Network Control Policy.

Not sure where it is on the GUI (it may be CLI only) but its described at http://ciscosystems.com/en/US/docs/unified_computing/ucs/sw/cli/config/guide/1.0.2/CLI_Config_Guide_1.0.2_chapter15.html

The config item is

set uplink-fail-action {link-down| warning}

The description is

Specifies the action to be taken when no uplink port is available in end-host mode.

Use the link-down keyword to change the operational state of a vNIC to down when uplink connectivity is lost on the fabric interconnect, and facilitate fabric failover for vNICs. Use the warning keyword to maintain server-to-server connectivity even when no uplink port is available, and disable fabric failover when uplink connectivity is lost on the fabric interconnect. The default uplink failure action is link-down.

When you have no uplinks all the NICs look down! This is by design. Remember you have a fabric A and a fabric B. When you have a fabric that gets isolated do you want it to fail the Ethernet links so that the blades know about it and the network failover software in your OS can send the data over the working fabric. Or do you want your blade to not know and keep sending data out, but if its destined for outside the environment its not going to get there.

Hopefully that adds some more clarity to your situation.

Rodos

[stechamb]

Hi Lee, as you probably know you need an external route to switch between the fabrics (that crossover link is for control plane, not data plane) and between VLANs - so if you want blades inside UCS to talk to each other with not external LAN, then all blades must be communicating via the Virtual InterFace (VIF) on the same fabric and VLAN.

In your (crippled) environment with no northbound LAN you could force the situation by only using one fabric in standalone mode and making sure the VLANs are correct.

Let me know how you get on :-)

Steve

[Lee Harris]

Thank you all for your replies. I'm new to this UCS stuff, and not having a Cisco networking background is not helping. We now have managed to configure the environment so our Blades can at least talk to each other internally within UCS, but the more work we do in this environment, the more obvious it seems that we are going to need an uplink switch. Right now we're trying to figure out how we can get something existing within the UCS environment to be able to talk to the UCS Manager interface without uplinks or an external switch... but I think I already know the answer to this one.

Thanks again, your help is very much appreciated.

Regards - Lee