Cisco UCS 5108 Port Configuration

[abrown0011111111]

Hello All,

I am new to the Cisco UCS server world and am setting up a Cisco UCS 5108 blade server. The server has two Cisco UCS 6324 Fabric Interconnects that I have done the initial configuration on and I am attempting to configure the ports for the blades. Looking through different articles and tutorials after setting the global policies, I see everyone setting up uplink and server ports. From what I've read the uplink ports are plugged directly into the switches (I am working with two cisco nexus switches), and the server ports are used to connect to the chassis. 

I am wondering once the server ports are configured what exactly are they supposed to connect to? I assumed they would also connect to the nexus switch with the uplink ports; however, everytime I configure the server ports and plug them in, the switch seems to get flooded and we lose all connectivity. If I unplug them, connectivity is restored almost immediately. 

The current configuration I am working with is two uplink ports on each fabric interconnect (4 total, 2 into each nexus switch), two server ports on each (4 total, 2 into each nexus switch). The only other item plugged into the nexus switches is a SAN which will be configured as boot and storage for the UCS 5108. 

[Kirk J · Accepted solution]

It sounds like you have a UCS Mini (6324), with 4 10Gb ports (each FI/IOM) along with with 40Gb QSFP port that can provide network uplink connectivity,,, or if configured as a server port, could be used to connect to a compatible Cisco UCS rack server, or connect to one additional 5108 Chassis with 2204XP IOMs.  The blades installed in your initial 5108 chassis housing the 6324 FI/IOMs have internal connectivity to the FIs/IOMs without the need to configure 'server' ports.. 

Please take a look at some of the visuals in the datasheet below.

http://www.cisco.com/c/en/us/products/collateral/servers-unified-computing/ucs-6300-series-fabric-interconnects/datasheet-c78-732207.html

The Unified ports can also be configured as FC ports for upstream FC switch connectivity, or directly connected to FC storage processors.

After looking at the datasheet, let me know if you have additional questions, and I'll attempt to address them.

You won't actually need to configure any ports as 'server' ports unless you are connecting rack servers.

Please configure any ethernet type SFPs connected to your upstream nexus switches as 'network'uplinks.  I'm assuming you are not planning on a disjoint layer 2 config (where each FI has multiple sets of uplinks going to different upstream devices, or the same device with different allowed vlans on each link).    If you are, we can have a separate thread on how you need to configure that.

Thanks,

Kirk..

[abrown0011111111]

Kirk,

Thank you for that explanation, it was very helpful. I do now have the ports configured as uplinks into the two nexus switches and it is working well.

I have been using the following guide which seems to provide a good basic configuration of a cisco UCS system with SAN boot.

https://speakvirtual.com/2013/02/08/cisco-ucs-101-installation-and-basic-config-2/

I do run into a couple of issues when creating the port channels and fiber port channels that I was wondering if you may be able to help answer. Whenever, I create the port channels and add the uplink ports I get an overall status failed and no operable members. I also am unable to set up any FC Port Channels in the SAN tab like in the above guide. I am not sure if there is additional configuration required to set these up.

Thank you again for your help!

[Kirk J]

For eth port channels or san port channels, are the corresponding links/port-channels on your northbound switches already configured with identical port channel vlans/vsans, links speeds, etc?

Everything needs to match within your local port channel, as well as the other side of the link.

There are a few bugs that can impact san port-channels depending on what san switches you are using.

What is on the other side of your eth port channels and San port channels?

Thanks,

Kirk