UCS C240M3 not showing all boot options

[miguelcantu]

Hi, Community.

I have a C240M3 hooked up to a clusted UCS environment on single-wire-mode, but for some reason I am unable to boot from the network device.

I've configured the RAID controller, consolodating all physical devices into one virtual drive for simplicity.

Here is a screen shot of my boot options:

You can see that my network device does not appear as an option.

Here is a screen shot of my PCIe LInk Speed configuration screen:

As a note, my server is configured with a service profile in UCS, with no errors. It's a pretty basic service profile.

Has anyone seen this before? Does anyone know how I can get at least my network device to appear as a boot option? Any help would be appreciated.

Versions:

BIOS Running Version: C240M3.1.5.1c.0.013120130509

CIMC Controller Running Version: 1.5(1b)

Fabric Interconnect A/B package version: 2.1(3b)A

UCS Manager package version: 2.1(3b)A

Thanks!

-Miguel A.

[miguelcantu]

Looks like the PCIe Inventory isn't recognizing the LOM ports for some reason?

Could this be a hardware issue? I see link lights on the LOM ports, but the BIOs isn't recognizing them.

[miguelcantu]

So, turns out the LOM ports are disabled when CMIC is in UCSM mode.

Once I switched CMIC from UCSM mode to standalone mode I was able to boot from the network device.

However, now i'm getting a bunch of errors in UCSM. I guess this is okay, right?
My understanding is that if you use CMIC in UCSM mode, you can't utilize the LOM ports. Is that correct?

-Miguel A.

[Walter Dey]

see eg.

http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/c-series_integration/ucsm2-2/b_C-Series-Integration_UCSM2-2.pdf

Rack-Mount Server CIMC is not accessible when you start managing the server with Cisco UCS Manager

[Keny Perez]

Miguel,

So you were trying to access CIMC while the server was itegrated with UCSM? That can't be done...

Regards to your question about a bunch of errors in UCSM being normal... what specific errors? You happen to have an screenshot perhaps ?

-Kenny

Cisco Support Community is also present in Spanish:

https://supportforums.cisco.com/community/spanish/data_center

[miguelcantu]

Hey, Keny. Thanks for your attention.

Essentially, I was trying to give my C240M3 internet access through our 1GbE network by connecting one of the onboard 1Gbps port to our top-of-the-rack switch.

I tried doing this while CMIC was in UCSM mode, but it failed. The server could not use the any 1GbE onboard ports at all. It was as if the 1GbE ports became completly non-exsistant to the server. The BIOS didn't even recognize them.

After changing the CMIC mode from UCS Domain mode to standalone mode, the 1GbE ports were then immediatly recognized by the server, and I was able to connect to the internet using the 1Gb onboard ports.

The reason I'm trying to do this is because it's really difficult for us to give our Fabirc Interconnects upstream connectivity to the internet as they only have 10GbE ports.

So, after changing the server CMIC from UCS Domain mode to standalone mode, I got the following erros in UCSM:

Is there no way to utilize the 1Gb onboard ports while the server is in UCS Domain mode?

-Miguel A.

[Walter Dey]

The reason I'm trying to do this is because it's really difficult for us to give our Fabirc Interconnects upstream connectivity to the internet as they only have 10GbE ports.

Sorry, 62xx as well as 61xx fabric interconnect support 1 and 10G uplink !!

Which model of fabric interconnect do you use ?

[miguelcantu]

Hey, Wdey. I run a Cisco UCS 6248UP as my Fabric Interconnects.
So 1Gb uplink is possilbe with this FI model? Do I have to buy the extension go get the 1GbE ports?

-Miguel A.

[Keny Perez]

Miguel,

Well, (please let me know if I missunderstood you) once integrate the rack server with UCSM, your Fabric Interconnects are you gateway to the rest of the world, if you try to integrate the rack server with UCSM and also send traffic through another port connected to another switch, it will not work fine.

Now, it is interesting to see the ports were just not seen at all even by BIOS, but in my experience I had never tried to connect any port on a integrated server to anything else besides a connection that would take me to the Fabric Interconnects (in other words, I have always connect them to a Fabric Extender), with the new version 2.2 is now possible to connect the rack servers to the FIs (without a FEX).

Now, for the lower 2 messages that say "Ha not ready" please connect to UCSM through SSH and run the command:

"Show cluster extended-state"

For the adapter with the link down, make sure there are properly seating cables and let me know.

Rate ALL helpful answers.

-Kenny

Cisco Support Community is also present in Spanish:

https://supportforums.cisco.com/community/spanish/data_center

[miguelcantu]

Hey, Keny.

What you just said makes perfect sense. I will look into getting my Fabric Interconnects upstream connectivity. I need a extension module to get 1GbE ports on the FIs, right?

When I had the C240 server CMIC in UCS Domain mode, these errors were not present. It only happened when I moved the CMIC to standalone mode.

Yes, everything is definitely cabled correctly.

ciscorax-A# show cluster extended-state

Cluster Id: 0xb70190da94e911e3-0x89f9002a6a153d84

Start time: Thu Feb 13 20:10:24 2014

Last election time: Thu Feb 13 20:11:56 2014

A: UP, PRIMARY

B: UP, SUBORDINATE

A: memb state UP, lead state PRIMARY, mgmt services state: UP

B: memb state UP, lead state SUBORDINATE, mgmt services state: UP

   heartbeat state PRIMARY_OK

INTERNAL NETWORK INTERFACES:

eth1, UP

eth2, UP

HA NOT READY

No device connected to this Fabric Interconnect

Thanks, again.

-Miguel A.

[Walter Dey]

I need a extension module to get 1GbE ports on the FIs, right?

NO ! Any Ethernet port on the fixed module of a 6248 /6296 can work as 1 or 10G.

And btw. UCS V 2.2.1b or newer allow you to connect your servers directly to the fabric interconnect without any fex.

[Keny Perez]

Miguel,

As wdey mentioned, any port can be configured for 1Gb, see this :

http://www.cisco.com/c/en/us/products/servers-unified-computing/ucs-6248up-48-port-fabric-interconnect/index.html

"High-performance, flexible, unified ports capable of line-rate,  low-latency, lossless 1/10 Gigabit Ethernet, Fibre Channel over Ethernet  (FCoE), and 4/2/1 and 8/4/2 Fibre Channel"

To configure the speed, you can use the GUI guide:

http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/sw/gui/config/guide/2-0/b_UCSM_GUI_Configuration_Guide_2_0/b_UCSM_GUI_Configuration_Guide_2_0_chapter_0101.html#d44081e1131a1635     Look for "Changing the Properties of an Uplink Ethernet Port"

Note: Ports configured as "server" will always be 10Gb.

Finally, If the domain is configured with no blade chassis then you will not have IOMs of course; you will see HA not ready cause HA is completed through the chassis Midplane (in a blade chassis) which the IOMs communicate through and that is not done through the N2Ks.  This also happens when in a blade chassis you have only one IOM.

Dont forget to rate the helpful answer wdey and I have provided so far 

-Kenny

Cisco Support Community is also present in Spanish:

https://supportforums.cisco.com/community/spanish/data_center