Cisco UCS Manager not discovering C240 server

[miguelcantu]

Hello, Cisco community!

Currenlty UCS manager shows two Fabric Interconnects, and two Fabric Extenders, but for some reason my server isn't showing up

Here is the birds eye view:

I have my C240 connected with two 10GbE twinax cables, each going to one fabric extender. (2 FEX total)

I have two 10GbE twinax cables connected from the fiber module of each FEX to each Fabric interconnects. (2 FI total)

The fabric interconnect ports L1 and L2 ports are connected to their corresponding ports. So their connected to each other.

I run 'show cluster extended-state' on the primary FI and this is what i get:

FI-A# show cluster extended-state

Cluster Id: 0xebaad0e28ded11e3-0x86d4002a6a153d84

Start time: Wed Feb  5 16:07:44 2014

Last election time: Wed Feb  5 16:23:26 2014

A: UP, PRIMARY

B: UP, SUBORDINATE

A: memb state UP, lead state PRIMARY, mgmt services state: UP

B: memb state UP, lead state SUBORDINATE, mgmt services state: UP

   heartbeat state PRIMARY_OK

INTERNAL NETWORK INTERFACES:

eth1, UP

eth2, UP

HA NOT READY

No device connected to this Fabric Interconnect

After configuring the server-side ports on each FI, the server still doesn't show up

Any idea are highly appreciated!!

-Alex

[Jennifer Casella]

Hey Alex -

How's it going?  What type of adapter card do you have in your server?  If you only have one cable to each FEX, can I assume VIC 1225 in single-wire mode?  Before the UCSM can recognize your rack mount server, you need to make sure that you are at the appropriate firmware level and that CIMC settings are reset to factory default so the FI can discover it via DHCP.  What type of fabric interconnects do you have in your setup?  The C240 M3 should be at CIMC/BIOS level 1.4(6)/1.4(7a)... this is an M3 server, correct?  Let me know if you can share more details.

Thanks,

Jen

[Walter Dey]

Do a ACK of the 2 FEX !

[miguelcantu]

Hey, Jennifer. Thank you for your response!
I am indeed using a VIC 1225 in single-wire mode.

I remember CIMC on the C240 M3 was giving an IP before I configured the FIs, could this be a problem?

I have a Cisco UCS 6248UP Fabric Interconnect.

The C240 M3 is at firmware version 1.5(3). Could this be the problem?

Thank you again, Jen!

-Alex

[Jennifer Casella]

Hey Alex -

The CIMC needs to be in DHCP mode in order to get an IP out to it from the FI.  Go into the CIMC settings and restore them to Factory Defaults.  There should not be any static IP configured on the CIMC for UCSM integration.

Jen

[miguelcantu]

Hi, Jen.

I should of mentioned that my C240M3 has upstream connectivity through a 1GbE link connected to the top-of-rack switch (a cisco catalyst).

CIMC probably got it's IP address from the DHCP server running on the 1GbE network -- this could be my issue.

Currently, my Fabric Interconnects don't have upstream connectivity. I tried this to see if it did anything:

1. Unplugged C240 from the 1GbE network

2. Left the VIC1225 plugged into the FEX, which are plugged into the FIs.

3. Reset the CMIC to Factory Defaults

CMIC couldn't get an IP address after I did this, and the server still won't show up on UCS Manager.

Do the Fabric Interconnects need to have upstream connectivity for this to work?

UCS Manager shows my two Fabric Interconnects and Fabric Extenders, but the server still won't show up.

-Alex

[Walter Dey]

Did you configure any KVM out of band IP management pool: LAN / IP Pool / IP Pool ext-mgt

If not, your CIMC will not get a IP address.

Even without Nortbound connectivity, you should see your server.

However, any vnic/vha will be down (this is controlled by the Network Control Policy, which by default is set to:

Action on uplink fail = down)

[Jennifer Casella]

Agreed - server discovery should still happen without upstream connectivity.  You might also want to doublecheck to make sure the adapter card and server BIOS versions are in line with the configuration guide specs, as this matters for discovery as well.

Jen

[miguelcantu]

It's good to know that the Fabric Interconnects don't need upstream connectivity, so we can scratch that from our list of possible issues.

For clarity, here are the important firware versions:

CMIC - 1.5(3), cisco recommends a minimum of 1.4(6)

BIOS - 1.5.1c, cisco recommends a minumum of 1.4.7a

vNIC - 2.1(1c), cisco recommends a minumum of 2.1(0.367e)

So, we can cross out firware mis-matches from the list.

wdey,

I tried going trough UCS Manager and looking where i could configure the out-of-band IP management pool, but I can't find the right place to do that. Here is a screen shot of the VLAN tab of my UCS Manager.

Could you eleborate more on the process of how to do this? This might fix the issue.

Also, I noticed that in the BIOs setting screen, under Advanced -> LOM and PCIe Slots configuration -> PCIe Slot ID: 2(VIC 1225) it says Option ROM Status Not Loaded. Do you think this might have something to do with server discovery?

Thanks alot for the help guys It's really much apprecaited.

-Alex

[Walter Dey]

Here the IP pool for KVM (CIMC), which has to be in the same IP subnet (Vlan) as the out of band IP address of your

FI 's and VIP.

[miguelcantu]

That's interesting.
My UCS Manager does show IP Pools ext-mgmt. Do I have to do anything in particular to get this to show up?

-Alex

[Walter Dey]

Did you configure the pool ! this doesn't happen automatically; I've seen situations, where there was no IP pool, or the pool was exhausted, and discovery of servers failed; no communication with CIMC's possible

[Walter Dey · Accepted solution]

Hi Miguel

It looks to me, that you are running a rather old UCS version.

In my screenshot, you see the KVM ext-mgt pool under LAN tab

In a older version (I believe < 2.1), this was under the ADMIN tab. Please have a look

[miguelcantu]

You're right, wdey. I'm running UCS Manager v2.0(5a). You know..I bet you this is the reason server discover isn't working.

I was reading through the single-wire integration requirements, and they state

• A Cisco UCS system that is running Cisco UCS Manager release 2.1.

I'm probably going to have to upgrade.

What do you think?

-Alex

[Walter Dey]

Hi Alex

2.2.1c has just been posted ! I would definitely advice to move to this release ! I have seen quite some trouble with CIMC, BIOS versions, in versions > 2.2.1b.

Very simple: just upgrade first UCSM, and then go with autoinstall infrastructure, and autoinstall server

I'm convinced your problem is solved, finally !

Good luck