UCS240M3 Network Interfaces missing on ESXi 5.1U1

[avansamb]

All,

I just recieved two new UCS240M3 servers, and after installing ESXi5.1, it displays "No compatible network adaptor found.".  I already have three UCS240M3s that have been running ESXi5.1 for sometime without an issue.

In the VMKernel log I see the following error message,

igb 0000:02:00.0 The NVM Checksum Is Not Valid

I tried upgrading all firmware using the ucs-c240-huu-1.5.4-3.iso, no change.

Anybody seen this before, or have an idea?

Thanks,

Aaron

[Manuel Velasco]

Hi Aaron,

From what I can tell, the ISO that you are using doesn't have the drivers for these adapters.

Did you use the cisco custom image for the OS installation? If you didn't use the custom image,please try this and let me know (see below for download links)

Download Cisco Custom Image for ESXi 5.1.0 GA

https://my.vmware.com/web/vmware/details?downloadGroup=CISCO-ESXI-5.1.0-GA-25SEP2012&productId=285

Download Cisco Custom Image for ESXi 5.1.0 U1 GA (update 1)

https://my.vmware.com/web/vmware/details?downloadGroup=CISCO-ESXI-5.1.0U1-MAY2013&productId=327

Be kind and Mark as Answer if I helped.

[avansamb]

Thanks for the quick response Manuel,

I initially tried the vanilla 5.1 ISO from vmware that is running on my other UCS240M3 servers.  Once that failed I went looking for updated versions and have since tried vanilla 5.1U1, vanilla 5.5, and the Cisco Custom ISO of 5.1U1 and 5.5.  All with the same results.  I even tried adding the latest igb driver, 5.0.5.1, after I had installed the Cisco Custom 5.1U1.  No luck there either.

[Manuel Velasco]

Do you have any network adapters installed on this server or are you only using the onboard nics?

If you only have onboard nics, make sure to enable these on the bios (see below). 

[avansamb]

Only using the onboard NICs.

Looks like they are all enabled.

[Manuel Velasco]

Lets make sure the CIMC is pushing this information into the BIOS.

1) Reboot your server

2) When you see the option prese F2 to get into the BIOS

3) Go to the Advance options

4) Under Advaced options go to "LOM and PCIe Slots and Configuration"

5) Make sure the onboard nic are enable there and save

If this doesn't work, my next question would be:

When you tried upgrading the firmware, did you see if the embedded network controller was presented for upgrade(see below)?  If it is not there it might be possible that the server doesn't see this controller at all.

Also send me a screenshot like the one below

[avansamb]

Yeah, the update saw the controller "Intel I350 LOM", I believe that is the one right?  If so, it had an update with this last updater.

BIOS looks right.

[Manuel Velasco]

At this point the last recommendation I have is to try to install a different OS like windows to make sure this is no an installation issue. Also where are you installing the ESXi, I mean are you installing it on the flex flash SD card or the embedded controller? Or what kind of RAID controller do you have on this server?

If the other OS have the same issue you might want to open a TAC case. Note that for windows you have to download the drivers from the cisco site. The intel drivers are there.

[avansamb]

Installing a different OS was also next on my list.  Is there an OS that should just install and run without needing to add drivers?  Maybe recent RedHat or Cent?

I have installed the Hypervisor on the flex flash SD card.  Shouldn't matter if I install on the RAID volume right?  I have the LSI MegaRAID SAS 9271-8i card.

Thanks again for the help.

[Cristian Munoz]

Hi Aaron,

Can we check what the ESXi host is seeing?

We can run this command to see what interfaces are available through SSH.

~ # esxcfg-nics -l

For example:

Name    PCI                    Driver      Link      Speed           Duplex   MAC Address     MTU     Description

-----------------------------------------------------------------------------------------------------------------------------------------                  

....

vmnic0  0000:1b:00.00     igb  <=             100Mbps        Full       ...15:5e               1500     Intel ....

In case the LOM port is displayed (igb), check the driver version and firmware version are compatible with ESXi 5.1U1.

~ # ethtool -i vmnic0

Check this in a different C240 to verify if they are the same?

This may occur cause a missing drivers.The following KB can give us another clue.

Network cards not detected on an ESXi/ESX host.

kb.vmware.com/kb/1034782

I hope this helps!

- Chris,

[avansamb]

It looks like a colleague solved this for me.

After a cold(remove the power cords) restart of the hardware, all seems to be working now.

My co-worker thought there used to be messages when upgrading the LOM firmware telling you to do a cold resart, I didn't see any when applying the latest firmware.

I can only guess the firmware was messed up somehow, and once it was upgraded and activated on cold restart, seems to be ok know.

Thanks Manuel for the help and debugging.