UCS-5801 FI 6248UP upgrade failure

[Fernando Bello]

Hello all,

I have done an upgrade of the UCS system. I moved from 2.0 to the lastest new one. While performing the upgrade everything seemed ok, but after the whole process was finished, both FI 6248UP will not come up. LEDs seems to be ok, but neither the console port is responding nor the UCS Manager IP address. Although I've changed the CONSOLE PORT settings, it will not respond. I connected it to see while the FI was booting and not output received.

Is there any way to reset o check what is happening?

Thanks in advance,

Fernando

[Walter Dey]

Hi Fernando

Did you a manual install, or autoinstall ?

Connect to the management port or console, and issue CLI: show cluster status ?

Can you tell us, in which order you did the upgrade ? very strange that both FI are dead ?

[Fernando Bello]

Hello Walter,

I did the upgrade using the cisco documentation for upgrading UCS system from 2.0 to 2.2.x

I connect the cable to the console port, but no output is shown. No boot messages, no login prompt, nothing is displayed on the screen. I hit enter serveral times or try to write a command but nothing is displayed as well. I've checked the the console cable and adapter and it is working fine. In addition, all lights are green on both Fabric Interconnect, the PSU, the FANs, but if I hit the ID button it does not start flashing a blue light. The stat led it is supposed to be green or amber, but it is off and it never turns on although I waited almost 40 minutes.

thanks in advance for your help,

Fernando

[Walter Dey]

Ciao Fernando

Was this system working ok before the upgrade, or is it a new installation.

I would connect to the console, power off and then power on the fabric interconnect, and see what's going on. I can hardly believe, that nothing is happening, and this for both FI's.

Cheers Walter.

[Fernando Bello]

Hello all,

The system was working ok before the upgrade. I took some pictures to show you what is going on.

On the Picture above, you will see the console cable connected and the laptop display showing nothing although I hit enter. All leds stay green, no amber and then back to green as in normal POST process.

The next picture shows the LEDs status. No console responding as I hit several time the ENTER key.

In red, you can see that the stat led is off. It never gets amber or green. On the other side, the FAN leds are always green. They never change to amber and then back to green as in normal POST procedure.

Finally, I've changed the CONSOLE Cable to a WLC.

As you can see, the WLC displays the console output as I hit enter several times.

So, the CONSOLE CABLE is fine. The CONSOLE settings 9600 8N1 is correct.

Not sure what could have happened to the FIs.

Appreciate a word of advise if there is a fix to this problem.

Thanks in advance,

Fernando

[Walter Dey · Accepted solution]

Hi Fernando

I don't understand, why both FI are dead ? did you kick the upgrade on the second FI, knowing that the initial one is not ok ?

I have seen FI's (and IOM's) going bad during an upgrade ! I would urgently open a TAC case, nothing you can do on your side.

Walter.

[Fernando Bello]

Hello Walter,

Actually I haven't done the upgrade. I was just called in to fix the problem :). Yes, we all believe that we will need to talk to the TAC in order to see if there is a fix to this problem but I would believe that we will need to do an RMA.

Thanks for your help on this matter.

Fernando

[joseph.dunn1]

What was the final resolution? This just happened to us on a 2.0 to 2.2 upgrade, luckily we noticed it after the first FI and the environment is limping along mid upgrade on one FI.

Thanks

[Walter Dey]

Hi Joseph

After experiencing a lot of trouble in the last 12  months with upgrades, I follow the following procedure, which seems to work very well:

Open a TAC case and tell them that you intend to upgrade firmware from x to Y.

Post show tech files

let them check show tech files with any known bugs and Cisco field notices.

[Felipe de Mello · Accepted solution]

Performing a bug scrub or field notice scrub is a special service provided by AS, not TAC. It's best practice to read the release notes and upgrade guides prior to doing an upgrade as well as collecting an UCSM techsupport, Full State backup, and an All Configuration (w/ preserve identities) backup.

TAC should be contacted should anything go wrong during the upgrade or the documentation/process is unclear.

[joseph.dunn1]

We contacted TAC before, during and after and have a RMA on the way. Was quite the debacle. New FI will be here tomorrow and TAC will walk us through replacement and then help us complete FW upgrade.

[Walter Dey]

When opening a TAC case (proactive) you can specify:
Problem Subcategory: UCS-B Upgrade
Problem Type: Configuration Assistance
Software Version: 2.2.3d
Problem Details: Firmware Upgrade 2.2.(3d) to 2.2.(6e) please check for any bugs and caveats

P.S. AS doesn't exist and/or have not the proper expertise in some geographies.

[joseph.dunn1]

We had a 3rd party consultant in walking us through it, and had opened a tac case before hand to verify our infrastructures supported max level. And had a walk through from Cisco. We followed that.. and the FI still died and the new one will be here tomorrow.

Fun Fun.. just not sure how that will affect our in progress firmware upgrade, but we will have a tac tech doing/walking us through that.

[gruberwilli]

Hi,

it's a old case, but I am in the same situation, that I can't connect to the console. Mine FI was rebooted and since then, no way how to get to the console. How is your FI? Did you find any solution which can be done byself?

TIA, Willi

[Keny Perez]

Are your servers still up or this is a new installation?  do you get some sort of error message(s) when you connect to the console port?

Do you happen to be using a USB-to-Console cable converter? You might be missing the drivers for the converter... If using a converter, you may need to check what COM port # is the one you are supposed to use and let us know.

-Kenny