First Time UCS Firmware Upgrade

[elliottj1]

Hey everyone. I am new to UCS only since August 2015. So, I am a little timid about a request TAC has put on me. 

Currently we are running firmware 2.1(3A) across our UCS server systems. We have a server today that is causing an issue, which is relevant to a new bug they found in this firmware. 

They are requesting I upgrade to another Firmware. However, I have no test environment, so I am stuck 1. Trying to choose the best firmware. Looks like 2.2 (6e) is the latest. We are running Vmware 5.1. All systems SAN Boot over FC. 

I have a word document TAC gave me on the upgrade instructions, looks simple enough. I am just also worried about the ability to roll back to an older firmware if issues crop up with a newer version. 

They also indicated upgrading firmware will cause no downtime. Call me crazy (since I am new to UCS) but how is it possible to upgrade the firmware on a Host without it rebooting?

Thanks for all the thoughts. 

[Clifford Aldan]

Follow the guide Upgrading Cisco UCS from Release 2.1 to Release 2.2 to perform the upgrade

UCS upgrading is a 3-phase approach

Phase 1: Upgrading UCS infrastructure components

• Infrastructure components consist of the UCS Manager software, the Fabric Interconnects, and the IO Modules on each Chassis 
• The FIs reboot one at a time, starting with the secondary
• The guide discusses verifying that the path is up before proceeding with the primary, which won't reboot on its own - requires manual execution to perform 
• When the FI reboots, all IOMs connected to that FI will reboot as well, effectively downing that path
• During the FI reboots, only one path is down at any given time, so the network itself is still available, albeit in a degraded state (one path at a time until both FIs/IOMs have been upgraded) 

Phase 2: Upgrading UCS blade server firmware

• Performed via Host Firmware Package policy
• Upgrades the CIMC, BIOS, Board Controller, CNA/Network Adapters, Storage controllers, etc
• Ensure maintenance policy is set to "User Ack" to prevent instant reboot of all blades
• With "User Ack" enabled, this allows rebooting of the blades to be staggered
• Perform reboot of each host for the policy to take affect 
• For example, if your blades are running VMware ESXi and are members of an ESXi cluster with HA/DRS enabled, you can set each ESXi host into maintenance mode (to evac all VMs off that host safely) and reboot them one by one 
• Once the server is done, it will be back online and can exit maintenance mode to host VMs 

Phase 3: Upgrading OS network and storage drivers

• Not mentioned in the upgrade guide, but is necessary to ensure drivers are meeting compatibility based on the UCS release
• Refer to the Cisco UCS HW and SW Interoperability utility for more information on supported drivers