So I'm going to make a few assumptions here, and if these assumptions are wrong, I suggest you get professional assistance: 1) this is test/dev or a lab environment 2) You have/know how to generate the backup files off the UCSM interface. 3) you're not familiar with running UCSM upgrades, but you've have/understand the procedure to rebuild the FI from a Full State backup if needed. You're going to want to read the release notes for the firmware you're going to, check that the version you're on is supported, and verify there are no known issues with the blade hardware/VIC/Host OS etc. There is not usually a simple yes or no answer to these kinds of upgrades. I'm just scanning the documentation, and I'm already seeing some caveats you'll want to double check prior to upgrading from 2.1 to 2.2. https://www.cisco.com/c/en/us/support/servers-unified-computing/ucs-manager/products-installation-guides-list.html https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/sw/upgrading/from2-1/to2-2/b_UpgradingCiscoUCSFrom21To22.html#topic_6505D0253916438AB0D832BA8840A9FA https://www.cisco.com/c/en/us/support/servers-unified-computing/ucs-manager/products-release-notes-list.html https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/release/notes/CiscoUCSManager-RN-22.html https://ucshcltool.cloudapps.cisco.com/public/ I'd also like to point out that your FIs look like they are EOL, and you will want to be very careful making changes. Make sure you have valid backups of everything, and that you have no faults on the system. https://www.cisco.com/c/en/us/products/collateral/servers-unified-computing/ucs-6100-series-fabric-interconnects/eol_c51-709473.html https://www.cisco.com/c/en/us/products/collateral/servers-unified-computing/ucs-b-series-blade-servers/eos-eol-notice-c51-733210.html That being said, these upgrades should not be rushed, they should be planned out. I would hesitate to start these with a single day for validation. I would want to generate an inventory list of what hardware is in the UCS environment, double check hardware / firmware compatibility, and verify that the to - from code is supported. Also verify the drivers installed on the guest OS, as they will more than likely need to be updated after a firmware upgrade. These should be done in a maintenance window as they can impact production if there are issues. Ideally because they are built for redundancy there are protections in place, but if you don't notice the issue, and upgrade both FI's, or don't have User Acknowledge set in the service profile you can cause outages.
... View more
There are also hyper-visor specific changes that need to be applied. See KB articles from VMware: https://kb.vmware.com/s/article/55636 https://kb.vmware.com/s/article/55806 You can also disable the warning if these are not production and you do not intend to resolve the issue. There were also some fixes in the 6.7U2 vSphere, but I've not read through them yet.
... View more
I'd like to also add some suggestions, first make sure you have two policies, a No-Scrub policy and a Scrub policy created in UCSM. Make sure you apply this No-Scrub policy to the appropriate section for your service profiles. If they are bound to an updating template, assign it there and if they are individually created profiles, you may need to edit each one individually. Set all scrub options to no. The policy with the Scrub option enabled should only be used for a scenario when you want to wipe that boot OS off the SD media to re-purpose the blade for another function or want to re-install the image for a fresh start.
If you currently have ESXi installed and already configured I would take a backup before you make any changes that you are not sure are disruptive. The backup of your VMware configuration will allow you to avoid rebuilding the entire ESXi host each time an issue occurs. See https://kb.vmware.com/s/article/2042141 for reverence on how to backup an ESXi host. With VMware Enterprise Plus you can also make host profiles in vCenter to the same effect, but the ESXi backup will work on any version regardless of vCenter integration.
... View more
Kirk is absolutely correct,
Make sure you read the entire upgrade guide, and understand the pre-requisites and requirements for each part. It walks you through the whole process and I usually spend about 4-6 hours to prep for any UCSM firmware upgrades, including environmental discovery, downloading code, pre-staging code, and reviewing the upgrade process for that version of firmware.
A few other recommendations:
Take a full state backup and an all config backup of the UCS system before any upgrade.
You will want to add all applicable firmware packages to the firmware page from the equipment tab before upgrading the infrastructure.
Check to make sure the FI's have space for a new version of firmware to be staged up, if this is an older system with multiple code versions still on the FI's, you might need to remove some of the older code to make sure you have room for the new upgrade firmware packages.
Make sure all maintenance policies are set to User Ack before initiating any code upgrades. If a stand alone service profile was set, or the service profiles are not bound to an updating template (initializing templates will not push out changes, best practice is to have an updating template for all of your blades, depending on hardware/OS installed you will probably need more than one updating service profile template for your infrastructure) you could miss the maintenance policy and it would reboot immediately. **If you're re-binding a service profile back to a template, make a clone first, this will give you a backup if you need to revert back to the original settings. When you re-bind a profile to an updating template you are going to remove any customized settings you might not have been aware of for that profile. The updating template will push it's configuration from the template and over-write anything that was different.**
If you have any hardware/OS that will be required to stay at a lower firmware version, make sure your service profile templates have a firmware policy set, and it is at the required versions selected. If there is no firmware policy in the service profile template, it will upgrade the blades when you run the B-firmware package.
Read the upgrade document completely, and understand the process flow required. I usually upgrade the packages, with the infrastructure first, the b-series second, and the c-series 3rd. Pre-stage the code to the server blades with the Auto-install option once your infrastructure upgrade is completed. The upgrade process allows for setting the new code in the backup firmware version, and when you enter your upgrade window you can resume the upgrade and initiate the server reboots so you won't have to wait to stage the code out to the blades.
I always recommend to my customers that they perform any code upgrade in a designated maintenance window in the event any issues are encountered.
Your IOM cards and FI's are going to be going down one at a time. If your configuration is not symmetrical, make sure you understand what is connected to what, and when it will go offline. I've seen scenarios where backup servers were only connected to FI-A and few other odd configurations, which would have impact on the infrastructure upgrade.
For UCS Mini FI's I would be particularly careful, and make sure that you're in a scheduled maintenance window. I've had a UCSM infrastructure upgrade cause network disruption on a UCS Mini in this scenario, both FI's in the IOM slots went into discovery mode and lost their network, and the chassis had to be re-acknowledged after the upgrade completed. I had TAC on the case after the first FI rebooted, and there was network disruption. They were not able to find a cause, and TAC continued the upgrade, unfortunately the same issue occurred and the network disruption happened twice. This has made me extra cautious on UCS Mini firmware upgrades. This caused a brief outage during the upgrade, and post-upgrade the 5108 chassis had to be re-acknowledged which was an additional outage for that entire chassis.
Before you acknowledge the server reboots, put your installed OS into a state that is ready to be rebooted. Migrate all VMs off and put a VMware host in Maintenance mode (especially if there is no DRS on the VMware cluster), shut down a Windows server etc to ensure that there are no issues bringing down that blade because it will reboot.
The UCSM firmware upgrade process is relatively easy, just make sure you've prepared properly. Checked all your installed hardware compatibility, software compatibility, and any dependencies from your running OS versions. For example, VMware might need the VIB files updated for the enic and fnic drivers after the UCSM upgrade completes etc.
And lastly, be patient, the upgrade process will take some time, and if this is a production environment make sure you have an appropriate window for this maintenance because you're not going to be rebooting all blades at the same time, you will want to wait for the first one to complete, check the environment, and move on down the list.
Hope this was somewhat helpful.
... View more