Introduction
This document provides a review of best practices for NX-OS upgrades on the Cisco MDS 9000 series switches. NX-OS release 5.2.x is used as an example throughout this document. NX-OS command syntax and DCNM-SAN GUI displays may vary slightly depending on the code. Impact on upgrade procedures may vary based on the current NX-OS release and advanced features running on the Cisco MDS SAN. Please review NX-OS release notes prior to upgrading to any code release. Verify that a backup has been completed for all Cisco MDS switch configurations prior to the activity.
Requirements
This document is designed for NX-OS 5.2.x release. If your switch is running software that is earlier than Cisco NX-OS Release 5.0.x, you must upgrade to Release 5.0.x before you can upgrade to Release 5.2.x. For releases 3.3.2, 3.3.3 and 3.3.4x, upgrade to release 4.1.x or 4.2.x and then to release 5.0.x and so forth.
NX-OS Software Images
Each switch is shipped with the Cisco MDS NX-OS operating system for Cisco MDS 9000 Family switches. The Cisco MDS NX-OS software consists of two images: the kickstart image and the system image. To download new Cisco MDS 9000 Family software including Cisco NX-OS and Cisco DCNM management software, refer to the Storage Networking Software download site.
Preparing for NX-OS upgrade
Complete these steps during the week prior to the upgrade.
Step 1. Verify current Sup & Code. Use following commands
show module (verifies if new code will be compatible)
dir Bootflash://sup-active (verifies enough space on active supervisor for code)
dir bootflash://sup-standby (verifies enough space on backup supervisor for code)
Step 2. Download the code version required from Cisco website.
a) You must be logged in with your CCO account
b) Download both System and Kickstart images
c) Make sure to download code for each type of switch platform required. Make sure that Sup-1 vs Sup-2 code is downloaded from the results of the “show module” command.
d) Put code in <Server>\code\ folder on a central server
e) Rename code to short names. Examples (kick522a.bin, sys522a.bin)
f) Gather “show tech details” output from each switch.
Step 3. Clear any old code files off of supervisors (NOT current code level). Use following commands
dir bootflash:
delete bootflash:filename
Step 4. (T)ftp new code to each switch
a) “copy sftp://user@<IP Address>/code/***.bin bootflash:
b) Use password=”<password>”
c) dir bootflash: - make sure code is in there
d) Note: Perform this copy one switch at time, otherwise the FTP server may overload and consume more time.
e) Run install check command on each switch to verify code version works
show install all impact system bootflash:sys_code.bin kickstart bootflash:kick_code.bin
Steps before the Upgrade
Step 1. Log and Feature validation
a) Show log last 200 (to check for any dormant issues)
b) Show cfs lock (fix any current locks or clear them)
c) Show zone status (look at each line that has the word “session:” make sure it says ‘none’). If there is a lock, it must be cleared before proceeding.
Step 2. Backup syslog and ftp out to share
a) Show logging logfile >switchname.log
b) Show flogi database (write down summary total for each switch)
c) Show flogi database >switchnameflogi.log
d) Copy switchname.log sftp://<User>@<IP Address>/code
e) Copy switchnameflogi.log sftp:// <User>@<IP Address>/code
f) Clear logging logfile
g) Clear cores
h) Clear counters interface all
Step 3. Check DCNM for any stalled issues
Step 4. Take screenshots of every switch using DM (save locally for later)
Step 5. Use SANScreen or Other application to check for any path related issues
Step 6. Contact SAs accordingly if any host is single-pathed
Step 7. Inside DCNM - Click on Admin, Callhome, then click on the Profiles tab (If Applicable). It should look like the following
If not showing similar, then update. To update, click anywhere in the “system env…” area and add/remove what is not showing. Apply the change, then save the configuration to the switch.
Step 8. Copy off the Custom Port-Monitor policies if any from the running config (If applicable)
a) Show running-config | beg port-monitor
b) Delete all custom port-monitor configurations
Conf
No port-monitor name <Custom Policy1 >
Delete port-monitor name <Custom Policy1>
No port-monitor name <Custom Policy2 >
Delete port-monitor name <Custom Policy2 >
End
c) DCNM-SAN (Remove/Disable Alerts to NOC during Upgrades) (If Applicable)
1) Click on SAN in Logical Domains then Events/RMON in Physical Attributes pane
2) Check both 64bit & 32bit Alarms to remove all alarms
3) Shift-Click all items to highlight them all
4) Click on the Red Delete button and complete with the validation test.
NX-OS Upgrade
It is recommended to upgrade MDS switches in following order:
• Single (1) Core MDS switch
• Remaining Cores & Storage Edges
• Host Edge switches
Follow these to upgrade NX-OS
Step 1. For all (non-FCIP) fabric using DCNM-SAN GUI
a) In DCNM – Select Tools/Install/Software
b) Check all switches to be upgraded and hit next
c) Click Skip Image Download check and hit next
d) Click window in each switch System and Kickstart to choose the code and hit next
e) Click the Parallel radial button and hit Finish
f) Monitor the status until complete
Step 2. For all (FCIP) fabrics using CLI (optional)
a) Install all kickstart bootflash:kickcode system bootflash:syscode
1) Say ‘Yes’ at the validation test output
2) If files don’t exist, they could possibly be on backup sup. Use following commands
copy bootflash://sup-remote/sys_code.bin bootflash:
copy bootflash://sup-standby/kick_code.bin bootflash:
b) Keep entering ‘show modules’ until all show upgraded.
c) ‘show install all status’ also works. You will get logged out from CLI once during the process. Log back in as soon as possible
d) When complete, move on to next switch or troubleshoot.
Post Upgrade Checks
This section provides step-by-step guidance on validating the upgrade process.
Step 1. Re-verify SANScreen or other Application paths. Contact SAs accordingly
Step 2. Back out only when absolutely necessary. Some back outs require switch reboots
Step 3. Verify against DM (Device Manager) screenshot
a) Use command "show flogi database" for this.
Step 4. For Core switches
a) show fcs ie (looking for all switch names to show up as ISL’d to core)
b) show cfs peers (looking for all switch IPs connected to core)
c) show interface fcx/x (use any port-channel to see that traffic is flowing)
Step 5. For All switches
a) show module
b) show version
c) show flogi database
d) show clock
e) dir bootflash:
Step 6. Remove old code after all switches in fabric have been upgraded
a) dir bootflash:
b) delete bootflash://sup-remote/filename
c) delete bootflash://sup-standby/filename
Step 7. Add Port-Monitor back in using previous configs or add new configs (if applicable)
a) Copy the new custom port-monitor configs back into the running-config and activate
b) Save the running-config with "copy run start"
Step 8. Verify that all devices are still logged into their Zones/Zonesets
a) Use command "show zoneset active vsan XX". All entries should have an asterisk next to the zone member entry.
Related Information
