Nexus N7k 10 slot ISSU failure

[sujohnm]

Hi, 

I am trying to ISSU upgrade our Nexus 7K 10 slot switch from 5.2.7 to 6.2.8a The ISSU has failed multiple times due to the Standby Supervisor failing to come back online  with the message

"

Install has failed. Return code 0x4093001e (Standby failed to come online)

Please identify the cause of the failure, and try 'install all' again.

"

Start type: SRV_OPTION_RESTART_STATELESS (23)

Death reason: SYSMGR_DEATH_REASON_NEED_COPYRS (19)

Last heartbeat 0.00 secs ago

System image name: n7000-s1-dk9.6.2.8a.bin

System image version: 6.2(8a) S2

Exit code: SYSMGR_EXITCODE_NEED_COPYRS (66)

Has anyone else experienced this before and found a solution?, any suggestions are welcome at this stage. I've followed the ISSU guidelines to the core. Please help!!

thanks, Sujohn

[Reza Sharifi]

Hi,

Are you following these guide lines before attempting to upgrade?

efore attempting to use ISSU to upgrade to any software image version, follow these guidelines:

• Scheduling Schedule the upgrade when your network is stable and steady. Ensure that everyone who has access to the device or the network is not configuring the device or the network during this time. You cannot configure a device during an upgrade.
• Space Verify that sufficient space is available in the location where you are copying the images. This location includes the active and standby supervisor module bootflash: (internal to the device). Internal bootflash: has approximately 250 MB of free space available.
• Hardware Avoid power interruption during any install procedure, which can corrupt the software image.
• Connectivity to remote servers
  • Configure the IPv4 address or IPv6 address for the 10/100/1000 BASE-T Ethernet port connection (interface mgmt0).
  • Ensure that the device has a route to the remote server. The device and the remote server must be in the same subnetwork if you do not have a router to route traffic between subnets.
• Software images
  • Ensure that the specified system and kickstart images are compatible with each other.
  • If the kickstart image is not specified, the device uses the current running kickstart image.
  • If you specify a different system image, ensure that it is compatible with the running kickstart image.
  • Retrieve the images in one of two ways:

    Locally

    Images are locally available on the switch.

    Remotely

    Images are in a remote location and you specify the destination using the remote server parameters and the filename to be used locally.

  • Before an upgrade from Cisco NX-OS Release 6.1(x) to Release 6.2, apply either "limit-resource module-type f1" or "limit-resource module-type f2" to the storage VDC, and check that the following storage VDC configurations are removed:
    • Shared F2(F1) interfaces with a storage VDC that supports only F1(F2)
    • F1 and F2 interfaces in the same storage VDC
  • The default Control Plane Policing (CoPP) policy does not change when you upgrade the Cisco NX-OS software.
  • CoPP MAC policies are supported beginning with Cisco NX-OS Release 5.1, and default policies are installed upon execution of the initial setup script. However, if you use ISSU to upgrade to Cisco NX-OS Release 6.0(1), the default CoPP policies for the following features must be manually configured: FabricPath, OTV, L2PT, LLDP, DHCP, and DOT1X. For more information on the default CoPP policies, see the Cisco Nexus 7000 Series NX-OS Security Configuration Guide.
  • When you upgrade to Cisco NX-OS Release 6.0(1), the policy attached to the control plane is treated as a user-configured policy. Check the CoPP profile using the show copp profile command and make any required changes.
  • The upgrade to Cisco NX-OS Release 6.0(1) in an OTV network is disruptive. You must upgrade all edge devices in the site and configure the site identifier on all edge devices in the site before traffic is restored. You can prepare OTV for ISSU to Cisco NX-OS Release 6.0(1) in a dual-homed site to minimize this disruption. See the Cisco Nexus 7000 Series NX-OS OTV Configuration Guide for information on how to prepare OTV for ISSU to Cisco NX-OS Release 6.0(1) in a dual-homed site. An edge device with an older Cisco NX-OS release in the same site can cause traffic loops. You should upgrade all edge devices in the site during the same upgrade window. You do not need to upgrade edge devices in other sites as OTV interoperates between sites with different Cisco NX-OS versions.
  • The upgrade from Cisco NX-OS Release 5.2(1) or from Release 6.0(1) to Release 6.1(1) in an OTV network is non-disruptive.
  • VPC peers can only operate dissimilar versions of the Cisco NX-OS software during the upgrade or downgrade process. Operating VPC peers with dissimilar versions, after the upgrade or downgrade process is complete, is not supported.
  • Starting with Cisco NX-OS Release 6.1(1), Supervisor 2 is supported. Therefore, there is no upgrade of Supervisor 2 from a previous Cisco NX-OS release.

Link:

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/sw/6_x/nx-os/upgrade/guide/b_Cisco_Nexus_7000_Series_NX-OS_Software_Upgrade_and_Downgrade_Guide_Release_6-x.html#con_317522

HTH

[sujohnm]

Hi Reza, 

I've followed all the instructions properly and even had other collegues review my prep work. Any ideas will help at this stage

thanks, Sujohn

[Bilal Nawaz]

Just out of interest are you using F Cards with M cards? Reason I ask is because i want to give you a warning if that is the case. Performing ISSU to 6.2.8a will cause the proxy routing used by F card via the M cards to fail or mis-function on certain flows (Completely Random) dependant on the FIB entries being mis-programmed in to the wrong SPL banks. See Bug ID CSCup96876

Better off doing a traditional upgrade of reloading to this version if you can.

How many SUPs do you have in the N7K? The below table shows that you can go from 5.2.7 to 6.2.8a in ISSU but cannot do ISSD in case.

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/sw/6_x/nx-os/release/notes/62_nx-os_release_note.html#84013

hth

Bilal

[perry.barham]

I know this is an old thread but I had a similar problem with error message "error code
SYSMGR_EXITCODE_NEED_COPYRS (66)" when attempting to upgrade from 6.1x to 6.2x and it was related to memory requirements.  I had 4GB and needed 8GB of RAM to support 6.2.   People who run into should check your RAM requirements on the release notes.