Problem with VSS IOS Upgrade

joe.ho · ‎10-13-2010

I am upgrading the 6509 VSS from s72033-ipbasek9-mz.122-33.SXI3.bin to s72033-ipbasek9-mz.122-33.SXI4a.bin. Everything went fine until the last step.

CWCDCCORE1#issu commitversion

Building configuration…

[OK]

%issu commitversiion executed successfully

From the doc it should be just the standby reload but both chassis reload instead. The "primary" came back fine but the "secondary" get stuck at this spot.

System Bootstrap, Version 12.2(17r)SX7, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 2009 by cisco Systems, Inc.
Cat6k-Sup720/RP platform with 1048576 Kbytes of main memory

Download Start

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Download Completed! Booting the image.
Self decompressing the image : #################################################
################################################################################
######### [OK]

Restricted Rights Legend

Use, duplication, or disclosure by the Government is
subject to restrictions as set forth in subparagraph
(c) of the Commercial Computer Software - Restricted
Rights clause at FAR sec. 52.227-19 and subparagraph
(c) (1) (ii) of the Rights in Technical Data and Computer
Software clause at DFARS sec. 252.227-7013.

           cisco Systems, Inc.
           170 West Tasman Drive
           San Jose, California 95134-1706

Cisco IOS Software, s72033_rp Software (s72033_rp-IPBASEK9-M), Version 12.2(33)S
XI4a, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2010 by Cisco Systems, Inc.
Compiled Fri 16-Jul-10 19:51 by prod_rel_team
Image text-base: 0x40101328, data-base: 0x43698270

From the primary I can issue reload to the secondary but still get stuck at the same spot. Looks like it is RPR install of SSO.

CWCDCCORE1#sh red
Redundant System Information :
------------------------------
       Available system uptime = 36 minutes
Switchovers system experienced = 0
              Standby failures = 2
        Last switchover reason = none

                 Hardware Mode = Duplex
    Configured Redundancy Mode = sso
     Operating Redundancy Mode = rpr    Reason: Software mismat
              Maintenance Mode = Disabled
                Communications = Up

Current Processor Information :
-------------------------------
               Active Location = slot 1/5
        Current Software state = ACTIVE
       Uptime in current state = 35 minutes
                 Image Version = Cisco IOS Software, s72033_rp Software (s72033_
rp-IPBASEK9-M), Version 12.2(33)SXI4a, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2010 by Cisco Systems, Inc.
Compiled Fri 16-Jul-10 19:51 by prod_rel_team
                          BOOT = ,12;sup-bootdisk:s72033-ipbasek9-mz.122-33.SXI4
a.bin,12;sup-bootdisk:s72033-ipbasek9-mz.122-33.SXI3.bin,1;
        Configuration register = 0x2102

Peer Processor Information :
----------------------------
              Standby Location = slot 2/5
        Current Software state = STANDBY COLD
       Uptime in current state = 15 minutes
                 Image Version = Cisco IOS Software, s72033_rp Software (s72033_
rp-IPBASEK9-M), Version 12.2(33)SXI4a, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2010 by Cisco Systems, Inc.
Compiled Fri 16-Jul-10 19:51 by prod_rel_team
                          BOOT = sup-bootdisk:s72033-ipbasek9-mz.122-33.SXI4a.bi
n,12;sup-bootdisk:s72033-ipbasek9-mz.122-33.SXI3.bin,1;
        Configuration register = 0x2102

What can I do at this point?

joe.ho · ‎10-13-2010

From the log I am getting

%ISSU-SW1_SP-3-PEER_IMAGE_INCOMPATIBLE: Peer image (s72
033_sp-IPBASEK9-M), version (12.2.(33)SXI4a) on peer uid (995442725) is incompat
ible
Oct 13 17:59:16.103 MDT: %PFREDUN-SW1_SP-4-INCOMPATIBLE: Defaulting to RPR mode
(Runtime incompatible)

Error Message Decoder

1. %ISSU-3-PEER_IMAGE_INCOMPATIBLE: Peer image (%s), version (%s) on peer uid (%d) is incompatible
The peer client is incompatible.

Recommended Action: Synchronize the configurations and reload the standby supervisor.

Related documents- No specific documents apply to this error message.

Just by reloading it doesn't help. Can someone tell me how to sync the config on the standby sup?

Thanks!!!

Reza Sharifi · ‎10-13-2010

You cannot sync the config if they are not in SSO or in the case a VSS pair. What I recommend is to load a fresh copy of the exact image into the stand-by switch and reload it. If the first solution does not work, the second solution is after loading the new image on the stand-by, change the boot variable and reboot it and see if it comes up as VSS pair.

unfortunately eFSU does not work all the time the way it supposed to

HTH

Reza

joe.ho · ‎10-13-2010

The show command do show the boot variable is pointing to the correct file and the file do exist in the slavesup-bootdisk. If I want to set the boot variable again, do you know the command to set the boot variable on the standby sup?

CWCDCCORE1#sh boot
BOOT variable = ,12;sup-bootdisk:s72033-ipbasek9-mz.122-33.SXI4a.bin,12;sup-boot
disk:s72033-ipbasek9-mz.122-33.SXI3.bin,1;
CONFIG_FILE variable does not exist
BOOTLDR variable does not exist
Configuration register is 0x2102

Standby is up
Standby has 1048576K/65536K bytes of memory.

Standby BOOT variable = sup-bootdisk:s72033-ipbasek9-mz.122-33.SXI4a.bin,12;sup-
bootdisk:s72033-ipbasek9-mz.122-33.SXI3.bin,1;
Standby CONFIG_FILE variable does not exist
Standby BOOTLDR variable does not exist
Standby Configuration register is 0x2102

CWCDCCORE1#sh slavesup-bootdisk
-#- --length-- -----date/time------ path
1     56696548 Jul 17 2010 08:23:42 +00:00 s72033-ipbasek9-mz.122-33.SXI3.bin
2     33554432 Jul 17 2010 08:21:14 +00:00 sea_console.dat
3     33554432 Jul 17 2010 13:06:26 +00:00 sea_log.dat
4     68792548 Oct 13 2010 03:54:04 +00:00 s72033-ipbasek9-mz.122-33.SXI4a.bin

831946752 bytes available (192610304 bytes used)

CWCDCCORE1#sh sup-bootdisk
-#- --length-- -----date/time------ path
1     33554432 Jul 17 2010 01:36:58 -06:00 sea_console.dat
2     56696548 Jul 17 2010 01:39:36 -06:00 s72033-ipbasek9-mz.122-33.SXI3.bin
3     33554432 Jul 17 2010 05:55:02 -06:00 sea_log.dat
4       466004 Sep 27 2010 23:39:10 -06:00 crashinfo_20100927-233910-MDT
5     68792548 Oct 12 2010 21:48:28 -06:00 s72033-ipbasek9-mz.122-33.SXI4a.bin

831471616 bytes available (193085440 bytes used)

Reza Sharifi · ‎10-13-2010

try this:

boot system flash slavesup-bootdisk:s72033-ipbasek9-mz.122-33.SXI4a.bin

and reboot the stand-by

since you have been getting the IOS mismatch error, make sure you load good copy of the image into flash

joe.ho · ‎10-13-2010

I tried the command but doesn't work.

I ended up open a tac case. The tac went into the standby to clear the boot variable and that resolve the issue.

rommon 1 > set
PS1=rommon ! >
?=0
LOG_PREFIX_VERSION=1
RANDOM_NUM=1517966774
SWITCH_NUMBER=2
NT_K=0:0:0:0
BOOT=,12;bootdisk:s72033-ipbasek9-mz.122-33.SXI4a.bin,12;bootdisk:s72033-ipbasek
9-mz.122-33.SXI3.bin,1;slavesup-bootdisk:s72033-ipbasek9-mz.122-33.SXI4a.bin,1;
RET_2_RTS=04:15:41 UTC Thu Oct 14 2010
RET_2_RCALTS=1287029740
BSI=0
rommon 2 > BOOT=
rommon 3 > SET

monitor: command "SET" not found
rommon 4 > set
PS1=rommon ! >
LOG_PREFIX_VERSION=1
RANDOM_NUM=1517966774
SWITCH_NUMBER=2
NT_K=0:0:0:0
RET_2_RTS=04:15:41 UTC Thu Oct 14 2010
RET_2_RCALTS=1287029740
BSI=0
?=0
BOOT=
rommon 5 > dir bootflash:

Initializing ATA monitor library...
Directory of bootflash:

2      56696548 -rw-     s72033-ipbasek9-mz.122-33.SXI3.bin
363    33554432 -rw-     sea_console.dat
3797    33554432 -rw-     sea_log.dat
7559    68792548 -rw-     s72033-ipbasek9-mz.122-33.SXI4a.bin
rommon 6 > boot bootflash:s72033-ipbasek9-mz.122-33.SXI4a.bin
Loading image, please wait ...

Initializing ATA monitor library...

Self extracting the image... [OK]
Self decompressing the image : #################################################
################################################################################
###################################################################### [OK]