Nexus 3048 fails to upgrade

[--Jay4Jay--]

hi All, 

A Nexus 3048 is running NXOS: version 7.0(3)I3(1).
Looking to upgrade to version 7.0.3.I7.10, and eventually to version 9.3.
After seemingly succesful upgrade , the Nexus keeps running the old version.

Details:

Following this guide, which says to Upgrade from NX-OS 7.x to NX-OS 7.x. , before upgrading to 9.3
https://www.cisco.com/c/en/us/support/docs/switches/nexus-3000-series-switches/216037-nexus-3000-and-3100-nx-os-software-upgra.html#upgrade-7x-to-7x

Because of the limited space on the flash drive I am using a compact image file n3000-compact.7.0.3.I7.10.bin.
I copied the bin file to the bootflash, then  run the install with this command:

install all nxos bootflash:n3000-compact.7.0.3.I7.10.bin
The installation succeeds. However, after the reboot of the Nexus the upgrade seems to have failed as the original version is still showing wen sending the command " show version".

switch# show version

Software
BIOS: version 4.5.0
NXOS: version 7.0(3)I3(1)
BIOS compile time: 11/09/2017
NXOS image file is:
NXOS compile time: 2/8/2016 20:00:00

Hardware
cisco Nexus 3048 Chassis
Intel(R) Celeron(R) CPU P4505 @ 1.87GHz with 3903820 kB of memory.
Processor Board ID FOC23074Y53

Device name: switch
bootflash: 30170112 kB
usb1: 30302208 kB (expansion flash)

Kernel uptime is 0 day(s), 4 hour(s), 24 minute(s), 37 second(s)

Last reset at 142875 usecs after Sat Jan 30 13:53:28 2010

Reason: Reset Requested by CLI command reload
System version: 7.0(3)I3(1)
Service:

The "show boot" command is showing the target version, which is a remnant of the7.0.3.I7.10  installation procedure:

switch# show boot
Current Boot Variables:

sup-1
NXOS variable = bootflash:/n3000-compact.7.0.3.I7.10.bin
No module boot variable set

Boot Variables on next reload:

sup-1
NXOS variable = bootflash:/n3000-compact.7.0.3.I7.10.bin
No module boot variable set
switch#

[balaji.bandi]

Compact installation 

# install all nxos bootflash:nxos.7.0.3.I7.9.bin compact

Can you post :

# dir | include bin

[--Jay4Jay--]

Hi Balaji, 

Any idea why our installation "succeeds" but does not really upgrade the Nexus?
Is that solved with this command ?

install all nxos bootflash:nxos.7.0.3.I7.9.bin compact

Here's the bin files on the flash.

switch# dir | include bin
477067039 Jan 30 07:33:45 2010 n3000-compact.7.0.3.I7.10.bin
621192864 Jan 30 08:22:40 2010 n3000-compact.9.3.9.bin
1961096192 Jan 30 15:57:28 2010 nxos.9.3.9.bin
switch#

[balaji.bandi]

621192864 Jan 30 08:22:40 2010 n3000-compact.9.3.9.bin
1961096192 Jan 30 15:57:28 2010 nxos.9.3.9.bin

why do you have these images?

i do not see any images of old, can you boot post complete output ? (looks we miss something here ?)

[--Jay4Jay--]

The problem is that after reload, a kickstart image is loaded from a previous upgrade.

"Press ctrl L to go to loader prompt in 2 secs

Booting kickstart image: bootflash:/nxos.7.0.3.I3.1.bin
CardIndex = 11056"

But as I shared before , the current boot commands are ...

switch# show boot
Current Boot Variables:

sup-1
NXOS variable = bootflash:/n3000-compact.7.0.3.I7.10.bin
No module boot variable set

Boot Variables on next reload:

sup-1
NXOS variable = bootflash:/n3000-compact.7.0.3.I7.10.bin
No module boot variable set
switch#

Full boot log:

Booting kickstart image: bootflash:/nxos.7.0.3.I3.1.bin
CardIndex = 11056
Image valid
graphics_set_kernel_params enter:
INIT: version 2.88 booting
Skipping ata_piix for n3k.
Unsquashing rootfs ...

Loading IGB driver ...
Installing SSE module ... done
Creating the sse device node ... done
Loading I2C driver ...
Installing CCTRL driver for card_type 30 ...
CCTRL driver for card_index 11056 ...
7.30: Interrupt throttling disabled. No cctrl irq detected.
Not Micron SSD...

Checking all filesystems.Current boot disk sda3..
...Skipping LOGFLASH check for N3k...
.Skipping plog check for N3k...

Skipping installing default sprom values...
Configuring network ...
Installing LC netdev ...
Installing veobc ...
Installing OBFL driver ...
invalid group file entry
delete line 'aaa-db-operator:508:'? No
grpck: no changes
..done Sat Jan 30 18:44:28 UTC 2010
tune2fs 1.42.1 (17-Feb-2012)
Setting reserved blocks percentage to 0% (0 blocks)
Starting portmap daemon...
creating NFS state directory: done
starting 8 nfsd kernel threads: done
starting mountd: done
starting statd: done
Saving image for img-sync ...
Loading system software
Installing local RPMS
Patch Repository Setup completed successfully
Uncompressing system image: Sat Jan 30 18:45:40 UTC 2010
blogger: nothing to do.

..done Sat Jan 30 18:45:40 UTC 2010
Creating /dev/mcelog
Starting mcelog daemon
Removing dme lib
Moving N3K specific syslog config file
INIT: Entering runlevel: 3
Running S93thirdparty-script...

Populating conf files for hybrid sysmgr ...
Starting hybrid sysmgr ...
2010 Jan 30 18:46:41 switch %$ VDC-1 %$ Jan 30 18:46:41 %KERN-2-SYSTEM_MSG: [ 9.632037] Initializing NVRAM Block 6 - kernel
2010 Jan 30 18:46:41 switch %$ VDC-1 %$ Jan 30 18:46:41 %KERN-2-SYSTEM_MSG: [ 10.440801] hwport mode=6type 2. mod_no 0, inst_no 0 - kernel
2010 Jan 30 18:46:47 switch %$ VDC-1 %$ %PLATFORM-2-PS_FAIL: Power supply 1 failed or shut down (Serial number DTN1844P1PZ)
2010 Jan 30 18:46:47 switch %$ VDC-1 %$ %PLATFORM-2-PS_OK: Power supply 2 ok (Serial number DTN1844P35X)
2010 Jan 30 18:46:47 switch %$ VDC-1 %$ %PLATFORM-2-PS_FANOK: Fan in Power supply 2 ok
2010 Jan 30 18:46:47 switch %$ VDC-1 %$ %PLATFORM-2-PS_ABSENT: Power supply 1 is absent/shutdown, ps-redundancy might be affected
2010 Jan 30 18:46:47 switch %$ VDC-1 %$ %PLATFORM-2-PS_RED_MODE_CHG: Power supply operational redundancy mode changed to non-redundant
2010 Jan 30 18:46:47 switch %$ VDC-1 %$ %PLATFORM-2-FANMOD_FAN_OK: Fan module 1 (Fan1(sys_fan1) fan) ok
2010 Jan 30 18:46:48 switch %$ VDC-1 %$ %USER-2-SYSTEM_MSG: <<%USBHSD-2-USB_SWAP>> USB insertion or removal detected - usbhsd
2010 Jan 30 18:46:49 switch %$ VDC-1 %$ %DAEMON-2-SYSTEM_MSG: <<%ASCII-CFG-2-CONF_CONTROL>> Binary restore - ascii-cfg[11148]
2010 Jan 30 18:46:52 switch %$ VDC-1 %$ netstack: Registration with cli server complete
2010 Jan 30 18:47:06 switch %$ VDC-1 %$ %USER-2-SYSTEM_MSG: <<%USBHSD-2-DISK_MOUNT_FAIL>> disk mount unsuccessful on USB1: - usbhsd
2010 Jan 30 18:47:18 switch %$ VDC-1 %$ %USER-2-SYSTEM_MSG: ssnmgr_app_init called on ssnmgr up - aclmgr
2010 Jan 30 18:47:23 switch %$ VDC-1 %$ %USER-0-SYSTEM_MSG: end of default policer - copp
2010 Jan 30 18:47:33 switch %$ VDC-1 %$ %USER-2-SYSTEM_MSG: IPV6 Netlink thread init successful - icmpv6
2010 Jan 30 18:47:33 switch %$ VDC-1 %$ %VDC_MGR-2-VDC_ONLINE: vdc 1 has come online

User Access Verification
switch login: 2010 Jan 30 18:48:01 switch %$ VDC-1 %$ %ASCII-CFG-2-CONF_CONTROL: System ready

 

[balaji.bandi]

Booting kickstart image: bootflash:/nxos.7.0.3.I3.1.bin

 as per the output we do not see the image  at all

can you post show modules 

if you want to compact you need to as below :

# install all nxos bootflash:nxos.7.0.3.I7.9.bin compact

[--Jay4Jay--]

switch# show module
Mod Ports             Module-Type                       Model          Status
--- ----- ------------------------------------- --------------------- ---------
1    52   48x1GE + 4x10G Supervisor             N3K-C3048TP-1GE       active *

Mod  Sw                Hw     Slot
---  ----------------  ------ ----
1    7.0(3)I3(1)       1.6    NA


Mod  MAC-Address(es)                         Serial-Num
---  --------------------------------------  ----------
1    6c-8b-d3-47-4a-c8 to 6c-8b-d3-47-4a-fb  

Mod  Online Diag Status
---  ------------------
1    Pass

* this terminal session
switch#

[--Jay4Jay--]

The question is where this command is configured , stored and ran from ...

Booting kickstart image: bootflash:/nxos.7.0.3.I3.1.bin

 While I have another image selected as can be viewed in the show boot command:

switch# show boot
Current Boot Variables:

sup-1
NXOS variable = bootflash:/n3000-compact.7.0.3.I7.10.bin
No module boot variable set

Boot Variables on next reload:

sup-1
NXOS variable = bootflash:/n3000-compact.7.0.3.I7.10.bin
No module boot variable set
switch#

[--Jay4Jay--]

Attempted to run command
"install all nxos bootflash:nxos.7.0.3.I7.9.bin compact"

It fails "Compact of bootflash:/nxos.7.0.3.I7.9.bin failed. Original image might have got deleted. You may need to copy it again".
Ran MD5 checksum , which is correct, same as published on Cisco download site.

switch# install all nxos bootflash:nxos.7.0.3.I7.9.bin compact
Installer will perform compatibility check first. Please wait.
Compacting bootflash:/nxos.7.0.3.I7.9.bin
.......
Compact of bootflash:/nxos.7.0.3.I7.9.bin failed. Original image might have got deleted. You may need to copy it again
switch# show file ^C
switch# ^C
switch# ^C
switch# dir
       4096    Jan 30 13:21:36 2010  .rpmstore/
       4096    Jan 29 15:02:09 2010  .swtam/
       1812    Jan 29 15:05:49 2010  20100129_150340_poap_29289_init.log
       7690    Jan 30 13:23:58 2010  20100130_132219_poap_12846_init.log
       2870    Jan 30 13:38:14 2010  20100130_133743_poap_12628_init.log
       2813    Jan 30 13:44:49 2010  bios_daemon.dbg
          0    Jan 29 15:02:01 2010  bootflash_sync_list
       4096    Jan 30 06:47:57 2010  logflash/
       4096    Jan 30 13:40:01 2010  lost+found/
  982694912    Jan 30 21:31:24 2010  nxos.7.0.3.I7.9.bin
 1961096192    Jan 30 15:57:28 2010  nxos.9.3.9.bin
       1296    Jan 30 10:47:09 2010  patch_control.log
        232    Jan 30 10:47:09 2010  patch_debug.log
          0    Jan 29 15:08:04 2010  platform-sdk.cmd
       4096    Jan 29 15:03:40 2010  scripts/
       1024    Jan 30 18:44:25 2010  sprom_2_0_1
       1024    Jan 30 18:44:25 2010  sprom_3_0_0
       4096    Jan 29 15:02:14 2010  virtual-instance/

Usage for bootflash://
 3337474048 bytes used
27312472064 bytes free
30649946112 bytes total
switch# show file nxos.7.0.3.I7.9.bin md5sum
d31d5b556cc4d92f2ff2d83b5df7b943