Re: 9300L upgrade path from 17.06.05 to 17.12.04

StarLog · ‎11-11-2024

We tried to do a direct upgrade from 17.06.05 to 17.12.04 and received the errors below.
The switch is still on the older version.

This is the command I used:

install add file usbflash0:cat9k_lite_iosxe.17.12.04.SPA.bin activate commit

PWSWCO01#dir flash:
Directory of flash:/

131104 -rw- 482681593 Nov 11 2024 21:57:03 +00:00 cat3k_caa-universalk9.16.12.12.SPA.bin
131089 -rw- 5242880 Nov 11 2024 06:28:58 +00:00 ssd
131075 -rw- 2097152 Nov 1 2024 09:00:28 +00:00 nvram_config_bkup
131074 -rw- 2097152 Nov 1 2024 09:00:28 +00:00 nvram_config
131101 -rw- 1156 Dec 5 2023 22:36:25 +00:00 vlan.dat
196609 drwx 4096 Oct 10 2023 01:30:03 +00:00 .installer
131103 -rw- 976 Oct 10 2023 01:28:37 +00:00 stby-vlan.dat
131100 -rw- 0 Oct 10 2023 01:26:29 +00:00 dope_hist
131097 -rw- 16953 Oct 10 2023 01:26:29 +00:00 rdope_out.txt
131098 -rw- 89 Oct 10 2023 01:26:28 +00:00 rdope.log
245761 drwx 4096 Oct 10 2023 01:26:12 +00:00 .prst_sync
131092 -rwx 2049 Oct 10 2023 01:25:33 +00:00 svl_ipc.tcl
131091 -rw- 134899 Oct 10 2023 01:25:33 +00:00 memleak.tcl
131090 -rw- 395 Oct 10 2023 01:25:26 +00:00 boothelper.log
286753 drwx 4096 Oct 10 2023 01:25:25 +00:00 dc_profile_dir
131088 -rw- 444 Oct 10 2023 01:25:12 +00:00 bootloader_evt_handle.log
327681 drwx 4096 Sep 14 2023 14:37:53 +00:00 pnp-info
196611 drwx 4096 Jul 24 2023 14:14:07 +00:00 .dbpersist
196619 drwx 4096 Jul 24 2023 14:07:21 +00:00 Apps
131102 -rw- 1440 Jul 24 2023 14:05:09 +00:00 2023_07_24T14_05_08.580029.xml
196612 drwx 4096 Jul 24 2023 14:04:48 +00:00 pnp-tech
212993 drwx 4096 Jul 24 2023 13:57:33 +00:00 core
352257 drwx 4096 Jul 24 2023 13:57:17 +00:00 guest-share
335873 drwx 4096 Jul 24 2023 13:57:13 +00:00 onep
311304 drwx 4096 Jul 24 2023 13:48:12 +00:00 .rommon_sync
131096 -rw- 1261 Jul 24 2023 13:48:10 +00:00 Baltimore.ca
131095 -rw- 1606 Jul 24 2023 13:48:10 +00:00 StarfieldServicesRootCA.ca
131094 -rw- 1338 Jul 24 2023 13:48:10 +00:00 DigiCertGlobalRootCA.ca
311297 drwx 4096 Jul 24 2023 13:48:09 +00:00 Tbot
294913 drwx 4096 Jul 24 2023 13:48:06 +00:00 fp_cc_crash
286755 drwx 4096 Jul 24 2023 13:48:06 +00:00 sys_report
286721 drwx 4096 Jul 24 2023 13:48:05 +00:00 tech_support
270337 drwx 4096 Jul 24 2023 13:48:02 +00:00 ss_disc
262145 drwx 4096 Jul 24 2023 13:47:56 +00:00 .rollback_timer
131087 -rw- 48649463 Jul 24 2023 13:45:26 +00:00 cat9k-rpboot.17.06.05.SPA.pkg
131086 -rw- 6514 Jul 24 2023 13:45:26 +00:00 packages.conf
131085 -rw- 9220 Jul 24 2023 13:45:26 +00:00 cat9k-wlc.17.06.05.SPA.pkg
131084 -rw- 17523716 Jul 24 2023 13:45:26 +00:00 cat9k-webui.17.06.05.SPA.pkg
131083 -rw- 32056328 Jul 24 2023 13:45:26 +00:00 cat9k-srdriver.17.06.05.SPA.pkg
131082 -rw- 61326340 Jul 24 2023 13:45:26 +00:00 cat9k-sipspa.17.06.05.SPA.pkg
131081 -rw- 37864456 Jul 24 2023 13:45:26 +00:00 cat9k-sipbase.17.06.05.SPA.pkg
131080 -rw- 698438660 Jul 24 2023 13:45:26 +00:00 cat9k-rpbase.17.06.05.SPA.pkg
131079 -rw- 9220 Jul 24 2023 13:45:25 +00:00 cat9k-lni.17.06.05.SPA.pkg
131078 -rw- 2098184 Jul 24 2023 13:45:25 +00:00 cat9k-guestshell.17.06.05.SPA.pkg
131077 -rw- 110638088 Jul 24 2023 13:45:25 +00:00 cat9k-espbase.17.06.05.SPA.pkg
131076 -rw- 19760140 Jul 24 2023 13:45:25 +00:00 cat9k-cc_srdriver.17.06.05.SPA.pkg

11353194496 bytes total (9035943936 bytes free)
PWSWCO01#install add file usbflash0:cat9k_lite_iosxe.17.12.04.SPA.bin activate commit
install_add_activate_commit: START Mon Nov 11 22:03:24 UTC 2024
Copying usbflash0:cat9k_lite_iosxe.17.12.04.SPA.bin to flash:cat9k_lite_iosxe.17.12.04.SPA.bin
install_add_activate_commit: Adding PACKAGE
install_add_activate_commit: Checking whether new add is allowed ....

--- Starting initial file syncing ---
[1]: Copying flash:cat9k_lite_iosxe.17.12.04.SPA.bin from switch 1 to switch 2
[2]: Finished copying to switch 2
Info: Finished copying flash:cat9k_lite_iosxe.17.12.04.SPA.bin to the selected switch(es)
Finished initial file syncing

--- Starting Add ---
Performing Add on all members
[1] Add package(s) on switch 1
[1] Finished Add on switch 1
[2] Add package(s) on switch 2
[2] Finished Add on switch 2
Checking status of Add on [1 2]
Add: Passed on [1 2]
Finished Add

Image added. Version: 17.12.04.0.4826
install_add_activate_commit: Activating PACKAGE
awk: fatal: cannot open file `/tmp/issu_prepare/rp/0/0/srdriver/mount/.pkginfo' for reading (No such file or directory)
awk: fatal: cannot open file `/tmp/issu_prepare/rp/0/0/rp_webui/mount/.pkginfo' for reading (No such file or directory)
awk: fatal: cannot open file `/tmp/issu_prepare/cc/0/0/cc_srdriver/mount/.pkginfo' for reading (No such file or directory)
awk: fatal: cannot open file `/tmp/issu_prepare/rp/0/0/rp_base/mount/.pkginfo' for reading (No such file or directory)
awk: fatal: cannot open file `/tmp/issu_prepare/rp/0/0/rp_security/mount/.pkginfo' for reading (No such file or directory)
awk: fatal: cannot open file `/tmp/issu_prepare/rp/0/0/rp_daemons/mount/.pkginfo' for reading (No such file or directory)
awk: fatal: cannot open file `/tmp/issu_prepare/fp/0/0/fp/mount/.pkginfo' for reading (No such file or directory)awk:
fatal: cannot open file `/tmp/issu_prepare/cc/0/0/cc/mount/.pkginfo' for reading (No such file or directory)
awk: fatal: cannot open file `/tmp/issu_prepare/rp/0/0/rp_iosd/mount/.pkginfo' for reading (No such file or directory)

Flavio Miranda · ‎11-11-2024

@StarLog

Can I see the output of "show version" command?

balaji.bandi · ‎11-11-2024

I see 2 packages files here :

131086 -rw- 6514 Jul 24 2023 13:45:26 +00:00 packages.conf
131085 -rw- 9220 Jul 24 2023 13:45:26 +00:00 cat9k-wlc.17.06.05.SPA.pkg

to understand what version running correctly, show version and show boot (output help here)

run the command - #install remove inactive (this will remove any old packages that clean the space)

then re-run the install command that should work. it will upgrade ROMMON to 17.11 so give enough time to upgrade, make sure you take the running config out of the box.

also read the release notes any caveats effect your environment

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

StarLog · ‎11-12-2024

Hi Flavio & BB,

PWSWCO01#sh ver
Cisco IOS XE Software, Version 17.06.05
Cisco IOS Software [Bengaluru], Catalyst L3 Switch Software (CAT9K_IOSXE), Version 17.6.5, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2023 by Cisco Systems, Inc.
Compiled Wed 25-Jan-23 16:15 by mcpre

Cisco IOS-XE software, Copyright (c) 2005-2023 by cisco Systems, Inc.
All rights reserved. Certain components of Cisco IOS-XE software are
licensed under the GNU General Public License ("GPL") Version 2.0. The
software code licensed under GPL Version 2.0 is free software that comes
with ABSOLUTELY NO WARRANTY. You can redistribute and/or modify such
GPL code under the terms of GPL Version 2.0. For more details, see the
documentation or "License Notice" file accompanying the IOS-XE software,
or the applicable URL provided on the flyer accompanying the IOS-XE
software.

ROM: IOS-XE ROMMON
BOOTLDR: System Bootstrap, Version 17.6.6r, RELEASE SOFTWARE (P)

PWSWCO01 uptime is 1 year, 4 weeks, 6 days, 10 hours, 6 minutes
Uptime for this control processor is 1 year, 4 weeks, 6 days, 10 hours, 7 minutes
System returned to ROM by Reload Command at 01:23:38 UTC Tue Oct 10 2023
System restarted at 01:26:08 UTC Tue Oct 10 2023
System image file is "flash:packages.conf"
Last reload reason: Reload Command

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

Technology Package License Information:

------------------------------------------------------------------------------
Technology-package Technology-package
Current Type Next reboot
------------------------------------------------------------------------------
network-advantage Smart License network-advantage
dna-advantage Subscription Smart License dna-advantage
AIR License Level: AIR DNA Advantage
Next reload AIR license Level: AIR DNA Advantage

Smart Licensing Status: Registration Not Applicable/Not Applicable

cisco C9300-24P (X86) processor with 1317020K/6147K bytes of memory.
Processor board ID FJC272910X9
3 Virtual Ethernet interfaces
56 Gigabit Ethernet interfaces
16 Ten Gigabit Ethernet interfaces
4 TwentyFive Gigabit Ethernet interfaces
4 Forty Gigabit Ethernet interfaces
2048K bytes of non-volatile configuration memory.
8388608K bytes of physical memory.
1638400K bytes of Crash Files at crashinfo:.
1638400K bytes of Crash Files at crashinfo-2:.
11264000K bytes of Flash at flash:.
11264000K bytes of Flash at flash-2:.

Base Ethernet MAC Address : 48:1b:a4:c5:7c:80
Motherboard Assembly Number : 73-18271-04
Motherboard Serial Number : FJZ2728104J
Model Revision Number : J0
Motherboard Revision Number : A0
Model Number : C9300-24P
System Serial Number : FJC272910X9
CLEI Code Number : INM2T10ARC

Switch Ports Model SW Version SW Image Mode
------ ----- ----- ---------- ---------- ----
* 1 41 C9300-24P 17.06.05 CAT9K_IOSXE INSTALL
2 41 C9300-24P 17.06.05 CAT9K_IOSXE INSTALL

Switch 02
---------
Switch uptime : 1 year, 4 weeks, 6 days, 10 hours, 7 minutes

Base Ethernet MAC Address : 68:e5:9e:31:9e:00
Motherboard Assembly Number : 73-18271-04
Motherboard Serial Number : FJZ27281456
Model Revision Number : J0
Motherboard Revision Number : A0
Model Number : C9300-24P
System Serial Number : FJC272827QH
Last reload reason : Reload Command
CLEI Code Number : INM2T10ARC

PWSWCO01#sh boot
---------------------------
Switch 1
---------------------------
Current Boot Variables:
BOOT variable does not exist

Boot Variables on next reload:
BOOT variable = flash:packages.conf
Manual Boot = no
Enable Break = no
Boot Mode = DEVICE
iPXE Timeout = 0
PWSWCO01#

Flavio Miranda · ‎11-12-2024

@StarLog

On the error, there is an information that call the attention

awk: fatal: cannot open file `/tmp/issu_prepare

You are not using ISSU apparently and we can see something related.

I would break your stack and try to upgrade individually, just in case.

StarLog · ‎11-12-2024

In that case do the following commands look ok?

Switch# install remove inactive
Do you want to remove the above files? [y/n]y
Switch# copy tftp://10.8.0.6/image/cat9k_lite_iosxe.17.12.04.SPAbin flash:
Switch# dir flash:*.bin
Switch(config)# no boot manual
Switch(config)# exit
Switch# write memory

Switch# install add file flash: cat9k_lite_iosxe.17.12.04.SPA.bin activate commit
Switch# dir flash:*.pkg
Switch# dir flash:*.conf
Switch# reload
Switch# show version

Flavio Miranda · ‎11-12-2024

Yes, seems ok to me.

StarLog · ‎11-12-2024

OK, give me a couple days as we have to schedule the upgrade time with the departments.

Stay Tuned.

StarLog · ‎11-20-2024

Hi Flavio and BB,

I had the opportunity to try again to upgrade this switch.
The only thing I did not have time to do was the following recommendation from the s/w.

FAILED: install_commit : Cannot commit; please use 'install activate' to activate added packages OR 'install remove inactive' to remove added subpackages.

Ran out of time. Can you suggest what the next command I should try, see below from the upgrade:

PWSWCO01#dir flash:
Directory of flash:/

131075 -rw- 2097152 Nov 19 2024 21:15:46 +00:00 nvram_config_bkup
131074 -rw- 2097152 Nov 19 2024 21:15:46 +00:00 nvram_config
131105 -rwx 470296483 Nov 19 2024 21:14:45 +00:00 cat9k_lite_iosxe.17.12.04.SPA.bin
196609 drwx 4096 Nov 19 2024 21:13:41 +00:00 .installer
131089 -rw- 5242880 Nov 19 2024 06:30:42 +00:00 ssd
131101 -rw- 1156 Dec 5 2023 22:36:25 +00:00 vlan.dat
131103 -rw- 976 Oct 10 2023 01:28:37 +00:00 stby-vlan.dat
131100 -rw- 0 Oct 10 2023 01:26:29 +00:00 dope_hist
131097 -rw- 16953 Oct 10 2023 01:26:29 +00:00 rdope_out.txt
131098 -rw- 89 Oct 10 2023 01:26:28 +00:00 rdope.log
245761 drwx 4096 Oct 10 2023 01:26:12 +00:00 .prst_sync
131092 -rwx 2049 Oct 10 2023 01:25:33 +00:00 svl_ipc.tcl
131091 -rw- 134899 Oct 10 2023 01:25:33 +00:00 memleak.tcl
131090 -rw- 395 Oct 10 2023 01:25:26 +00:00 boothelper.log
286753 drwx 4096 Oct 10 2023 01:25:25 +00:00 dc_profile_dir
131088 -rw- 444 Oct 10 2023 01:25:12 +00:00 bootloader_evt_handle.log
327681 drwx 4096 Sep 14 2023 14:37:53 +00:00 pnp-info
196611 drwx 4096 Jul 24 2023 14:14:07 +00:00 .dbpersist
196619 drwx 4096 Jul 24 2023 14:07:21 +00:00 Apps
131102 -rw- 1440 Jul 24 2023 14:05:09 +00:00 2023_07_24T14_05_08.580029.xml
196612 drwx 4096 Jul 24 2023 14:04:48 +00:00 pnp-tech
212993 drwx 4096 Jul 24 2023 13:57:33 +00:00 core
352257 drwx 4096 Jul 24 2023 13:57:17 +00:00 guest-share
335873 drwx 4096 Jul 24 2023 13:57:13 +00:00 onep
311304 drwx 4096 Jul 24 2023 13:48:12 +00:00 .rommon_sync
131096 -rw- 1261 Jul 24 2023 13:48:10 +00:00 Baltimore.ca
131095 -rw- 1606 Jul 24 2023 13:48:10 +00:00 StarfieldServicesRootCA.ca
131094 -rw- 1338 Jul 24 2023 13:48:10 +00:00 DigiCertGlobalRootCA.ca
311297 drwx 4096 Jul 24 2023 13:48:09 +00:00 Tbot
294913 drwx 4096 Jul 24 2023 13:48:06 +00:00 fp_cc_crash
286755 drwx 4096 Jul 24 2023 13:48:06 +00:00 sys_report
286721 drwx 4096 Jul 24 2023 13:48:05 +00:00 tech_support
270337 drwx 4096 Jul 24 2023 13:48:02 +00:00 ss_disc
262145 drwx 4096 Jul 24 2023 13:47:56 +00:00 .rollback_timer
131087 -rw- 48649463 Jul 24 2023 13:45:26 +00:00 cat9k-rpboot.17.06.05.SPA.pkg
131086 -rw- 6514 Jul 24 2023 13:45:26 +00:00 packages.conf
131085 -rw- 9220 Jul 24 2023 13:45:26 +00:00 cat9k-wlc.17.06.05.SPA.pkg
131084 -rw- 17523716 Jul 24 2023 13:45:26 +00:00 cat9k-webui.17.06.05.SPA.pkg
131083 -rw- 32056328 Jul 24 2023 13:45:26 +00:00 cat9k-srdriver.17.06.05.SPA.pkg
131082 -rw- 61326340 Jul 24 2023 13:45:26 +00:00 cat9k-sipspa.17.06.05.SPA.pkg
131081 -rw- 37864456 Jul 24 2023 13:45:26 +00:00 cat9k-sipbase.17.06.05.SPA.pkg
131080 -rw- 698438660 Jul 24 2023 13:45:26 +00:00 cat9k-rpbase.17.06.05.SPA.pkg
131079 -rw- 9220 Jul 24 2023 13:45:25 +00:00 cat9k-lni.17.06.05.SPA.pkg
131078 -rw- 2098184 Jul 24 2023 13:45:25 +00:00 cat9k-guestshell.17.06.05.SPA.pkg
131077 -rw- 110638088 Jul 24 2023 13:45:25 +00:00 cat9k-espbase.17.06.05.SPA.pkg
131076 -rw- 19760140 Jul 24 2023 13:45:25 +00:00 cat9k-cc_srdriver.17.06.05.SPA.pkg

11353194496 bytes total (9048297472 bytes free)
PWSWCO01#boot system flash:packages.conf
^
% Invalid input detected at '^' marker.

PWSWCO01#$ file usbflash0:cat9k_lite_iosxe.17.12.04.SPA.bin activate commit
install_add_activate_commit: START Tue Nov 19 21:44:27 UTC 2024
Copying usbflash0:cat9k_lite_iosxe.17.12.04.SPA.bin to flash:cat9k_lite_iosxe.17 .12.04.SPA.bin
install_add_activate_commit: Adding PACKAGE
install_add_activate_commit: Checking whether new add is allowed ....

--- Starting initial file syncing ---
[1]: Copying flash:cat9k_lite_iosxe.17.12.04.SPA.bin from switch 1 to switch 2
[2]: Finished copying to switch 2
Info: Finished copying flash:cat9k_lite_iosxe.17.12.04.SPA.bin to the selected switch(es)
Finished initial file syncing

--- Starting Add ---
Performing Add on all members
[1] Add package(s) on switch 1
[1] Finished Add on switch 1
[2] Add package(s) on switch 2
[2] Finished Add on switch 2
Checking status of Add on [1 2]
Add: Passed on [1 2]
Finished Add

Image added. Version: 17.12.04.0.4826
install_add_activate_commit: Activating PACKAGE
FAILED: install_add_activate_commit : Lock held for excessive time for install_add_activate_commit.
Please run 'install commit' or 'install abort' to complete current operation..
PWSWCO01#install commit
install_commit: START Tue Nov 19 21:47:01 UTC 2024
install_commit: Committing SMU
FAILED: install_commit : Cannot commit; please use 'install activate' to activate added packages OR 'install remove inactive' to remove added subpackages.

Leo Laohoo · ‎11-20-2024

@StarLog wrote:

awk: fatal: cannot open file `/tmp/issu_prepare/rp/0/0/srdriver/mount/.pkginfo' for reading (No such file or directory)
awk: fatal: cannot open file `/tmp/issu_prepare/rp/0/0/rp_webui/mount/.pkginfo' for reading (No such file or directory)
awk: fatal: cannot open file `/tmp/issu_prepare/cc/0/0/cc_srdriver/mount/.pkginfo' for reading (No such file or directory)
awk: fatal: cannot open file `/tmp/issu_prepare/rp/0/0/rp_base/mount/.pkginfo' for reading (No such file or directory)
awk: fatal: cannot open file `/tmp/issu_prepare/rp/0/0/rp_security/mount/.pkginfo' for reading (No such file or directory)
awk: fatal: cannot open file `/tmp/issu_prepare/rp/0/0/rp_daemons/mount/.pkginfo' for reading (No such file or directory)
awk: fatal: cannot open file `/tmp/issu_prepare/fp/0/0/fp/mount/.pkginfo' for reading (No such file or directory)awk:
fatal: cannot open file `/tmp/issu_prepare/cc/0/0/cc/mount/.pkginfo' for reading (No such file or directory)
awk: fatal: cannot open file `/tmp/issu_prepare/rp/0/0/rp_iosd/mount/.pkginfo' for reading (No such file or directory)

Do this:

conf t
 service internal
end
wr
clear install state

This will cause the switch to reboot. After the reboot:

conf t
 NO service internal
end
wr

After this is done, then do the following:

copy usbflash0:cat9k_lite_iosxe.17.12.04.SPA.bin flash:

And then:

install add file usbflash0:cat9k_lite_iosxe.17.12.04.SPA.bin activate commit

IMPORTANT:

Never, ever, initiate the "install add file" from the USB flash drive. Don't ever do that again.

StarLog · ‎11-20-2024

Hi Leo,

Copy that, did not know.

Will see if we can have a window to install this week.

Leo Laohoo · ‎11-20-2024

Let us know how it goes.

The issue (or the bug) is caused by a failed ISSU install that has hovering around the system (even after a reboot) like a bad smell.