Solved: Cannot upgrade WS-C3850-24P-E from 03.06.09E to Gibraltar 16.x

Amills · ‎02-28-2022

I got a WS-C3850-24P-E switch running 03.06.09E and I want to upgrade to the latest firmware.

Switch always fails to update.

3850#software install file flash:cat3k_caa-universalk9.16.09.01.SPA.bin switch 1 force
Preparing install operation ...
[1]: Starting install operation
[1]: Expanding bundle flash:cat3k_caa-universalk9.16.09.01.SPA.bin
[1]: Copying package files
[1]: Package files copied
[1]: Finished expanding bundle flash:cat3k_caa-universalk9.16.09.01.SPA.bin
[1]: Verifying and copying expanded package files to flash:
[1]: % flash: requires 443052 KB of free space, but only 315844 KB is available. Operation aborted.
[1]: % An internal error was encountered. Operation aborted.

TEST#software install file flash:cat3k_caa-universalk9.16.09.01.SPA.bin
Preparing install operation ...
[1]: Starting install operation
[1]: Expanding bundle flash:cat3k_caa-universalk9.16.09.01.SPA.bin
[1]: Copying package files
[1]: Package files copied
[1]: Finished expanding bundle flash:cat3k_caa-universalk9.16.09.01.SPA.bin
[1]: Verifying and copying expanded package files to flash:
[1]: % flash: requires 443052 KB of free space, but only 315844 KB is available. Operation aborted.
[1]: % An internal error was encountered. Operation aborted.

I've tried installing 16.x Denali and Fuji firmware and it doesn't work either.

I found this thread but nothing suggested worked:

https://community.cisco.com/t5/other-network-architecture/update-c3850-from-3-x-to-16-x/td-p/3822156

Commands that use "request" do not work.

Leo Laohoo · ‎02-28-2022

@Amills wrote:

[1]: % flash: requires 443052 KB of free space, but only 315844 KB is available. Operation aborted.
[1]: % An internal error was encountered. Operation aborted.

This could, potentially, be the problem or a well-known bug.

Please post the complete output to the following commands:

dir
sh version

Also, refer to my attachment.

View solution in original post

Leo Laohoo · ‎02-28-2022

@Amills wrote:

[1]: % flash: requires 443052 KB of free space, but only 315844 KB is available. Operation aborted.
[1]: % An internal error was encountered. Operation aborted.

This could, potentially, be the problem or a well-known bug.

Please post the complete output to the following commands:

dir
sh version

Also, refer to my attachment.

Amills · ‎03-02-2022

TEST#dir
Directory of flash:/

23234 -rwx 2097152 Mar 2 2022 15:26:19 +00:00 nvram_config
23235 -rw- 35 Feb 24 2022 22:32:04 +00:00 pnp-tech-time
23236 -rw- 38543 Feb 24 2022 22:32:19 +00:00 pnp-tech-discovery-summary
23237 -rw- 453688163 Feb 28 2022 20:47:16 +00:00 cat3k_caa-universalk9.16.09.01.SPA.bin
23239 -rw- 22171522 Feb 28 2022 20:39:37 +00:00 cat3k_caa-rpbase.16.03.01.SPA.pkg
23240 -rw- 266148468 Feb 28 2022 20:39:39 +00:00 cat3k_caa-rpcore.16.03.01.SPA.pkg
23242 -rw- 9067132 Feb 28 2022 20:39:39 +00:00 cat3k_caa-srdriver.16.03.01.SPA.pkg
61954 -rw- 1238 Feb 24 2022 22:33:24 +00:00 packages.conf
23243 -rw- 178428528 Feb 28 2022 20:40:05 +00:00 cat3k_caa-wcm.16.03.01.SPA.pkg
69697 drwx 4096 Aug 27 2015 10:07:59 +00:00 mnt
23244 -rw- 13333112 Feb 28 2022 20:40:07 +00:00 cat3k_caa-webui.16.03.01.SPA.pkg
23251 -rw- 83284288 Feb 24 2022 22:33:18 +00:00 cat3k_caa-base.SPA.03.06.09E.pkg
23252 -rw- 3949504 Feb 24 2022 22:33:18 +00:00 cat3k_caa-drivers.SPA.03.06.09E.pkg
23253 -rw- 33805552 Feb 24 2022 22:33:19 +00:00 cat3k_caa-infra.SPA.03.06.09E.pkg
23254 -rw- 42815044 Feb 24 2022 22:33:19 +00:00 cat3k_caa-iosd-universalk9.SPA.152-2.E9.pkg
69698 drwx 4096 Aug 27 2015 10:27:34 +00:00 virtual-instance
77442 drwx 4096 Aug 27 2015 10:27:34 +00:00 vman_fdb
85196 drwx 4096 Mar 2 2022 15:25:32 +00:00 dc_profile_dir
85198 drwx 4096 Aug 27 2015 10:29:08 +00:00 wnweb_store
23241 -rw- 916 Nov 15 2019 14:44:00 +00:00 vlan.dat
85200 -rw- 28460176 Feb 24 2022 22:33:19 +00:00 cat3k_caa-platform.SPA.03.06.09E.pkg
85201 -rw- 112671812 Feb 24 2022 22:33:19 +00:00 cat3k_caa-wcm.SPA.10.2.190.0.pkg

1621966848 bytes total (323432448 bytes free)
TEST# sh ver
Cisco IOS Software, IOS-XE Software, Catalyst L3 Switch Software (CAT3K_CAA-UNIVERSALK9-M), Version 03.06.09E RELEASE SOFTWARE (fc4)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2018 by Cisco Systems, Inc.
Compiled Sat 08-Sep-18 18:21 by prod_rel_team

Cisco IOS-XE software, Copyright (c) 2005-2015 by cisco Systems, Inc.
All rights reserved. Certain components of Cisco IOS-XE software are
licensed under the GNU General Public License ("GPL") Version 2.0. The
software code licensed under GPL Version 2.0 is free software that comes
with ABSOLUTELY NO WARRANTY. You can redistribute and/or modify such
GPL code under the terms of GPL Version 2.0.
(http://www.gnu.org/licenses/gpl-2.0.html) For more details, see the
documentation or "License Notice" file accompanying the IOS-XE software,
or the applicable URL provided on the flyer accompanying the IOS-XE
software.

ROM: IOS-XE ROMMON
BOOTLDR: CAT3K_CAA Boot Loader (CAT3K_CAA-HBOOT-M) Version 1.18, RELEASE SOFTWARE (P)

TEST uptime is 14 minutes
Uptime for this control processor is 17 minutes
System returned to ROM by Power Failure
System image file is "flash:packages.conf"
Last reload reason: Power Failure

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

License Level: Ipservices
License Type: Permanent
Next reload license Level: Ipservices

cisco WS-C3850-24P (MIPS) processor with 4194304K bytes of physical memory.
Processor board ID FCW1927C0R0
1 Virtual Ethernet interface
28 Gigabit Ethernet interfaces
4 Ten Gigabit Ethernet interfaces
2048K bytes of non-volatile configuration memory.
4194304K bytes of physical memory.
250456K bytes of Crash Files at crashinfo:.
1609272K bytes of Flash at flash:.
0K bytes of Dummy USB Flash at usbflash0:.
0K bytes of at webui:.

Base Ethernet MAC Address : 38:ed:18:57:db:80
Motherboard Assembly Number : 73-15805-04
Motherboard Serial Number : FOC192725K1
Model Revision Number : U0
Motherboard Revision Number : A0
Model Number : WS-C3850-24P
System Serial Number : FCW1927C0R0

Switch Ports Model SW Version SW Image Mode
------ ----- ----- ---------- ---------- ----
* 1 32 WS-C3850-24P 03.06.09E cat3k_caa-universalk9 INSTALL

Configuration register is 0x102

TEST#$

Leo Laohoo · ‎03-02-2022

Thanks for the output.
It is not a bug. The switch has two (and a bit) firmware packages.
Read my attachment -- It is not long and it is not confusing. Should help clean up the flash and then update the firmware of the switch.

Amills · ‎03-02-2022

Ill give the attachment a shot, however, I should have done software clean before getting that output. I had only 1 software firmware on there but after the failed installs the packages from both firmware versions is in flash. I can clear it out with a software clean but the new install still fails

Leo Laohoo · ‎03-02-2022

@Amills wrote:

I can clear it out with a software clean but the new install still fails

Clean the flash and rerun the install command.
If it fails, show us the entire message.

In a more important question: Are you really sure you want to use 16.X.X?

Amills · ‎03-03-2022

Doing the install command with it on-reboot did the trick. I am able to upgrade the IOS.

software clean force

software install file flash: cat3k_caa-universalk9.16.03.07.SPA.bin new force verb on-reboot

The reasoning for going to 16.x.x is because every switch in production is currently on 16.3.7 version, I want to upgrade to the latest, 16.12.05b but before I did that I wanted to upgrade a switch in the lab and get it on that version. When the lab switch failed to install 16.3.7 I tested with other IOS versions to see if they worked which is why there is version 16.9.1 files in the show flash output I shared.

Leo Laohoo · ‎03-03-2022

@Amills wrote:

16.12.05b but before I did that I wanted to upgrade a switch in the lab and get it on that version.

3650/3850 has very "conservative" resources (CPU and memory).

3.6.X, 16.6.X and 16.9.X are very good versions to go to.

16.12.X, however, is not a stable train. Even more unstable if the switches are in a stack. If there is a need to use 16.12.X then I strongly recommend the CPU/Memory utilization is monitored daily. Reboot every 6 months, if necessary.