Cat3650 won't boot into IOS

[John Trumbell]

Hi

I've attempted to upgrade the IOS of 2 of my cat3650's. After rebooting them the one I didn't delete enough files to have room for the new IOS the second I did and it's giving me another error.

First Switch

switch: boot
Not enough free memory for entire image
Not enough free memory for entire image
flash:cat3k_caa-universalk9.16.03.07.SPA.bin: bad/non-bootable Nova bundle file

Error loading "flash:cat3k_caa-universalk9.16.03.07.SPA.bin"

Second switch

switch: boot flash:cat3k_caa-universalk9.16.03.07.SPA.bin
Not enough free memory for entire image
Not enough free memory for entire image
flash:cat3k_caa-universalk9.16.03.07.SPA.bin: bad/non-bootable Nova bundle file

Error loading "flash:cat3k_caa-universalk9.16.03.07.SPA.bin"

If I try and delete a file I get the following error message.

switch: copy flash:packages.conf flash:packages.old
flash:packages.old: read only file system

Here's the dir flash: from the second switch

switch: dir flash:
Directory of flash:/

77441 drwx 4096 .
2 drwx 4096 ..
77442 -rw- 2097152 nvram_config
77443 -rw- 79110340 cat3k_caa-base.SPA.03.03.04SE.pkg
77444 -rw- 6521532 cat3k_caa-drivers.SPA.03.03.04SE.pkg
77445 -rw- 34530288 cat3k_caa-infra.SPA.03.03.04SE.pkg
77449 -rw- 1247 packages.conf
77450 -rw- 537489817 cat3k_caa-universalk9.16.03.07.SPA.bin

821465088 bytes available (826429440 bytes used)

Here's the show boot from both switches

100Wel-SW8#sh boot
---------------------------
Switch 1
---------------------------
Current Boot Variables:
BOOT variable = flash:cat3k_caa-universalk9.16.0;

Boot Variables on next reload:
BOOT variable = flash:packages.conf
Allow Dev Key = yes
Manual Boot = no
Enable Break = no

Any thoughts???

Thanks

John

[Felipe A. Amaya]

Hi 

If you need to delete files you may need to try the following:

delete /force /recursive flash: (name of file)

hope pe it helps 

[John Trumbell]

I believe I tried that and it gave me the following error, but I'll try it again after the call I'm about to go on to. thanks

switch: delete /force flash:cat3k_caa-wcm.SPA.10.1.140.0.pkg
Are you sure you want to delete "/force" (y/n)?y
File "/force" not deleted -- no such device
Are you sure you want to delete "flash:cat3k_caa-wcm.SPA.10.1.140.0.pkg" (y/n)?y
File "flash:cat3k_caa-wcm.SPA.10.1.140.0.pkg" not deleted -- read only file system

[eglinsky2012 · Accepted solution]

I had this exact issue going to 16.3.7 on an older 3650 running 3.6.4E with bootloader 1.18. Opened a TAC case seeking RMA thinking there was a memory issue with the switch.

Here is TAC's response:

When reviewing the table on the link below we can see that we should have the bootloader version to at least 3.2 which would mean that the device has been booted at some point on 16.1.X. Which would allow you to boot the 16.3.7 in bundle mode.

So for this particular deployment of IOS-XE 16.X versions in Bundle mode you could check if the switch has a bootloader code of 3.2 or later and then you could go to 16.3.7 directly. For the ones that have a bootloader code less than 3.2 then you could boot any 16.3.X code below 16.3.5 to have the bootloader version updated and then boot 16.3.7.

Automatic Boot Loader Upgrade

https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst3850/software/release/16-3/release_notes/ol-16-3-3850.html#pgfId-1127089

I took a more convoluted upgrade path than I believe I needed to, but I would put the 16.3.3 BIN file on a flash drive and boot off that, which will upgrade the bootloader to 4.38. Then you can boot the 16.3.7 BIN file, and the bootloader will be upgraded to 4.56.

This link about upgrading in bundle mode was also provided:

https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst3850/software/release/16-3/release_notes/ol-16-3-3850.html#pgfId-1430489

"Starting from 16.3.5 release, you will not be able to boot Cisco IOS XE Denali 16.3.5 in bundle mode via flash drive for the first time with a Cisco IOS XE 3.xE boot loader. The Cisco IOS XE 3.xE boot loaders have a limitation that they cannot boot an image larger than 512MB via flash. Refer to the upgrade sections in install mode."

I think the boot loader requirements are different for different hardware revisions of the 3650, since that was the only switch out of over 100 that we had this issue with, and we had another 3650 running 3.6.4E and the bootloader was not upgraded when I went to 16.3.3.

[John Trumbell]

Thanks, I guess I have a little fun ahead of me. :)

I guess I'll be booting the devices from a USB drive, to recover the switches. then do my upgrades, this time reading the release notes. lol

[eglinsky2012]

If you have both the 16.3.3 and 16.3.7 BIN files on the flash drive, after iOS 16.3.3 is running off the flash, you can copy the BIN from USB to flash. If I recall, that would be:

copy usbflash0:filename.bin flash:filename.bin

Then change the boot variable accordingly:

boot system flash:filename.bin

Then you should be all set to remove the flash drive and reload.

[Leo Laohoo]

Emergency Recovery