Howdy, so I didn't try this

s-phipps · ‎08-11-2015

We need to stack 3850 switches for our application. I can use 'software install file' on the switches individually from USB, but when they are configured in a stack the install fails.

Two different laptops:

HP Specter360x Windows 10 - 64

IMB Thinkpad Windows 7 - 64

USB: PNY 8 & 16 Gig, Cruser 4 & 8 Gig, Unnamed 1 Gig

USB, TFTP fail in stacks on front ports and management interface.

Messages are:

In stacks of 5 or less:

All USB stick show something like this:

[%IOSXE-3-PLATFORM: 1 process kernel: FAT: FAT read failed (blocknr 3832)
%IOSXE-3-PLATFORM: 1 process kernel: FAT: FAT read failed (blocknr 3832)
%IOSXE-3-PLATFORM: 1 process kernel: FAT: FAT read failed (blocknr 3832)
%IOSXE-3-PLATFORM: 1 process kernel: FAT: FAT read failed (blocknr 3832)
%IOSXE-3-PLATFORM: 1 process kernel: FAT: FAT read failed (blocknr 3832)
%IOSXE-3-PLATFORM: 1 process kernel: FAT: FAT read failed (blocknr 3832)
%IOSXE-3-PLATFORM: 1 process kernel: FAT: FAT read failed (blocknr 3832)
%IOSXE-3-PLATFORM: 1 process kernel: FAT: FAT read failed (blocknr 3832)
%IOSXE-3-PLATFORM: 1 process kernel: FAT: FAT read failed (blocknr 3832)

1gig USB:

Preparing install operation ...
[1]: Copying software from active switch 1 to switches 2,3,4
[4]: % usbflash0: requires 303074 KB of free space, but only 0 KB is available
[4]: % Failed to copy file usbflash0://cat3k_caa-universalk9.SPA.03.07.00.E.152-3.E.bin from active switch 1 to switch 4, operation aborted
[2]: % usbflash0: requires 303074 KB of free space, but only 0 KB is available
[2]: % Failed to copy file usbflash0://cat3k_caa-universalk9.SPA.03.07.00.E.152-3.E.bin from active switch 1 to switch 2, operation aborted
[3]: % usbflash0: requires 303074 KB of free space, but only 0 KB is available
[3]: % Failed to copy file usbflash0://cat3k_caa-universalk9.SPA.03.07.00.E.152-3.E.bin from active switch 1 to switch 3, operation aborted

Failed to copy file

In a stack of 9(nine) the usb stick is FORMATTED, or the file is deleted(ERASED).

This happens if you run copy, verify /md5, or software install file... The USB is erased.

Is anyone else having this issue?

InayathUlla Sharieff · ‎08-11-2015

Hello S-Phipps,

Plesae find quick guide on upgrading and booting, see below as a reference.

Recovering from a 3850 boot failure.

There are multiple reasons a 3850 may fail to boot correctly including a corrupt boot image, a corrupt packages.conf file, missing files, etc. Below are a few different possible recovery methods to try. I will also explain the two possible mode options, Install and bundle and why you might want to use one or the other.

Install vs. Bundle Mode

There are a few difference in the two modes, I would recommend reading over the config guide for more in-depth details. The recommended mode during operation is INSTALL mode because it allows for more features and requires fewer resources when booting.

++Install Mode

This is the out-of-the-box mode that your switch will be in. INSTALL mode uses a package provisioning file named packages.conf to boot the switch.

If you happen to be in bundle mode upon boot, you can simply boot your switch in install mode by booting the software package provisioning file that resides in flash. If packages.conf doesn�t exist in flash, you need to expand the bundle into the flash file system by running

Switch# software expand file flash: cat3k_caa-universalk9.SPA.03.02.02.SE.150-1.EX2.bin to flash:

Once this completes, you will have all the needed files in flash. You can then change the boot statement to boot to packages.conf

Switch#Config t
Switch(config)# no boot system
Switch(config): boot system switch all flash:packages.conf (do not modify this file, unless necessary)
Switch#write memory

The provisioning file contains a list of software packages to boot, mount, and run. The ISO file system in each installed package is mounted to the root file system directly from flash.

NOTE **Auto-upgrade is disabled, by default. (once in install mode - execute the following command in global config: software auto-upgrade enable )

NOTE **Auto-upgrade includes an auto-copy process and an auto-extract process.

++Bundle Mode

As noted previously, bundle mode consumes more memory than booting in install mode because the packages are extracted from the bundle and copied to the RAM. If you decide to convert to bundle mode, you will first need to download the .bin file from CCO if you don�t already have it in flash. Once in flash, you can simply change your boot statement to point to the (.bin) file:

Switch#Config t
Switch(config)# no boot system
Switch(config): boot system switch all flash: cat3k_caa-universalk9.SPA.03.02.02.SE.150-1.EX2.bin
Switch#write memory

The provisioning file contained in a bundle is used to decide which packages to boot, mount, and run. Packages are extracted from the bundle and copied to RAM.

NOTE **Auto install and smart install functionality is not supported in bundle boot mode.

Recovery Methods

USB

The 3850 has a USB port on the front that can be used for both console access and also the ability to utilize a flash drive for image backup and recovery.

If you happen to be stuck at the switch: prompt with a corrupt image or .conf file, you can easily boot to a file stored on the USB drive.

1. Verify that the flashdrive is recognized and the .bin file exists

switch: dir usbflash0:
Directory of usbflash0:/

74 -rw- 223734376 cat3k_caa-universalk9.SPA.03.02.02.SE.150-1.EX2.bin

2. Boot to the USB image

switch: boot usbflash0:cat3k_caa-universalk9.SPA.03.02.02.SE.150-1.EX2.bin

Corrupt packages.conf

I�ve seen instances in which packages.conf continually calls files that no longer exist in flash. You can boot to an image from ROMMON just fine, however upon reload it will call packages.conf again and fail to boot. If this happens, I recommend backing up the existing packages.conf file by renaming it or deleting all together. NOTE: The previous step is mandatory as the next step will fail if a .conf file already exists. You can then run an BUNDLE extract which will create a new packages.conf file.

1. Once booted up (in BUNDLE mode) verify the files in flash

Switch#dir flash:
Directory of flash:/

15500 -rwx 1243 Aug 1 2013 07:04:02 +00:00 packages.conf

2. Copy or rename the existing packages.conf file

Switch#cp flash:packages.conf flash:packages.conf.badop flash:packages.conf flash:packages.conf.bad
Destination filename [packages.conf.bad]?
Copy in progress...C
1243 bytes copied in 0.140 secs (8879 bytes/sec)
Switch#dir flash:
Directory of flash:/
15500 -rwx 1243 Aug 1 2013 07:04:02 +00:00 packages.conf
15502 -rw- 1243 Aug 1 2013 11:53:51 +00:00 packages.conf.bad

3. Delete packages.conf

Switch#del flash:packages.conf
Delete filename [packages.conf]?
Delete flash:/packages.conf? [confirm]

4. Expand BUNDLE to create new packages.conf

Switch#software expand running switch 1 to flash:
Preparing expand operation ...
[1]: Expanding the running bundle
[1]: Copying package files
[1]: Package files copied
[1]: Finished expanding the running bundle

5. Verify boot

Switch#show boot
---------------------------
Switch 1
---------------------------
Current Boot Variables:
BOOT variable does not exist

Boot Variables on next reload:
BOOT variable = flash:packages.conf;
Manual Boot = no
Enable Break = no

6. Reload Switch

switch#reload
Reload command is being issued on Active unit, this will reload the whole stack
Proceed with reload? [confirm]

Emergency Recovery

If all else fails, the 3850 has a �trap door� method of recovering the system. All you need is a terminal connected to the management port of the 3850 running a tftp server. Download a valid image file from CCO and store it in the root of the tftp server.

On the switch, you are most likely stuck at the switch: prompt. If however you are in some sort of boot loop, you can use the �mode� button on the front of the switch to break the cycle. Simply hold the button for roughly 10 seconds and the switch should react by breaking the cycle and stopping at a switch: prompt. The following steps will walk you through the recovery:

1. Set the switch IP

switch: set IP_ADDR 192.0.2.123/255.255.255.0

2. Set the default gateway

switch: set DEFAULT_ROUTER 192.0.2.1

3.Test connectivity by pinging terminal (that contains the tftp server)

switch: ping 192.0.2.1
ping 192.0.2.1 with 32 bytes of data ...
Host 192.0.2.1 is alive.

4. Verify that the emergency files exist in the switches file system

switch: dir sda9:
Directory of sda9:/

2 drwx 1024 .
2 drwx 1024 ..
11 -rwx 18958824 cat3k_caa-recovery.bin

36903936 bytes available (20866048 bytes used)

5. Run the emergency install feature

switch: emergency-install tftp://192.0.2.1/cat3k_caa-universalk9.SPA.03.02.02.SE.150-1.EX2.bin
The bootflash will be erased during install operation, continue (y/n)?Y
Starting emergency recovery (tftp://192.0.2.1/cat3k_caa-universalk9.SPA.03.02.02.SE.150-1.EX2.bin)...
Reading full image into memory......................done
Nova Bundle Image
--------------------------------------
Kernel Address : 0x6042f5d8
Kernel Size : 0x317ccc/3243212
Initramfs Address : 0x607472a4
Initramfs Size : 0xdc6546/14443846
Compression Format: .mzip

Bootable image at @ ram:0x6042f5d8
Bootable image segment 0 address range [0x81100000, 0x81b80000] is in range [0x80180000, 0x90000000].
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
File "sda9:cat3k_caa-recovery.bin" uncompressed and installed, entry point: 0x811060f0
Loading Linux kernel with entry point 0x811060f0 ...
Bootloader: Done loading app on core_mask: 0xf

### Launching Linux Kernel (flags = 0x5)

Initiating Emergency Installation of bundle tftp://192.0.2.1/cat3k_caa-universalk9.SPA.03.02.02.SE.150-1.EX2.bin

Downloading bundle tftp://192.0.2.1/cat3k_caa-universalk9.SPA.03.02.02.SE.150-1.EX2.bin...

Validating bundle tftp://192.0.2.1/cat3k_caa-universalk9.SPA.03.02.02.SE.150-1.EX2.bin...
Installing bundle tftp://192.0.2.1/cat3k_caa-universalk9.SPA.03.02.02.SE.150-1.EX2.bin...
Verifying bundle tftp://192.0.2.1/cat3k_caa-universalk9.SPA.03.02.02.SE.150-1.EX2.bin...
Package cat3k_caa-base.SPA.03.02.02.SE.pkg is Digitally Signed
Package cat3k_caa-drivers.SPA.03.02.02.SE.pkg is Digitally Signed
Package cat3k_caa-infra.SPA.03.02.02.SE.pkg is Digitally Signed
Package cat3k_caa-iosd-universalk9.SPA.150-1.EX2.pkg is Digitally Signed
Package cat3k_caa-platform.SPA.03.02.02.SE.pkg is Digitally Signed
Package cat3k_caa-wcm.SPA.10.0.111.0.pkg is Digitally Signed
Preparing flash...
Syncing device...
Emergency Install successful... Rebooting
Restarting system.

Please let me know if you have any further questions.

HTH

Regards

Inayath

s-phipps · ‎08-11-2015

Thank you so much for the reply. However, none of these match my issue. What I am encountering is a complete failure of the upgrade process, the copy process, or the verify process on stacks of 3850. If I stack these switches, I cannot even run the upgrade from the flash0: on the switch 1. But I can break the switch stack and run all these commands, copy, verify, and upgrade using software install.

jhager001 · ‎02-17-2016

Just having to do this across ~40 locations, i've strongly found out that 3850's are not great platforms.... especially with the 25-30 minute boot time...

I've found if you have only one switch, you can software install file from USB

if you have more than 1 switch, you can copy IOS.bin from USB to flash and install from flash

If you run into Signature verification failed (which has happened multiple times), then software install file from tftp. This route may take a little longer depending on bandwidth between sites, but it works like a charm every time.

Reuben Farrelly · ‎03-08-2017

I managed to figure out how to get around this problem - running IOS XE Denali 16.3. Basically you will need to copy the consolidated bin file to each member of the stack and then upgrade each one individually, like this:

request platform software package install switch 2 file flash-2:cat3k_caa-universalk9.16.03.03.SPA.bin on-reboot retain-source-file new verbose

Repeat for 1, 3, 4, 5 etc - just do them one at a time.

I haven't tried copying the bin file to ALL members and then running the command with 'all' instead of the switch numbers, but at least individually it works fine. What does not work is if you try and install over ftp to all members. It just fails with weird messages about the file not being present.

JAYESHPALLIKOL · ‎04-19-2020

%IOSXEBOOT-4-ABORT: (rp/0): Package provisioning file {/tmp/packages.conf} does not exist.

got this error after this step
switch: emergency-install tftp://192.0.2.1/cat3k_caa-universalk9ldpe.SPA.03.06.04.E.152-2.E4.bin

Corey Koellein · ‎10-25-2015

I am seeing the same issue on ANY stacked 3850s. I can update individual switches 100% of the time. But stack installs fail 100% of the time.

Output just like OP:

<QUOTE>

Preparing install operation ...
[1]: Copying software from active switch 1 to switches 2,3,4
[4]: % usbflash0: requires 303074 KB of free space, but only 0 KB is available
[4]: % Failed to copy file usbflash0://cat3k_caa-universalk9.SPA.03.07.00.E.152-3.E.bin from active switch 1 to switch 4, operation aborted
[2]: % usbflash0: requires 303074 KB of free space, but only 0 KB is available
[2]: % Failed to copy file usbflash0://cat3k_caa-universalk9.SPA.03.07.00.E.152-3.E.bin from active switch 1 to switch 2, operation aborted
[3]: % usbflash0: requires 303074 KB of free space, but only 0 KB is available
[3]: % Failed to copy file usbflash0://cat3k_caa-universalk9.SPA.03.07.00.E.152-3.E.bin from active switch 1 to switch 3, operation aborted

Failed to copy file

</QUOTE>

I have tried several IOSXE versions. Mostly in the 03.03.xx family.

I have tried several different USBflash drives.

I use the same USBflash drives successfully to install on single switches, with 100% success rate.

HOW DO I UPDATE A STACK?!

s-phipps · ‎10-25-2015

Cory,

The TAC I opened for this is SR: 635926489. They said they could not reproduce this, but you and I both know that is untrue. I was told that to upgrade the stack, you have to put a usb drive in each of the stack members. Truthfully, I have never heard anything so ridiculous in my life. I actually tried this method, and it failed as bad as it did with one USB drive.

----

++++Resolution Sumary++
The correct operation when doing software install or software expand is to have a USB device install in each switch of the stack. From the tac toi ppt

When using the software expand command in a stack with a source bundle that resides in a local storage device, the source bundle is first copied from the local storage device on the active switch to
the corresponding local device on all other switches.

(Same statement is also seen for software install)

So when the customer sees the fail message % usbflash0: requires 303074 KB of free space, but only 0 KB is available, that is expected because he does not have a USB in the other switches.

The behavior changed in 03.06.xx and 03.07.xx images to only require 1 USB flash device in the active switch

Customer will open an enhacement request to modify this behavior.

----

I think they just wanted to close the ticket. You see they just posted the 'upgrade' notes for an answer, which did not address the issue I asked about.

Sadly, to upgrade a stack with USB, TFTP, or FTP, you must break the stack. If you try to upgrade from any 3.3.x code to 3.6.x, or 3.7.x, you may run into another upgrade bug: CSCuw82216. This one happens more frequently in a stack, as I lost 3(three) switches, which went back RMA in a stack. After that, I broke all the stacks to upgrade individually. I lost another 3(three) out of 93 to this individually. Here's what TAC sent back. The case is still open: SR: 636382307

----

Hi Steve,
Also, software defect has been filed recently for the corruption of the flash. No fix yet.

https://tools.cisco.com/bugsearch/bug/CSCuw82216/?reffering_site=dumpcr

(3650 and 3850 shares same architecture).
Regards,
Juan Fernndez
Cisco TAC - LAN Switching

----

They say the bug is fixed in 3.6.5, and 3.7.2, but you have to get there from 3.3, and that's what will trip you up. I wish I had better news. You can run the file from USB in bundle-mode, but the switch won't boot from flash after that. You're basically replacing flash: with usbflash0: It goes strait to rommon, or continually reboots from flash.

Corey Koellein · ‎10-26-2015

Well, even though its bad news... or I should say, what has become standard fare over the last few years from Cisco... I can't tell you how much I appreciate you validating this for me. At least now I feel confident just adding the time to ALLLLLLLLLLL of my switch maintenance from now on to compensate. It is what it is. This is what happens when monopolies take hold, even if they are born out of meteoric success of the company's early years. If they'd spend HALF the time and money they seem to spend on spamming me marketing hype for new products that aren't ready for production... oh what's the use! :-)

Take care, and Thanks, again!

kc

Corey Koellein · ‎10-26-2015

FOR THOSE WHO MAY FIND THIS POST, THIS IS THE BEST RESOLUTION I HAVE FOUND, AND WHAT I AM TAKING TO PRODUCTION:

Through several recent trials and reinforced by a detailed forum response, I have concluded that there is no way to make a 3850 stack upgrade work properly.

Individual switches upgrade flawlessly 100% of the time. However, for a STACK, the following procedure is highly recommended.

1) **** IDENTIFY AND “SHUT” OR DISCONNECT EACH UPLINK **** (No split-braining or looping, please.)

2) Disconnect one-half of each stack-data cable.

3) Insert usbflash in a switch. Wait for it to initialize. It WILL BE usbflash0:

4) From priv exec mode, execute:

- software install file usbflash0:/cat<TAB>

- software install file usbflash0:/cat3k_caa-universalk9.SPA.03.03.05.SE.150-1.EZ5.bin new verbose

("verbose" will let you keep an eye on the output in case of errors.)

5) Accept reload prompt.

6) Move console cable and usbflash to next switch.

7) From priv exec mode, execute:

- software install file usbflash0:/cat<TAB>

- software install file usbflash0:/cat3k_caa-universalk9.SPA.03.03.05.SE.150-1.EZ5.bin new verbose

8) Accept reload prompt.

9) Move console cable and usbflash to next switch…

10) Upon reloading the last switch, return console cable to the first switch.

11) From priv exec mode, execute:

- software clean

- confirm at delete prompt

12) Move console cable to next switch.

13) From priv exec mode, execute:

- software clean

- confirm at delete prompt

14) Move console cable to next switch…

15) Upon completing last switch, return console cable to the first switch.

16) Reconnect stack data cables. (STACK WILL REBOOT ANY TIME YOU STACK/ADD A FULLY BOOTED SWITCH TO ANOTHER FULLY BOOTED SWITCH OR STACK.)

17) Upon reboot, verify all switches show online with same version with a “show version” command.

18) **** ”NO SHUT” EACH PREVIOUSLY IDENTIFIED UPLINK ****

It IS possible to do a clean install using just “format flash” with no further parameters. However, if you do, the only way* to install the new OS is to reload afterward, and run “emergency-install” from switch: prompt. Even though “emergency-install” sounds scary, it works quite well, so that IS an option, though I know from chatting with coworkers that most of you won’t be comfortable with it. I have proven it out in production, though.

The command would be: (at the switch: prompt)

switch: emergency-install usbflash0:/cat3k_caa-universalk9.SPA.03.03.05.SE.150-1.EZ5.bin

*Throughout all of these steps, you may note additional options to explore – such as: provisioning files, bundle-mode, or specifying a switch number. Enjoy chasing those options. I’m not saying they won’t work. (Definitely not saying they will.) I am saying that my method has worked for me in several bench tests and in production trials with versions 03.03.03/03.03.04/03.03.05 on Oct 25th 2015. And the common sense method of expecting the stack to be able to manage copying some flash files does NOT work.

salleek · ‎08-18-2016

Howdy, so I didn't try this from USB stick. Went from 3.0.6 to 16.3.1. I did do it remotely via SSH. Several times I tried.

Steps to breakage:

1. SCP files to switch1 flash: and did md5 verify

2. Issue command as follows:

sea-c3650-stk#software install file flash:cat3k_caa-universalk9.16.03.01.SPA.bin new force
Preparing install operation ...
[1]: Copying software from active switch 1 to switch 2
[2]: % A different version of cat3k_caa-universalk9.16.03.01.SPA.bin already exists in flash:
[2]: % Failed to copy file flash:cat3k_caa-universalk9.16.03.01.SPA.bin from active switch 1 to switch 2, operation aborted

3. So I did this:

sea-c3650-stk#sho flash-2:
-#- --length-- ---------date/time--------- path
2 82665136 Jun 08 2016 00:49:05.0000000000 +00:00 cat3k_caa-base.SPA.03.06.04.E.pkg
3 4913852 Jun 08 2016 00:49:05.0000000000 +00:00 cat3k_caa-drivers.SPA.03.06.04.E.pkg
4 33784816 Jun 08 2016 00:49:05.0000000000 +00:00 cat3k_caa-infra.SPA.03.06.04.E.pkg
5 43021636 Jun 08 2016 00:49:05.0000000000 +00:00 cat3k_caa-iosd-universalk9.SPA.152-2.E4.pkg
6 27417488 Jun 08 2016 00:49:05.0000000000 +00:00 cat3k_caa-platform.SPA.03.06.04.E.pkg
7 111180608 Jun 08 2016 00:49:05.0000000000 +00:00 cat3k_caa-wcm.SPA.10.2.140.0.pkg
8 1244 Jun 08 2016 00:49:19.0000000000 +00:00 packages.conf
9 4096 Jul 29 2016 18:00:39.0000000000 +00:00 dc_profile_dir
10 183714 Jul 29 2016 18:00:39.0000000000 +00:00 dc_profile_dir/dc_default_profiles.txt
11 183714 Jul 29 2016 18:00:39.0000000000 +00:00 dc_profile_dir/dc_default_profiles.txt.bkp
12 2097152 Aug 18 2016 18:16:40.0000000000 +00:00 nvram_config
13 10272 Jul 29 2016 18:01:37.0000000000 +00:00 vlan.dat
14 5509 Jun 29 2016 05:09:48.0000000000 +00:00 -Jun-29-05-09-41.990-0
15 7966 Jul 06 2016 05:13:52.0000000000 +00:00 .Jul--5-22-13-34.031-1
16 7953 Jul 13 2016 05:14:02.0000000000 +00:00 .Jul-12-22-13-53.310-2
17 7953 Jul 20 2016 05:14:13.0000000000 +00:00 .Jul-19-22-14-03.724-3
18 7953 Jul 27 2016 05:14:23.0000000000 +00:00 .Jul-26-22-14-13.892-4
19 465900394 Jul 28 2016 23:11:02.0000000000 +00:00 cat3k_caa-universalk9.16.02.02.SPA.bin
20 9100 Jul 29 2016 15:52:59.0000000000 +00:00 -Jul-29-08-52-50.026-0
21 21904763 Jul 28 2016 23:24:20.0000000000 +00:00 cat3k_caa-rpbase.16.02.02.SPA.pkg
22 253350520 Jul 28 2016 23:24:21.0000000000 +00:00 cat3k_caa-rpcore.16.02.02.SPA.pkg
23 7330432 Jul 28 2016 23:24:21.0000000000 +00:00 cat3k_caa-srdriver.16.02.02.SPA.pkg
24 165884532 Jul 28 2016 23:24:36.0000000000 +00:00 cat3k_caa-wcm.16.02.02.SPA.pkg
25 17425020 Jul 28 2016 23:24:37.0000000000 +00:00 cat3k_caa-webui.16.02.02.SPA.pkg
26 10684 Jul 29 2016 17:44:36.0000000000 +00:00 -Jul-29-10-44-26.840-0
27 11491 Aug 05 2016 17:58:27.0000000000 +00:00 -Aug--5-10-58-07.922-1
28 11491 Aug 12 2016 17:58:38.0000000000 +00:00 -Aug-12-10-58-28.501-2
29 380284928 Aug 18 2016 21:32:16.0000000000 +00:00 cat3k_caa-universalk9.16.03.01.SPA.bin
0 bytes available (1622147072 bytes used)

I had no space on the secondary switch flash. So I did this:

del flash-2:<all the files to delete here>

Got this pretty stuff:

sea-c3650-stk#software install file flash:cat3k_caa-universalk9.16.03.01.SPA.bin new force
Preparing install operation ...
[1]: Copying software from active switch 1 to switch 2
[1]: Finished copying software to switch 2
[1 2]: Starting install operation
[1 2]: Expanding bundle flash:cat3k_caa-universalk9.16.03.01.SPA.bin
[1 2]: Copying package files
[1 2]: Package files copied
[1 2]: Finished expanding bundle flash:cat3k_caa-universalk9.16.03.01.SPA.bin
[1 2]: Verifying and copying expanded package files to flash:
[1 2]: Verified and copied expanded package files to flash:
[1 2]: Starting compatibility checks
[1 2]: Bypassing peer package compatibility checks due to 'force' command option
[1 2]: Finished compatibility checks
[1 2]: Starting application pre-installation processing
[1 2]: Finished application pre-installation processing
[1]: Old files list:
Removed cat3k_caa-base.SPA.03.06.04.E.pkg
Removed cat3k_caa-drivers.SPA.03.06.04.E.pkg
Removed cat3k_caa-infra.SPA.03.06.04.E.pkg
Removed cat3k_caa-iosd-universalk9.SPA.152-2.E4.pkg
Removed cat3k_caa-platform.SPA.03.06.04.E.pkg
Removed cat3k_caa-wcm.SPA.10.2.140.0.pkg
[2]: Old files list:
Removed cat3k_caa-base.SPA.03.06.04.E.pkg
Removed cat3k_caa-drivers.SPA.03.06.04.E.pkg
Removed cat3k_caa-infra.SPA.03.06.04.E.pkg
Removed cat3k_caa-iosd-universalk9.SPA.152-2.E4.pkg
Removed cat3k_caa-platform.SPA.03.06.04.E.pkg
Removed cat3k_caa-wcm.SPA.10.2.140.0.pkg
[1]: New files list:
Added cat3k_caa-rpbase.16.03.01.SPA.pkg
Added cat3k_caa-rpcore.16.03.01.SPA.pkg
Added cat3k_caa-srdriver.16.03.01.SPA.pkg
Added cat3k_caa-wcm.16.03.01.SPA.pkg
Added cat3k_caa-webui.16.03.01.SPA.pkg
[2]: New files list:
Added cat3k_caa-rpbase.16.03.01.SPA.pkg
Added cat3k_caa-rpcore.16.03.01.SPA.pkg
Added cat3k_caa-srdriver.16.03.01.SPA.pkg
Added cat3k_caa-wcm.16.03.01.SPA.pkg
Added cat3k_caa-webui.16.03.01.SPA.pkg
[1 2]: Creating pending provisioning file
[1 2]: Finished installing software. New software will load on reboot.
[1 2]: Committing provisioning file

[1 2]: Do you want to proceed with reload? [yes/no]:
% Please answer 'yes' or 'no'.
[1 2]: Do you want to proceed with reload? [yes/no]: yes
[1 2]: Reloading

bmcminn · ‎12-26-2017

reminder to read the release notes. Thank you salleek
https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst3850/software/release/16-2/release_notes/ol-16-2-3850.html#pgfId-1130408

jgarzaga · ‎06-03-2020

After upgrading 200+ I encounter this issue with several stack switches. This doesn`t have to do with the number of switches in the stack, or if there's enough space in the flash of the switch members'. For some reason, the switch is not allowing to copy the .bin file to its members regardless of the code running (right now I'm using 03.07.05E Install mode).

One thing that is worth trying is to reload the switch and do the install command one it boots. (In my environment I was not able to since it was productive and I didn't want to wait until the maintenance window to try. What I did was:

1.- Copy the .bin file to all switch members. Once the .bin file is on the active switch just copy it down to all.

stack-1#copy flash:new-code.bin flash-2:

stack-1#copy flash:new-code.bin flash-3:

.

stack-1#copy flash:new-code.bin flash-8:

2.- Install the code individually in all switch members using the on-reboot key at the end. That way you'll be able to reboot it after all switches get the install command.

stack-1#software install file flash-1:new-code.bin new verbose switch 1 on-reboot
stack-1#software install file flash-2:new-code.bin new verbose switch 2 on-reboot

.

stack-1#software install file flash-8:new-code.bin new verbose switch 8 on-reboot

3.- Double check the packages.conf file, you should see the packages of the new image showing up.

stack-1#term len 0

stack-1#more flash-1:packages.conf

stack-1#more flash-2:packages.conf

.

stack-1#more flash-8:packages.conf

4.- Check the boot system is pointing to packages.conf

stack-1#show boot system

---------------------------
Switch 1
---------------------------
Current Boot Variables:
BOOT variable = flash:packages.conf;

Boot Variables on next reload:
BOOT variable = flash:packages.conf;
Manual Boot = no
Enable Break = no
Boot Mode = DEVICE
iPXE Timeout = 0
---------------------------
Switch 2
---------------------------
Current Boot Variables:
BOOT variable = flash:packages.conf;

Boot Variables on next reload:
BOOT variable = flash:packages.conf;
Manual Boot = no
Enable Break = no
Boot Mode = DEVICE
iPXE Timeout = 0

5.- Reload the switch

stack-1#reload
Reload command is being issued on Active unit, this will reload the whole stack
Proceed with reload? [confirm]

I hope you find his post useful. :)

Cisco 3850 upgrade fails in a stack configuration stack of 9 erases USB stick