Is it possible to upgrade the IOS on a cisco 3850 stack in a non disruptive way (revisited)

[David Dobbs]

I've read the post from 3 years past regarding this subject and the consensus was you must reboot the entire switch stack to upgrade ios, and this was ok because device connectivity is lost anyway even if you rebooted the switches individually.

However, this does not address a design where port channels are used to connect a device (e.g. router) to two switches in a stack making the router highly available.  With this design, rebooting individual switches instead of the entire stack would be desirable to prevent service interruption.

So can someone tell me if this methodology will work--basically physically unstacking the switches, rebooting each individual switch, then restacking the switches:

Just load the new image to a USB and then copy it to the switches via the following commands:

1. Switch#copy usbflash0:[image name] flash:

2. Repeat the same command for the second switch and so on

3. Switch#copy usbflash0:[image name] flash2:

4. Once done change the boot system to the newly uploaded image

   Switch(config)# boot syst switch all flash:[image name]

5. Disconnect the stackwise cables isolating the switches

6. Reload each switch one at a time so you only loose one switch at a time out of the stack

7. After all switches have been upgraded, reattach stackwise cables to join all the switches to the stack.

Will this work for a non-disruptive ios upgrade?

[Leo Laohoo]

5. Disconnect the stackwise cables isolating the switches

I don't understand the logic behind step 5.  Disconnecting the stacking cable means sending the stack into a massive "split brain".   If this isn't a "disruption" of a massive scale then I don't know what is.  

6. Reload each switch one at a time so you only loose one switch at a time out of the stack

3650/3850 does NOT support Rolling Stock Upgrade.  Step 6 will cause more disruption because the stack members booting the new IOS version will be unable to join the existing stack because of version mismatch. 

7. After all switches have been upgraded, reattach stackwise cables to join all the switches to the stack.

Ever tried attaching a stacking cable when switch is powered up?   The switch will reboot and the entire stack will also reboot.  

Here's my recommendation:  Instead of "copying" the IOS into each switch, try the Install method and then organize for a 10 minute outage.  

[David Dobbs]

Leo, thanks for your input.

The logic for step 5 was to avoid the version mismatch on the stack while upgrading; however, I was unaware that reattaching stack cables will cause a stack reboot.  That's why I put this out in the forum to see if this "theory" was feasible.

Would it be possible to use the "reload slot {stack-member-number}" to reboot each individual switch without disconnecting the cabling to get the same effect, or would it complain too much and cause a reboot since the stack will have switches with version mismatch?

I'm not too concerned about the switches isolating themselves while there is a version mismatch.  The thinking is that once they all reboot with the same version the stack will reestablish itself. 

We thought we were designing our network that was redundant even during ios upgrades; however, maybe we need to re-evaluate the need for the stack function and just have redundant, non-stacked switches so that upgrading one does not affect the backup switch path.

Thanks for your input!

[Leo Laohoo]

We thought we were designing our network that was redundant even during ios upgrades; however, maybe we need to re-evaluate the need for the stack function and just have redundant, non-stacked switches so that upgrading one does not affect the backup switch path.

It'll cost more.  

The only system I can think of that will provide minimal hit would be a dual-supervisor on a chassis-based system, like 4500R+E, 6500E or 6807.  

[MitchellAxtell]

Even with those, it's not always possible to update without downtime.  For certain upgrades, updating the ROMMON is necessary, and that takes down both supervisors.

Separate switches are the more guaranteed method.

[David Dobbs]

I agree, separate switches Etherchanneled together would be cheaper solution and work in this situation, just lose the stack features/bandwidth.

[David Dobbs]

I figured out a way to upgrade the IOS on a 3850 stack while minimizing down time--assuming Etherchannels are being used between 3850's in a stack for device HA--doesn't apply if attached devices are not Etherchanneled, then they will be down entire reboot/upgrade cycle of the switch.

So instead of the long wait, the only downtime is the short time it takes Etherchannel to converge during state changes.

The attached file contains the method.

This method is specifically for a two switch stack; however, the general concept could apply to stacks with more than two switches.

The method boils down to:

1. Loading the new code on each switch

2. Configuring boot system to load new code

3. Powering down the 2nd (non-active) switch

4. Disconnecting the switch from the stack

5. Powering up the 2nd now isolated switch (upgrades)

6. Reloading the 1st, active switch (upgrades)

7. Power down 2nd switch and reconnect it to the stack

8. Boot up 2nd switch

The key is powering down and physically isolating the switches so that a stack reload doesn't happen, and letting Etherchannel handle moving between switches--fault tolerance.