Using the Auto-Upgrade feature on the cisco Catalyst 3850

Richard Primm · ‎01-26-2014

Introduction

There are a number of scenarios when you would need to introduce a new switch into an existing stack of 3850 switches. One particular example is when a new switch is purchased to expand the number of usable ports in the stack. To successfully add a new switch to a stack, you will need to ensure that the same software version is running on the new switch. Prior to IOS-XE release 3.3.1, the only way to ensure the versions matched was to stage the new switch prior to introducing it into the stack.

Prerequisites

The 3850, includes a feature called “Auto-Upgrade” which might be familiar if you are migrating from the 3750 series platform. The goal of this feature is to ensure that any newly added switch will automatically be provisioned by the stack members with the running IOS-XE version. One caveat to note however is that prior to 3.3.1, this feature did not work (Tracked by defect CSCui69999 ), and so you must ensure that your existing stack is running at least 3.3.1 in INSTALL mode.

** Auto-upgrade is disabled by default.

** Auto-upgrade is not available in bundle mode.

To utilize the “Auto-Upgrade” feature, you simply need to add the command “software auto-upgrade enable” in the running configuration of the existing stack. This will ensure that any newly added stack members will automatically be upgraded.

Configure

Once the switch is stacked and booted up, you will see a clear indication that there is a mismatch in versions and the new member will not fully join the stack. If you watch the syslog as the switch is trying to join, you will notice the “auto advise” feature will alert you that the newly added switch is running a different software version and mode (INSTALL & BUNDLE explained here). For this example, my new switch is running IOS-XE version 3.2.2 in BUNDLE mode.

%STACKMGR-1-STACK_LINK_CHANGE: STANDBY: 1 stack-mgr: Stack port 2 on switch 1 is up (3850-Stack-1)

%STACKMGR-1-STACK_LINK_CHANGE: 2 stack-mgr: Stack port 1 on switch 2 is up

%STACKMGR-6-SWITCH_ADDED: 2 stack-mgr: Switch 3 has been added to the stack.

%STACKMGR-6-SWITCH_ADDED: STANDBY:1 stack-mgr: Switch 3 has been added to the stack. (3850-Stack-1)

%INSTALLER-6-AUTO_ADVISE_SW_INITIATED: 2 installer: Auto advise initiated for switch 3

%INSTALLER-6-AUTO_ADVISE_SW: 2 installer: Switch 3 running bundled software has been added

%INSTALLER-6-AUTO_ADVISE_SW: 2 installer: to the stack that is running installed software.

%INSTALLER-6-AUTO_ADVISE_SW: 2 installer: The 'software auto-upgrade' command can be used to

%INSTALLER-6-AUTO_ADVISE_SW: 2 installer: convert switch 3 to the installed running mode by

%INSTALLER-6-AUTO_ADVISE_SW: 2 installer: installing its running software.

Once the newly joined member is fully booted, you can see that a mismatch has been detected.

3850-Stack#show switch

Switch/Stack Mac Address : 0c27.24cf.ab80 - Local Mac Address

Mac persistency wait time: Indefinite

H/W Current

Switch# Role Mac Address Priority Version State

------------------------------------------------------------

*1 Active 0c27.24cf.ab80 14 B0 Ready

2 Standby f41f.c238.a800 13 B0 Ready

3 Member b4e9.b0d3.6600 12 0 V-Mismatch

Enable the auto-upgrade feature

In global configuration mode, type the command “software auto-upgrade enable.” This will enable the feature for any new switches that join the stack.

3850-Stack(config)#software auto-upgrade enable

3850-Stack(config)#end

You will now have to reload the newly added switch only, a full stack reload is not necessary. In my case, the newly added switch is “switch 3” so I simply type “reload slot 3.” Remember that “slot” designates the switch in the stack (slot 1 = switch 1)

3850-Stack#reload slot 3

Proceed with reload? [confirm]

%STACKMGR-1-RELOAD_REQUEST: 1 stack-mgr: Received reload request for switch 3, reason Reload Slot Command

%STACKMGR-1-STACK_LINK_CHANGE: 1 stack-mgr: Stack port 2 on switch 1 is down

%STACKMGR-6-SWITCH_REMOVED: 1 stack-mgr: Switch 3 has been removed from the stack.

%STACKMGR-1-STACK_LINK_CHANGE: STANDBY:2 stack-mgr: Stack port 1 on switch 2 is down (3850-Stack-2)

Starting SWITCH-DELETE sequence, switch 3

SWITCH-DELETE sequence complete, switch 3

%STACKMGR-6-SWITCH_REMOVED: STANDBY:2 stack-mgr: Switch 3 has been removed from the stack. (3850-Stack-2)

Starting SWITCH-DELETE sequence, switch 3 (3850-Stack-2)

SWITCH-DELETE sequence complete, switch 3 (3850-Stack-2)

Wait a few minutes while the switch reloads in the background

%STACKMGR-1-STACK_LINK_CHANGE: 1 stack-mgr: Stack port 2 on switch 1 is up

3850-Stack#

%STACKMGR-1-STACK_LINK_CHANGE: STANDBY:2 stack-mgr: Stack port 1 on switch 2 is up (3850-Stack-2)

3850-Stack#

%STACKMGR-6-SWITCH_ADDED: 1 stack-mgr: Switch 3 has been added to the stack.

%STACKMGR-6-SWITCH_ADDED: STANDBY:2 stack-mgr: Switch 3 has been added to the stack. (3850-Stack-2)

3850-Stack#

First we can see that it does the conversion from BUNDLE to INSTALL mode followed by a reload.

%INSTALLER-6-AUTO_UPGRADE_SW_INITIATED: 1 installer: Auto upgrade initiated for switch 3

%INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Converting switch 3 to installed mode by

%INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: installing its running software

%INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Setting the boot var on switch 3

%INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Finished installing the running software on switch 3

%INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Reloading switch 3 to boot in installed mode

%STACKMGR-1-RELOAD_REQUEST: 1 stack-mgr: Received reload request for switch 3, reason Auto upgrade

%STACKMGR-1-STACK_LINK_CHANGE: 1 stack-mgr: Stack port 2 on switch 1 is down

%STACKMGR-6-SWITCH_REMOVED: 1 stack-mgr: Switch 3 has been r

3850-Stack#emoved from the stack.

%STACKMGR-1-STACK_LINK_CHANGE: STANDBY:2 stack-mgr: Stack port 1 on switch 2 is down (3850-Stack-2)

Starting SWITCH-DELETE sequence, switch 3

SWITCH-DELETE sequence complete, switch 3

%STACKMGR-6-SWITCH_REMOVED: STANDBY:2 stack-mgr: Switch 3 has been removed from the stack. (3850-Stack-2)

3850-Stack#

Starting SWITCH-DELETE sequence, switch 3 (3850-Stack-2)

SWITCH-DELETE sequence complete, switch 3 (3850-Stack-2)

Once it reboots, the upgrade will then continue

3850-Stack#

%INSTALLER-6-AUTO_UPGRADE_SW_INITIATED: 1 installer: Auto upgrade initiated for switch 3

%INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Searching stack for software to upgrade switch 3

%INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Found donor switch 1 to auto upgrade switch 3

%INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Upgrading switch 3 with software from switch 1

%INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Finished installing software on switch 3

%INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Reloading switch 3 to complete the auto upgrade

%STACKMGR-1-RELOAD_REQUEST: 1 stack-mgr: Received reload request for switch 3, reason Auto upgrade

%STACKMGR-1-STACK_LINK_CHANGE: 1 stack-mgr: Stack port 2 on switch 1 is down

%STACKMGR-6-SWITCH_REMOVED: 1 stack-mgr: Switch 3 has been removed from the stack.

%STACKMGR-1-STACK_LINK_CHANGE: STANDBY:2 stack-mgr: Stack por

3850-Stack#t 1 on switch 2 is down (3850-Stack-2)

Starting SWITCH-DELETE sequence, switch 3

SWITCH-DELETE sequence complete, switch 3

%STACKMGR-6-SWITCH_REMOVED: STANDBY:2 stack-mgr: Switch 3 has been removed from the stack. (3850-Stack-2)

Another reload is automatically performed. Once the switch is booted back up, it will successfully join the stack with the correct IOS-XE version and software mode.

%STACKMGR-6-SWITCH_ADDED: 1 stack-mgr: Switch 3 has been added to the stack.

%STACKMGR-6-SWITCH_ADDED: STANDBY:2 stack-mgr: Switch 3 has been added to the stack. (3850-Stack-2)

%STACKMGR-6-SWITCH_READY: STANDBY:2 stack-mgr: Switch 3 is ready. (3850-Stack-2)

%STACKMGR-6-SWITCH_READY: 1 stack-mgr: Switch 3 is ready.

Starting SWITCH-ADD sequence, switch 3

%NGWC_USB_CONSOLE-6-CONFIG_ENABLE: Switch 3: Console media-type changed to default

Starting SWITCH-ADD sequence, switch 3 (3850-Stack-2)

SWITCH-ADD sequence complete, switch 3 (3850-Stack-2)

SWITCH-ADD sequence complete, switch 3

Verify

3850-Stack#show switch

Switch/Stack Mac Address : 0c27.24cf.ab80 - Local Mac Address

Mac persistency wait time: Indefinite

H/W Current

Switch# Role Mac Address Priority Version State

------------------------------------------------------------

*1 Active 0c27.24cf.ab80 14 B0 Ready

2 Standby f41f.c238.a800 13 B0 Ready

3 Member b4e9.b0d3.6600 12 B0 Ready

3850-Stack#show version

Switch Ports Model SW Version SW Image Mode

------ ----- ----- ---------- ---------- ----

* 1 56 WS-C3850-48P 03.03.01SE cat3k_caa-universalk9 INSTALL

2 56 WS-C3850-48P 03.03.01SE cat3k_caa-universalk9 INSTALL

3 56 WS-C3850-48P 03.03.01SE cat3k_caa-universalk9 INSTALL

Reference:

http://www.cisco.com/en/US/docs/switches/lan/catalyst3850/software/release/3se/consolidated_guide/configuration_guide/b_consolidated_3850_3se_cg_chapter_01101101.html#con_1189959

rmkoperdraad · ‎03-06-2014

What happens if the newly added switch has a newer version? will it then downgrade the new switch?

walter baziuk · ‎12-14-2014

does it work with 3750

or 3750x?

aducey01 · ‎01-31-2017

I have the same question. I use 3850's in a 911 Communications Center environment and I really need a solid answer to this from Cisco. From the documentation, it looks as though any switch with whatever code that's incompatible with the stack gets upgraded or downgraded to what's on the stack and only the new switch will be rebooted....Is that true? Can I see documentation? I don't want to see random answers telling not to use auto-anything...that's not helpful.

It's a little disconcerting that there aren't any answers to the above question as it was posted 3 years ago.

-anne

Brian.Forster · ‎05-09-2017

Looks like it will do either.

http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst3750/software/troubleshooting/switch_stacks.html

"If the stack master detects that the software on a new switch does not match, it upgrades (or downgrades) the new switch with the active image that is running on the switch stack by using the automatic upgrade (auto-upgrade) and the automatic advise (auto-advise) features."

nathanielscriven · ‎05-20-2017

Ok so why in the hell does Cisco not include the 3.3.0 issue on any of the 3650 documentation? I researched the auto upgrade feature before working in a customer switch into a new 3650 stack and it crashed the entire damn stack. It'd be nice if the documentation I rely on for uptime and seamless implementation actually included this incredibly important detail.

http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst3650/software/release/3se/system_management/configuration_guide/b_sm_3se_3650_cg/b_sm_3se_3650_cg_chapter_010101.html

yuzhengzhu1 · ‎01-19-2018

I ran into a scenario where two C3850 switches formed a stack, despite having different IOS versions. I'm curious as to why this happened. Has anyone experienced anything like it before?

JBH-4A-Stack3#sh switch
Switch/Stack Mac Address : 38ed.188f.8a80 - Local Mac Address
Mac persistency wait time: Indefinite
H/W Current
Switch# Role Mac Address Priority Version State
------------------------------------------------------------
*1 Active 38ed.188f.8a80 15 V05 Ready
2 Standby 00a2.8954.4c80 14 V07 Ready

JBH-4A-Stack3#sh ver
Cisco IOS Software, IOS-XE Software, Catalyst L3 Switch Software (CAT3K_CAA-UNIVERSALK9-M), Version 03.06.06E RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2016 by Cisco Systems, Inc.
Compiled Sat 17-Dec-16 00:22 by prod_rel_team

Cisco IOS-XE software, Copyright (c) 2005-2015 by cisco Systems, Inc.
All rights reserved. Certain components of Cisco IOS-XE software are
licensed under the GNU General Public License ("GPL") Version 2.0. The
software code licensed under GPL Version 2.0 is free software that comes
with ABSOLUTELY NO WARRANTY. You can redistribute and/or modify such
GPL code under the terms of GPL Version 2.0.
(http://www.gnu.org/licenses/gpl-2.0.html) For more details, see the
documentation or "License Notice" file accompanying the IOS-XE software,
or the applicable URL provided on the flyer accompanying the IOS-XE
software.

ROM: IOS-XE ROMMON
BOOTLDR: CAT3K_CAA Boot Loader (CAT3K_CAA-HBOOT-M) Version 1.18, RELEASE SOFTWARE (P)

JBH-4A-Stack3 uptime is 40 weeks, 1 day, 29 minutes
Uptime for this control processor is 40 weeks, 1 day, 35 minutes
System returned to ROM by reload
System restarted at 16:09:04 EDT Thu Apr 13 2017
System image file is "flash:cat3k_caa-universalk9.SPA.03.06.06.E.152-2.E6.conf"
Last reload reason: Reload command

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

License Level: Lanbase
License Type: Permanent
Next reload license Level: Lanbase

cisco WS-C3850-48P (MIPS) processor with 4194304K bytes of physical memory.
Processor board ID FCW1927D1LJ
2 Virtual Ethernet interfaces
104 Gigabit Ethernet interfaces
8 Ten Gigabit Ethernet interfaces
2048K bytes of non-volatile configuration memory.
4194304K bytes of physical memory.
250456K bytes of Crash Files at crashinfo:.
252000K bytes of Crash Files at crashinfo-2:.
1609272K bytes of Flash at flash:.
1611414K bytes of Flash at flash-2:.
0K bytes of Dummy USB Flash at usbflash0:.
0K bytes of Dummy USB Flash at usbflash0-2:.
0K bytes of at webui:.

Base Ethernet MAC Address : 38:ed:18:8f:8a:80
Motherboard Assembly Number : 73-15800-07
Motherboard Serial Number : FOC192775WR
Model Revision Number : T0
Motherboard Revision Number : A0
Model Number : WS-C3850-48P
System Serial Number : FCW1927D1LJ

Switch Ports Model SW Version SW Image Mode
------ ----- ----- ---------- ---------- ----
* 1 56 WS-C3850-48P 03.06.06E cat3k_caa-universalk9 INSTALL
2 56 WS-C3850-48P 03.06.05E cat3k_caa-universalk9 INSTALL

Switch 02
---------
Switch uptime : 29 weeks, 2 days, 1 hour, 5 minutes
Base Ethernet MAC Address : 00:a2:89:54:4c:80
Motherboard Assembly Number : 73-15800-08
Motherboard Serial Number : FOC20270LZ7
Model Revision Number : AB0
Motherboard Revision Number : A0
Model Number : WS-C3850-48P
System Serial Number : FCW2027C1HA

Configuration register is 0x102

helalbd_net · ‎12-02-2019

This is helpful. I had the similar Issue and it is fixed.