3850 stacking behavior

[Kyle Morrison]

I had a disastrous IOS upgrade on one stack and I'm researching how to avoid that for the rest of my switches. Last week I noticed that the primary switch was in install mode and the rest in bundle. Today, I have two (so far) different outputs for this same switch.

Switch Ports Model SW Version SW Image Mode
------ ----- ----- ---------- ---------- ----
* 1 56 WS-C3850-48P BUNDLE UNKNOWN IMAGE NAME UNKNOWN INSTALL
2 56 WS-C3850-48P BUNDLE UNKNOWN IMAGE NAME UNKNOWN INSTALL
3 56 WS-C3850-48P BUNDLE UNKNOWN IMAGE NAME UNKNOWN INSTALL
4 56 WS-C3850-48P BUNDLE UNKNOWN IMAGE NAME UNKNOWN BUNDLE
5 56 WS-C3850-48P BUNDLE UNKNOWN IMAGE NAME UNKNOWN BUNDLE

AND

Switch Ports Model SW Version SW Image Mode
------ ----- ----- ---------- ---------- ----
* 1 56 WS-C3850-48P BUNDLE UNKNOWN IMAGE NAME UNKNOWN BUNDLE
2 56 WS-C3850-48P BUNDLE UNKNOWN IMAGE NAME UNKNOWN BUNDLE
3 56 WS-C3850-48P BUNDLE UNKNOWN IMAGE NAME UNKNOWN BUNDLE
4 56 WS-C3850-48P BUNDLE UNKNOWN IMAGE NAME UNKNOWN BUNDLE
5 56 WS-C3850-48P BUNDLE UNKNOWN IMAGE NAME UNKNOWN BUNDLE

I also get this:

spo-esb120-s1#sh ver running
%% Failed to retrieve running packages information for switch 1

%% Failed to retrieve running packages information for switch 2

%% Failed to retrieve running packages information for switch 3

%% Failed to retrieve running packages information for switch 4

%% Failed to retrieve running packages information for switch 5

spo-esb120-s1#dir flash:
Directory of flash:/

30978  -rwx     2097152  Jul 30 2018 00:03:14 +00:00  nvram_config
30979  -rw-   320405504   Aug 5 2018 18:28:26 +00:00  cat3k_caa-universalk9.SPA.03.07.05.E.152-3.E5.bin
30991  -rw-        1237   Oct 6 2016 01:21:50 +00:00  packages.conf
30986  -rwx         736  Jun 17 2018 21:16:22 +00:00  vlan.dat
   11  drwx        4096   Feb 2 2014 14:21:46 +00:00  mnt
92931  -rw-    83297376   Oct 6 2016 01:21:44 +00:00  cat3k_caa-base.SPA.03.06.05E.pkg
92932  -rw-     4012732   Oct 6 2016 01:21:44 +00:00  cat3k_caa-drivers.SPA.03.06.05E.pkg
92933  -rw-    33786860   Oct 6 2016 01:21:44 +00:00  cat3k_caa-infra.SPA.03.06.05E.pkg
92934  -rw-    43048256   Oct 6 2016 01:21:44 +00:00  cat3k_caa-iosd-universalk9.SPA.152-2.E5.pkg
92935  -rw-    28343180   Oct 6 2016 01:21:44 +00:00  cat3k_caa-platform.SPA.03.06.05E.pkg
92936  -rw-   111260480   Oct 6 2016 01:21:45 +00:00  cat3k_caa-wcm.SPA.10.2.150.0.pkg
30989  drwx        4096   Oct 6 2016 01:48:46 +00:00  dc_profile_dir

 The switch is supposedly running 03.06.05E from a packages.conf file. I am trying to upgrade to 03.07.05E.

Has anyone seen this changing between install and bundle seemingly by magic?

[Leo Laohoo]

Try this:
1. Copy the BIN file to each stack member.
2. Run the Install mode command on a per-stack-member at the time. (Install the BIN file to ONLY one stack member at a time).

[Kyle Morrison]

Would adding the .bin file to every switch keep it from (seemingly) switching between bundle and install mode? I don't know what mechanism (if any) there is for the stack manager to monitor or modify stack members.

I have to come up with a very detailed plan before I do any more upgrades. Personally, my method would be to

1. physically unstack the switches

2. get every switch in standalone mode running bundle software

3. plug the stack cables back in and be sure every switch joins the stack in bundle mode.

I think that is what you are suggesting sort of. Caveat to be sure that only one member at a time has the .bin file.

That is what I finally had to do with the orginal stack upgrade failure. On this one switch with the roaming install/bundle, I can't do a show ver running. It sits there for a long time then says it can't communicate. If I try show ver run | in Switch2 or Switch3, it returns nothing.

[Ben Samayoa]

Hello there!

Based on my own experience I have always done the IOS upgrades individually. In the end i have avoided any issues and many times weird ones. I would to yours same way. Via CLI. After doing the upgrade put it back into production. 

Best regards

Ben

If helpful please rate it.

[Kyle Morrison]

That does seem the least tricky option. The head office disagrees and thinks we should do the upgrades the way the Cisco documentation says.  Perhaps we can do that if I have a stack that has some consistency already. This particular stack is in too weird of a state to try and get the stack upgrade to work, in my opinion.

In the show ver the bundle is unknown, the image name is unknown and the mode keeps jumping between install and bundle.

[Leo Laohoo]

---

@Kyle Morrison wrote:

The head office disagrees and thinks we should do the upgrades the way the Cisco documentation says.

---

LOL!

[Ben Samayoa]

Hahahaha! I do understand totally!

The reality is different in so many cases. I am blessed to have been exposed to so many different types of scenarios where what s written in the books is not necessarily correct -  due to unforeseen issues - still to be documented.

regards,

Ben 

[Kyle Morrison]

I worked in the Cisco TAC for ten years. I know everything Cisco says is 100% correct!

[paul driver]

---

@Kyle Morrison wrote:

. The head office disagrees and thinks we should do the upgrades the way the Cisco documentation says

---

I would then do as they suggested as long as they take responsibility if it fails and cause unesessary outage

[Mohsin Alam]

I’ve seen this output on older code versions such as yours before. It doesn’t actually cause any impact and will be fixed upon reload.