Cisco 9200L stacking issues (adding/intial creation)

[Jochem Sparla]

I am having issues with initial creation of a stack of Cisco 9200L switches.

Previously, when using 2960-X switches, I used to follow the following procedure when building a stack from brand new switches:
- physically build complete stack (connect all stack modules and cables)
- power on first switch, let it boot completely; it becomes switch #1 and master
- power on second switch, let it boot completely; it becomes switch #2
- and so on for switches 3 to 8 (or less)
- set priority on switches for master election
- enter config for whole stack

I have even powered switches 2..8 with an 30 second interval (or possible even all at the same time), and everything would be fine. Worst case, I would have to renumber some switches when one would boot faster than the other.
It would all work fine.

However, now I am using 9200L switches. I use my 'safest' method (boot 1st, then 2nd, and so on, nothing simultaneously), but I run into problems. On multiple occasions, even the 2nd switch would not join the stack.

I see the stack port come up, and then the new switch reboots. After it's reboot, it will not join the stack. All stack modules and cables are recognized by the switch (as seen in 'show inventory'), but they say they are not connected. So I have two switches connected as stack, that are not working as a stack.

Or, I get strange reboot behavior, in which various switches in the stack that is being created, are rebooting. Not with clear messages in the CLI. This created boot loops, to the extend that the switched boot into the 'safe mode' ("switch:" prompt).

All switches are brand new, out of the box, running IOS 17.06.03.

I can't figure out what I am doing wrong.

The strangest thing is, that in both occasions, I fixed the problem by upgrading IOS, and started adding the switches again, one by one. (note: with the same modules and cables, so the hardware is fine, and properly connected)
The first project, updated all separate switches to 17.06.05. The second project, I upgraded the first switch to 17.09.04a, enabled 'install autoupgrade' (i enabled this already before), and let the master upgrade all other switches.
Some switches were so messed up by the previous actions, I had to boot them without any stack cables connected, renumber the switch and 'write erase'.

I worked on 5 stacks in my latest project. 3 of them worked fine running 17.06.03, building the initial stack without any problems. 2 of them had the problems. Which takes a lot of time and effort to fix, with the upgrades and endless reboots.

# are there known issues in 17.06.03 with stacking?
# does anyone recognize these issues?
# am I doing anything wrong in the way I initially build stacks? Can I improve anything?
# what is the 'best practice' in building a stack from scratch? and upgrading firmware when needed? (it takes so **bleep** long..)

[pieterh]

my experience is with the 9300(L) series not the 9200L
various IOS versions -> no problem at all when stacking

i understand all out-of-the-box with the same IOS-XE version ?
do they all also have the same mode (install vs bundle) ?

be aware "install autoupgrade" can take a considerable time!
in a larger stack within this time two switches further down the stack can form a stack on it's own with it's own stack-master
I guess this is what frustrated your installation

=> when using "install autoupgrade" you better add switches one-by-one ,
let them join the stack and wait until upgrade has completed before adding the next.

[Jochem Sparla]

Yes, all out-of-the-box, same IOS-XE version 17.06.03.
I did not check the same mode (install vs bundle). I am assuming they are the same, the switches are probably from the same shipment.

I am aware of the time that "install autoupgrade" takes. I only upgraded the firmware after I encountered the problems, in order to fix the problems. And in that case, definitely adding one-by-one.

I have seen (in one occasion) that within a stack, switches 2 to 8 formed there own stack, but that was only after switch 2 failed to add itself to the stack of switch 1. The "2 to 7" stack, then proceeded to have boot loop errors. This was the biggest headache stack, which I had to totally dismantle, reset every switch one-by-one, and re-add to the stack with upgraded firmware switch 1.

But I still don't understand how it happened, and how I can prevent it from happening again..

[pieterh]

without the collected logs at the time this occurred it is only guessing what has happened.

best guess is that one (or more) members did not have the same IOS version, license type (-E or -A) or a blank config