9500 Stackwise Virtual / Recovery mode

[suneq]

Hi,

I have a pair of Catalyst 9500 configured in Stackwise Virtual mode. The IOS version is 17.03.03.

I did a failover test in which I reloaded the active switch by using the command "redundancy force-switchover".

From what I read in the white paper https://www.cisco.com/c/en/us/products/collateral/switches/catalyst-9000/nb-06-cat-9k-stack-wp-cte-en.html#ReloadingtheStackWiseVirtualdomainanditsmembers

I understood that the switch reloaded will recover automatically and becomes the standby switch.

However, after a few tests, I noticed that the switch reloaded is always stuck in recovery mode and I had to reload it

manually before it can be seen again from the new active.

Maybe I miss something? Thanks for your help.

[Reza Sharifi]

Hi,

I understood that the switch reloaded will recover automatically and becomes the standby switch.

That is correct. Once you issue this command, it will make the backup switch primary and reboot the old primary switch.

What is the output of "sh redundancy" once the other switch boots?

HTH

[suneq]

Hi Reza, thanks for your confirmation.

I did a new test today, for simplicity's sake, let say that I have switch A and switch B, switch A is active, switch B is standby.

1. I reloaded the switch B: everything went fine, B rejoined the stack after the reboot. A is still active, B is standby

2. I reloaded the switch A: B became active, after the reboot A was stuck on recovery mode because both SVL ports were down, the DAD port was up. On the switch B, I can see the same: DAD port is up, both SVL ports were down

3. I reloaded the switch A again: same behavior, it was stuck on recovery mode.

4. I disconnected the DAD and SVL ports and reloaded the switch A again: it managed to boot successfully. I reconnected the DAD and SVL ports, DAD port went up, SVL ports remained down.

So actually, I have switch A and switch B both up, the DAD ports are up, SVL ports are down. They cannot see each other ("show switch"). 

It's a bit complicated. I don't know what I missed and what I have to do to re-establish the Stackwise Virtual. I do not have access to the DCs, each reboot requires a local guy to move between DCs and he went mad

[Reza Sharifi]

Hi,

It's a bit complicated. I don't know what I missed and what I have to do to re-establish the Stackwise Virtual. I do not have access to the DCs, each reboot requires a local guy to move between DCs and he went mad

I can just see that  

I hate to say this but for this type of install, the engineers who are doing the work usually need to be onsite as you never know what problem you run into and how many times you have to reboot especially when it comes to Cisco equipment. 

Questions: When all is working does "sh redun" command show both switches in SSO mode? Is there a way to post the output of "sh run" and "sh redun"?

HTH

[suneq]

Hi Reza,

Sorry for the late reply, I was off for a few days. We finally understood better what happened: for unknown reasons, when we reloaded the switch A (active one), their SVL ports stayed down after the reboot and it explains why the switch A was stuck in recovery mode. When we reset the SVL ports on switch B side, the SVL ports of switch A went up, the switch A rebooted and joined the Stackwise Virtual.

Why the SVL ports were down, is it a bug, a SFP issue.. that's a different story but now we know that at least the recovery behavior was normal. 

Thanks a lot for your help.