Can I upgrade/reload one switch at a time in a stack?

[rweir0001]

I have two Cisco WS-C3750X-48T-S switches in a stack and I want to upgrade the IOS. The problem is that these switches are used for our SAN so I can't reboot them at the same time. I don't want to crash the SAN. The two switches are configured for redundancy, so if one switch goes down the other one will still pass traffic to the SAN. I know how to upgrade the IOS but I always upgrade the stacked switches at the same time and do a reload. Can I just upgrade the IOS on one switch and reload it, before upgrading the second switch in the stack and reloading it? I know it will cause a version mismatch, but will it cause other issues? Seeing they are stacked, but configured for redundancy, should I disconnect the stacking cable and upgrade one switch at a time? I'd appreciate if someone could reply with steps I should take for upgrading/reloading one switch at a time in a stack, keeping in mind that it would be really bad if I crash storage.

Thanks!

[Bilal Nawaz]

It's advised not to take out stacking cables.

1) Ensure the IOS image is on all switches in the stack.

2) Change the boot statement

3) Save the config

4) Use the "reload slot #" command where # is the switch number in order to reboot one member at a time

[rweir0001]

Thanks, Bilal. How will that affect stacking? Will the two switches in the stack still communicate? How long should I leave the IOS versions mismatched? I'm assuming not very long, right? I'm just curious because I would like to test the new IOS to see if it is working correctly before before upgrading the other switch. 24 hours would be ideal, but probably not practical. I'm just being a little paranoid because taking down the SAN would be a huge problem. 

[Paul Chapman · Accepted solution]

Hi -

If the stack software version changes, then your stack will become partitioned when the first switch reboots.  (i.e. You will have 2 stacks of 1 switch and the stack links will be disabled)

This means that everything for your SAN connections must be split at the switch level, since the stack link won't work for a period of time while you are going through this procedure.

I would recommend completing the upgrade by rebooting the second switch as soon as you have confirmed storage access through all paths.

You will need to isolate any IP addresses on the switches (management or otherwise) to prevent IP conflicts after the first switch reboots.  With this in mind, I strongly recommend that you have physical access to the switches during this upgrade process.

Due to the complexity of this change, I recommend that you take an outage window and shut down the VMs or servers that rely on this storage system and simply upgrade both switches simultaneously.

PSC

[rweir0001]

Paul,

Thanks! This is great information but what do you mean by 'You will need to isolate any IP addresses on the switches (management or otherwise) to prevent IP conflicts after the first switch reboots". Can you give me a little more detail or maybe an example?

Rick

[Paul Chapman]

Hi -

When the stack is in a partitioned state, both switches will be "master" for the configuration that is in them.  The result is both will respond to ARP requests for any IP addresses in the configuration, hence IP conflict.  They will both have the same IP addresses.

PSC

[rweir0001]

So when you say to isolate the IP addresses on the switches do you mean that I should unplug one of the switches from the network, and upgrade it through the console?

[Paul Chapman]

Hi -

You can pre-load the IOS files to both switches first.  When you get to your outage window, you will need to disconnect or disable the uplinks on one of them to prevent IP conflicts.

PSC

[rweir0001]

Thanks! I will look into my options further. I would like to do what you said and and shut down the servers using the storage so I can upgrade both the switches at the same time but that might be a hard sell.

[Paul Chapman]

Hi -

Saving time is not my point.  The problem is that you're running in dangerous territory here.  Any global changes on this stack may cause a disconnect to the storage resulting in server failures.  My suggestion for shutting down critical systems is just a protective measure to try and prevent data loss.

If you have expertise in networking and you understand the ramifications of your change, then go for it.  If you have any doubts be conservative.

If the switches weren't stacked at all, then this wouldn't be much of a discussion.  You can do the upgrade with a stack, but it's very procedural work.

PSC

[JACKSu77777]

FFFF,I get a boom after trying your plan