Upgrading FWSM software and maintaining failover

[CSCO10576352]

Hi all, whilst researching the procedure to upgrade the software on an active/standby FWSM pair I read the below extract in the "Catalyst 6500 Series Switch and Cisco 7600 Series Router Firewall Services Module Configuration Guide, 3.2" (http://www.cisco.com/en/US/docs/security/fwsm/fwsm32/configuration/guide/swcnfg_f.html#wp1047472).

Upgrading Failover Pairs to a New Minor or Major Release

<.....>
While the units reload, all active connections are terminated. We recommend reloading both units at the same time because if both units are running, and the major or minor version number does not match (3.1 vs. 3.2), then both units become active. Two active units can cause networking problems.

The intention is to upgrade from  from 3.2(3) to 4.1(15) (so going to a new major version of code), and to do so from the FWSM CLI (not the maintenance partition)

The above essentially means taking both FWSM's out of service at the same time for the time it takes them to reload. This unfortunatley in our scenario would have quite drastic knock on effects for the solution in the application/host space were even a 5 minute downtime in the network would be problamatic.

My question is am I reading/interpretting this correctly?

In our lab environment I replicated the FWSM failover configuration between the two versions of code, my expectation was that both modules would become active, this did not occur at least in the output of the show failover command, the active/standby relationship appeared to be maintained but periodicaly the below warning was outputted to the console:

************WARNING****WARNING****WARNING********************************
   Mate version 4.1(15) is not identical with ours 3.2(3)
************WARNING****WARNING****WARNING*********************************

The lab allows me to test the basic FWSM operation but not the production traffic flows from a host/service persepctive but the above appears to contradict Cisco's own recomendationpn/procedure.

Thanks for taking the time to read through and any comments you may wish to share.

[CSCO10576352]

Doing some further searching on this forum I have found the below post which is interesting and possibly explains my query above:

https://supportforums.cisco.com/discussion/12047466/upgrading-fwsm-version-4x

Of particular interest is the below post by dinsharm:

Agree with Itzcoatl, Zero downtime is possible. Zero downtime was a problem with Old versions 2.x/3.1. But Starting from 3.2 onwards its posible. A Doc Bug was filed to correct the documentation CSCtr63007.

This would appear to explain why i see the warnings detecting the different versions of code but failover is maintained given we are currently running 3.2(3).

[keven.lincoln]

I would not have thought that it is possible to have a zero down time  cluster upgrade. Its a bit of a play on words. You can have virtually zero down time with respect to handling new incoming connections. However , you cannot have zero down time for existing connections. The problem I believe is that you cannot carry out state synchronisation between two devices with different software build levels. This is usually the case for major release versions but there may be exclusions for minor software release differences but not in my experience. . You can split the cluster and have one cluster member active while you upgrade the other. No downtime.  The down time comes when you bring the upgraded cluster member back on line. It cannot do a state sync with the device that has been the active member because of the software build difference. You loose existing connections. When you then upgrade the other cluster member there isn't any down time because when you re-establish the cluster they are at the same build level and state sync ok . So no down time for new connections ( or just a few seconds as you flip over) , but not for existing connections because you cannot do state sync between different software builds.