Upgrade OS ASA 5550 from ver 9.1(7)7 to 9.1(7)19 - what went wrong?

[Vaibhav]

Hi Team,

Need your help to troubleshoot OS upgrade ASA 5550 from ver 9.1(7)7 to 9.1(7)19.

Scenario:

Standby was running on old version [9.1(7)7] and was Active.

Primary was standby with latest version [9.1(7)19].

So, what basically I did:

Primary/Stby# failover active

After re-logging the device. version was 9.1(7)7

Primary/Act# show failover state

                  state.                 Last Failure   Reason      Date/Time

This host - Primary

                  Active.               None

Other host - Secondary

                    Standby Ready None

===Configuration state===

       Sync Done - STANDBY

===Communication state===

       Mac set

Primary/act# failover reload-standby

Primary/act# show failover state

                 state.                 Last Failure   Reason      Date/Time

This host - Primary

                  Active.               None

Other host - Secondary

                    Failed              Comm Failure.                 hr:mn:sec  EST MM DD YYYY

===Configuration state===

       Sync Done - STANDBY

===Communication state===

Now, Secondary Standby device is un-rechable. Please help troubleshooting. Seems like I don't have console access to secondary device also. Right now, Primary is Active with latest version. Please help urgently, I need to do 8 more fw upgreade but something I missed in hurry :-(

[Philip D'Ath]

All we can see is it has a "Comm Failure".  Perhaps try "failover reload-standby" again.

You could also power cycling the standby unit and see what happens.

[EricFG]

Hi, only to add some stone to the wall and help to find a solution.

Upgrading ASA Active/Passive cluster from 9.1(6)11 to 9.1(7)21 and after that 9.1(7)21 to 9.1(7)23.

During these upgrades, when rebooting the Standby with the new version it still reboot again and again when the active saw the device. (MALLOC in the console).

Debug:

- removing all cable from the standby device and rebooting it -> it worked

- replugging the FAILOVER cable -> MALLOC reboot

- rebooting the device again without cable -> it worked

- removing the failover configuration and replugging the FAILOVER cable -> it worked

- activating the failover -> MALLOC reboot

Solution:

- rebooting the device in romon with the FAILOVER cable in place -> it worked

- setting the failover configuration -> it worked

=> starting to received the configuration from the active device -> device secondary/standby

My concerns now is if one of the devices reboot, will they reboot in loop or not? Waiting Cisco to answer.

[EricFG]

In regard to Cisco this bug is doing the Malloc reboot:

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvh90947/?reffering_site=dumpcr

We will see when we will upgrade again. The question is why this is not inside the release note.

[EricFG]

All expired certificates have been expelled from the device but we are still experiencing the loop booting. Removing the Java Code Signer certificate will solve the issue.

[zierhut.chris1]

What is the impact, if we remove the Java Code Signer Certificate. For what is this needed?

[EricFG]

It's needed to sign the Java plugging you could use in the web vpn (ssh and rdp plugging for examples). The impact not using a certificate to sign them will be a pop-up window telling the plugging is not signed like https bad certificate in your browser. It's a trusted relationship between the client and the server and many GPO will refuse to accept that. Depend of Company security policy.

[zierhut.chris1]

Any word about a fixed release?

[EricFG]

It will be in May regarding TAC engineer information.