topic Re: High Availability Failure During Update in Network Security

High Availability Failure During Update

TCSPB — Fri, 19 Nov 2021 19:28:19 GMT

In the middle of updating a pair of FTD 1010's from 6.6.4 to 6.6.5, the upgrade failed due to the HA links going down for some reason during the upgrade. We use a port channel for HA and it was all down. When checking the CLI, I noticed that it showed the port channel interface as down and the physical ports as unassociated and admin down.

After many hours on the phone with Cisco TAC, I am still unable to get HA to function on these devices. HA was first configured to use a port channel, but I broke HA and reconfigured the HA link to be a single link. HA still will not come up and the ports just show as down. Cabling has been verified and the ports have link lights, it just does not show up in the firewall.

Has anyone here run into a similar issue on these when doing an update on a pair of HA 1010's like this?

Re: High Availability Failure During Update

balaji.bandi — Sat, 20 Nov 2021 00:40:47 GMT

I have used different models never come across this issue,

Can you draw some diagram of how these are connected if this is going via switch ? have you checked on the switch? do you see any Logs?

You mentioned you resolved using a single link ? is this interface part of the port-channel going to the same hardware switch? how about another link?

TAC is the special expertise, we can only advise based on our experience, they might have seen many cases may be aware of any bug? so TAC is not able to provide any feedback?

Re: High Availability Failure During Update

TCSPB — Sat, 20 Nov 2021 04:18:24 GMT

There is no switch in between the firewalls. Ports 7 & 8 were configured into an portchannel for HA. This configuration worked until the attempted upgrade and then stopped working. Odd thing was there were link lights on port 7 even though the ports were showing as down on the firewall.

Next week I'm getting someone on-site again so I can do some tests on the ports. This is the second pair of 1010's that we've had this happen to, so I was just wondering if anyone else had this happening.

Re: High Availability Failure During Update

balaji.bandi — Sat, 20 Nov 2021 06:47:31 GMT

You mean the FW connected back to back ?

Re: High Availability Failure During Update

Mohammed al Baqari — Sat, 20 Nov 2021 12:08:02 GMT

Hi,

I faced similar case. The fast recovery is to break HA, reimage the
secondary unit then create new HA. This will replicate the existing config
to 2nd pair.

As you know, when you break HA you can retain the existing config on one of
the unit that you want as primary.

I spent hours with TAC without proper resolution and reimage takes half the
time taken by TAC.

***** please remember to rate useful posts

Re: High Availability Failure During Update

TCSPB — Sat, 20 Nov 2021 23:45:28 GMT

Yes the firewalls are directly connected. Ports 7 & 8 were configured as a port channel for HA. From what we could see in FXOS, the ports were operational, but just showed as down.

Breaking HA and rebuilding using both the original port channel and also just port 7 as standalone did not fix this. Since this firewall is in a remote office, I'm working to setup someone local to work with on this.

Re: High Availability Failure During Update

Michael Bartholomæussen — Tue, 13 Feb 2024 13:00:14 GMT

We face the same issue on 7.3.1.1 to 7.4.1 - any solution or fix for this. Also on a pair on 1010's!

Re: High Availability Failure During Update

balaji.bandi — Tue, 13 Feb 2024 18:12:33 GMT

what is the need to of the Upgrade ? explain more about the issue. what error you getting ?

Re: High Availability Failure During Update

Michael Bartholomæussen — Tue, 13 Feb 2024 19:15:33 GMT

The upgrade is moving the code to stable train.
It's a regular HA pair upgrade to 7.4.1.1. FTD 1010
After upgrade of the first FTD, the port channel for the FO link doesn't come up. Then the HA is split and the upgrade halts.

Re: High Availability Failure During Update

balaji.bandi — Wed, 14 Feb 2024 00:27:05 GMT

if this managed by FMC ? i have not not seen upgrade any issue until 7.3.

Generally standby upgrades first in HA and Primary - we should not see any traffic interruption other than couple of ping loss.

worth raising TAC case.