Failover failure

petoria — Fri, 21 Feb 2020 17:31:40 GMT

Hello,

I have a firewall configured active/standy lan failover. Over the weekend there was something that happened and now the standby is active. Below i posted the failover state, for security purposes i just left out the actual ips on some. I know that one reason the failover has failed is due to the firepower module which will only run properly when the primary is active. However, usually i can just make the primary active and the active/standby will go back to normal. I know this is not ideal, but i have zero experience with firepower and i didn't configure anything on these firewalls and limited experience in them as well.

So after this issue happened, this time, both firewalls are showing active based on the active lights on the device itself, which is different than before. usually the primary had the orange led for standby. While troubleshooting this, i found when i console into the secondary firewall, the command line still shows its' hostname as the primary firewall. Is this correct? Is that supposed to happen? Or does it take over the hostname as it does the mac/ip when failover happens?

Another question i have is why would the firepower module fail to load anytime the secondary is active? The logs for the failover history are below as well showing the failure. Possible firepower hardware issue on the secondary maybe?

Failover On
Failover unit Secondary
Failover LAN Interface: FO-1 GigabitEthernet1/8 (up)
Reconnect timeout 0:00:00
Unit Poll frequency 1 seconds, holdtime 15 seconds
Interface Poll frequency 5 seconds, holdtime 25 seconds
Interface Policy 1
Monitored Interfaces 1 of 160 maximum
MAC Address Move Notification Interval not set
Version: Ours 9.9(1), Mate 9.9(1)
Serial Number: Ours JAD20430ECN, Mate Unknown
Last Failover at: 12:48:15 EDT Sep 22 2019
This host: Secondary - Active
Active time: 248367 (sec)
slot 1: ASA5516 hw/sw rev (1.1/9.9(1)) status (Up Sys)
Interface outside (our pub ip): Normal (Not-Monitored)
Interface inside (192.168.1.4): Normal (Waiting)
Interface dmz (dmz ip): Normal (Not-Monitored)
Interface idf-link (idf ip): No Link (Not-Monitored)
slot 2: SFR5516 hw/sw rev (N/A/5.4.1-211) status (Up/Up)
ASA FirePOWER, 5.4.1-211, Up, (Monitored)
slot 2: SFR5516 hw/sw rev (N/A/5.4.1-211) status (Up/Up)
ASA FirePOWER, 5.4.1-211, Up, (Monitored)
Other host: Primary - Failed
Active time: 4391 (sec)
slot 1: ASA5516 hw/sw rev (1.1/9.9(1)) status (Unknown/Unknown)
Interface outside (0.0.0.0): Unknown (Not-Monitored)
Interface inside (192.168.1.5): Unknown (Monitored)
Interface dmz (dmz ip): Unknown (Not-Monitored)
Interface idf-link (idf ip): Unknown (Not-Monitored)
slot 2: SFR5516 hw/sw rev (N/A/5.4.1-211) status (Unknown/Unknown)
ASA FirePOWER, 5.4.1-211, Unknown, (Monitored)
slot 2: SFR5516 hw/sw rev (N/A/5.4.1-211) status (Unknown/Unknown)
ASA FirePOWER, 5.4.1-211, Unknown, (Monitored)

Result of the command: "sh failover history"

==========================================================================
From State To State Reason
==========================================================================
11:37:57 EDT Sep 22 2019
Not Detected Negotiation No Error

11:38:01 EDT Sep 22 2019
Negotiation Cold Standby Detected an Active mate

11:38:02 EDT Sep 22 2019
Cold Standby Sync Config Detected an Active mate

11:38:21 EDT Sep 22 2019
Sync Config Sync File System Detected an Active mate

11:38:21 EDT Sep 22 2019
Sync File System Bulk Sync Detected an Active mate

11:38:37 EDT Sep 22 2019
Bulk Sync Standby Ready Detected an Active mate

11:38:37 EDT Sep 22 2019
Standby Ready Failed Detect service card failure

11:39:16 EDT Sep 22 2019
Failed Standby Ready My service card is as good as peer

11:39:36 EDT Sep 22 2019
Standby Ready Failed Detect service card failure

11:39:40 EDT Sep 22 2019
Failed Standby Ready My service card is as good as peer

12:48:15 EDT Sep 22 2019
Standby Ready Just Active HELLO not heard from mate

12:48:15 EDT Sep 22 2019
Just Active Active Drain HELLO not heard from mate

12:48:15 EDT Sep 22 2019
Active Drain Active Applying Config HELLO not heard from mate

12:48:15 EDT Sep 22 2019
Active Applying Config Active Config Applied HELLO not heard from mate

12:48:15 EDT Sep 22 2019
Active Config Applied Active HELLO not heard from mate

==========================================================================

Re: Failover failure

Marvin Rhoads — Thu, 26 Sep 2019 02:14:02 GMT

It appears you have a "split brain" situation where both ASA units believe they should be in the Active role.

"HELLO not heard from mate" means the mate is offline or the failover link is not communicating the HELLO keepalive messages.

The hostname should replicate between the units. In HA pairs we typically recommend modifying the prompt so that you can see the role and state immediately when you log in.

Re the Firepower module are you saying you only have it on one of the two units? You can exclude it from monitoring for failover purposes but this is not a recommended configuration for most situations.

Re: Failover failure

Fishel Erps — Fri, 20 Sep 2024 18:11:51 GMT

Marvin,

1) How do you have the prompt/hostname display differently for the secondary in an HA situation?

2) If the Link and Stateful interfaces are directly connected to the ASAs at each end, why would the hello not be heard from the mate?

topic Re: Failover failure in Network Security

Failover failure

Re: Failover failure

Re: Failover failure