ASA Failover State showing Failed after switch stacking/reboot

suneetmalhotra — Wed, 27 Apr 2022 12:25:51 GMT

So there were 2 Cisco ASA 5000 series boxes and their failover lan and state interfaces were connected via 2 C9300 switches.

FW1>>FailoverLAN>>>>>SW1>>>>>Po1<<<<<<<<<<SW2<<<<<<FailoverLAN<<FW2

FW1>>FailoverState>>>>>SW1>>>>>Po1<<<<<<<<<<SW2<<<<<<FailoverState<<FW2

SW1 and SW2 were 2 individual standalone switches. I recently converted each one of the switches into a stack of 2 unit. So now

Before:

SW1(standalone C9300)

SW2(standalone C9300)

After

SW1(stack of 2 C9300)

SW2(stack of 2 C9300)

During this activity, I failed over the HSRP to the other leg and performed the stacking activity,

After restoring everything to normal, I see that the failover state is showing Group1 and Group2 as Active on Secondary unit and Failed on Primary unit.

Below is the show failover result,

Failover On
Failover unit Secondary
Failover LAN Interface: failoverlan GigabitEthernet0/2 (up)
Reconnect timeout 0:00:00
Unit Poll frequency 1 seconds, holdtime 15 seconds
Interface Poll frequency 5 seconds, holdtime 25 seconds
Interface Policy 1
Monitored Interfaces 18 of 566 maximum
MAC Address Move Notification Interval not set
Version: Ours 9.13(1)16, Mate 9.13(1)16
Serial Number: Ours <CENSORED>, Mate <CENSORED>
Group 1 last failover at: 06:58:56 GMT Apr 7 2022
Group 2 last failover at: 05:35:17 GMT Apr 7 2022

This host: Secondary
Group 1 State: Active
Active time: 1744602 (sec)
Group 2 State: Active
Active time: 1749621 (sec)

slot 0: ASA5545 hw/sw rev (1.0/9.13(1)16) status (Up Sys)
admin Interface management (12.12.48.9): Normal (Not-Monitored)
C1-PIPSEC1 Interface Outside (12.12.48.99): Normal (Waiting)
C1-PIPSEC1 Interface CA_Internal (12.12.53.126): Normal (Not-Monitored)
C1-PIPSEC1 Interface CA_External (12.12.53.142): Normal (Not-Monitored)
C1-PIPSEC1 Interface CA_Management (12.12.53.158): Normal (Not-Monitored)
C1-PIPSEC1 Interface Inside (12.12.48.113): Unknown (Waiting)
C1-PIPSEC1 Interface D_G_C_I (12.12.49.125): Normal (Waiting)
C3-PIGW Interface Outside (13.13.100.13): Normal (Waiting)
C3-PIGW Interface Mickey (0.0.0.0): Normal (Not-Monitored)
C3-PIGW Interface O-T-I (12.12.53.13): Normal (Not-Monitored)
C3-PIGW Interface S-PROD (12.12.53.30): Unknown (Waiting)
C3-PIGW Interface S-ADM (12.12.53.46): Unknown (Waiting)
C3-PIGW Interface S-ESX (12.12.53.54): Unknown (Waiting)
C3-PIGW Interface S-ADM_QA (12.12.53.62): Unknown (Waiting)
C3-PIGW Interface S-QA (12.12.53.70): Unknown (Waiting)
C3-PIGW Interface GIP-2 (12.12.53.77): Normal (Not-Monitored)
C3-PIGW Interface GIP-3 (12.12.53.85): Normal (Not-Monitored)
C3-PIGW Interface GIP-4 (12.12.53.93): Normal (Not-Monitored)
C3-PIGW Interface GIP-1 (12.12.53.101): Normal (Not-Monitored)
C3-PIGW Interface Inside (12.12.52.113): Unknown (Waiting)
C3-PIGW Interface DMZ-3 (13.13.100.69): Failed (Waiting)
C3-PIGW Interface DMZ-2 (13.13.100.61): Normal (Waiting)
C3-PIGW Interface PEN-IPMI (12.12.53.166): Normal (Not-Monitored)
C3-PIGW Interface PEN-MGMT (12.12.53.110): Normal (Not-Monitored)
C3-PIGW Interface PEN-SERVICE (12.12.53.189): Normal (Not-Monitored)
C2-PSSL1 Interface Outside (12.12.50.99): Normal (Waiting)
C2-PSSL1 Interface Inside (12.12.50.113): Unknown (Waiting)
C2-PSSL1 Interface D_G_C_S (12.12.51.254): Failed (Waiting)
C4-PRA Interface Outside (12.12.54.99): Normal (Waiting)
C4-PRA Interface Inside (12.12.54.113): Normal (Waiting)
C4-PRA Interface D_G_C_R (12.12.55.254): Failed (Waiting)

Other host: Primary
Group 1 State: Failed
Active time: 42299808 (sec)
Group 2 State: Failed
Active time: 728 (sec)

slot 0: ASA5545 hw/sw rev (1.0/9.13(1)16) status (Up Sys)
admin Interface management (12.12.48.10): Normal (Not-Monitored)
C1-PIPSEC1 Interface Outside (12.12.48.100): Normal (Waiting)
C1-PIPSEC1 Interface CA_Internal (12.12.53.125): Normal (Not-Monitored)
C1-PIPSEC1 Interface CA_External (12.12.53.141): Normal (Not-Monitored)
C1-PIPSEC1 Interface CA_Management (12.12.53.157): Normal (Not-Monitored)
C1-PIPSEC1 Interface Inside (12.12.48.114): Normal (Waiting)
C1-PIPSEC1 Interface D_G_C_I (12.12.49.126): Normal (Waiting)
C3-PIGW Interface Outside (13.13.100.14): Normal (Waiting)
C3-PIGW Interface Mickey (0.0.0.0): Normal (Not-Monitored)
C3-PIGW Interface O-T-I (12.12.53.14): Normal (Not-Monitored)
C3-PIGW Interface S-PROD (12.12.53.29): Normal (Waiting)
C3-PIGW Interface S-ADM (12.12.53.45): Normal (Waiting)
C3-PIGW Interface S-ESX (12.12.53.53): Normal (Waiting)
C3-PIGW Interface S-ADM_QA (12.12.53.61): Normal (Waiting)
C3-PIGW Interface S-QA (12.12.53.69): Normal (Waiting)
C3-PIGW Interface GIP-2 (12.12.53.78): Normal (Not-Monitored)
C3-PIGW Interface GIP-3 (12.12.53.86): Normal (Not-Monitored)
C3-PIGW Interface GIP-4 (12.12.53.94): Normal (Not-Monitored)
C3-PIGW Interface GIP-1 (12.12.53.102): Normal (Not-Monitored)
C3-PIGW Interface Inside (12.12.52.114): Normal (Waiting)
C3-PIGW Interface DMZ-3 (13.13.100.70): Normal (Waiting)
C3-PIGW Interface DMZ-2 (13.13.100.62): Normal (Waiting)
C3-PIGW Interface PEN-IPMI (12.12.53.165): Normal (Not-Monitored)
C3-PIGW Interface PEN-MGMT (12.12.53.109): Normal (Not-Monitored)
C3-PIGW Interface PEN-SERVICE (12.12.53.190): Normal (Not-Monitored)
C2-PSSL1 Interface Outside (12.12.50.100): Normal (Waiting)
C2-PSSL1 Interface Inside (12.12.50.114): Normal (Waiting)
C2-PSSL1 Interface D_G_C_S (12.12.51.253): Normal (Waiting)
C4-PRA Interface Outside (12.12.54.100): Normal (Waiting)
C4-PRA Interface Inside (12.12.54.114): Normal (Waiting)
C4-PRA Interface D_G_C_R (12.12.55.253): Normal (Waiting)

Stateful Failover Logical Update Statistics
Link : failovertstate GigabitEthernet0/3 (up)
Stateful Obj xmit xerr rcv rerr
General 18388453021 0 20106534738 815
sys cmd 5884606 0 5884603 0
up time 0 0 0 0
RPC services 0 0 0 0
TCP conn 7030569350 0 9597360502 0
UDP conn 11007061799 0 10480932177 0
ARP tbl 18842 0 19445150 0
Xlate_Timeout 0 0 0 0
IPv6 ND tbl 0 0 0 0
VPN IKEv1 SA 0 0 0 0
VPN IKEv1 P2 0 0 0 0
VPN IKEv2 SA 0 0 0 0
VPN IKEv2 P2 0 0 0 0
VPN CTCP upd 0 0 0 0
VPN SDI upd 0 0 0 0
VPN DHCP upd 0 0 0 0
SIP Session 197017059 0 1818790 0
SIP Tx 147290675 0 965535 0
SIP Pinhole 610647 0 127893 815
Route Session 0 0 0 0
Router ID 0 0 0 0
User-Identity 43 0 88 0
CTS SGTNAME 0 0 0 0
CTS PAC 0 0 0 0
TrustSec-SXP 0 0 0 0
IPv6 Route 0 0 0 0
STS Table 0 0 0 0
Umbrella Device-ID 0 0 0 0

Show failover history

==========================================================================
Group From State To State Reason
==========================================================================
05:33:10 GMT Apr 7 2022
1 Standby Ready Bulk Sync No Error

05:33:10 GMT Apr 7 2022
2 Standby Ready Bulk Sync No Error

05:33:10 GMT Apr 7 2022
0 Bulk Sync Standby Ready Configuration mismatch

05:33:15 GMT Apr 7 2022
2 Bulk Sync Standby Ready No Error

05:33:20 GMT Apr 7 2022
1 Bulk Sync Standby Ready No Error

05:35:17 GMT Apr 7 2022
2 Standby Ready Just Active Failover state check

05:35:17 GMT Apr 7 2022
2 Just Active Active Drain Failover state check

05:35:17 GMT Apr 7 2022
2 Active Drain Active Applying Config Failover state check

05:35:17 GMT Apr 7 2022
2 Active Applying Config Active Config Applied Failover state check

05:35:17 GMT Apr 7 2022
2 Active Config Applied Active Failover state check

06:58:56 GMT Apr 7 2022
1 Standby Ready Just Active Interface check
This host:0
Other host:0

06:58:56 GMT Apr 7 2022
1 Just Active Active Drain Interface check
This host:0
Other host:0

06:58:56 GMT Apr 7 2022
1 Active Drain Active Applying Config Interface check
This host:0
Other host:0

06:58:56 GMT Apr 7 2022
1 Active Applying Config Active Config Applied Interface check
This host:0
Other host:0

06:58:56 GMT Apr 7 2022
1 Active Config Applied Active Interface check
This host:0
Other host:0

06:58:56 GMT Apr 7 2022
0 Standby Ready Just Active Interface check
This host:0
Other host:0

06:58:56 GMT Apr 7 2022
0 Just Active Active Drain Interface check
This host:0
Other host:0

06:58:56 GMT Apr 7 2022
0 Active Drain Active Applying Config Interface check
This host:0
Other host:0

06:58:56 GMT Apr 7 2022
0 Active Applying Config Active Config Applied Interface check
This host:0
Other host:0

06:58:56 GMT Apr 7 2022
0 Active Config Applied Active Interface check
This host:0
Other host:0

show failover state

State Last Failure Reason Date/Time
This host - Secondary
Group 1 Active Ifc Failure 07:07:08 GMT Apr 7 2022
Group 2 Active Ifc Failure 07:07:08 GMT Apr 7 2022
Other host - Primary
Group 1 Failed Ifc Failure 17:50:55 GMT Dec 3 2020
Group 2 Failed Ifc Failure 17:50:55 GMT Dec 3 2020

====Configuration State===
Sync Done
Sync Done - STANDBY
====Communication State===
Mac set

Re: ASA Failover State showing Failed after switch stacking/reboot

MHM Cisco World — Mon, 02 May 2022 10:18:51 GMT

Re: ASA Failover State showing Failed after switch stacking/reboot

Mark Elsen — Mon, 02 May 2022 10:28:27 GMT

@MHM Cisco World , Note that you can also follow or subscribe to post , if you go right at the subject of the post , go a little bit up with the mouse there are 3 vertical dots, click on that and the subsequently Subscribe. Then you don't need to bother posters mailbox with follow-message or not have this extra step in this post.

Re: ASA Failover State showing Failed after switch stacking/reboot

Mark Elsen — Mon, 02 May 2022 10:31:12 GMT

- Connect to current-active ASA (SSH) with https://cway.cisco.com/cli , at the top left run (or press) 'System Diagnostics'

Re: ASA Failover State showing Failed after switch stacking/reboot

suneetmalhotra — Mon, 16 May 2022 15:23:59 GMT

Thank you everyone for your help.

The issue got resolved after I run "failover reset" command on the failed unit.

topic ASA Failover State showing Failed after switch stacking/reboot in Network Security

ASA Failover State showing Failed after switch stacking/reboot

Re: ASA Failover State showing Failed after switch stacking/reboot

Re: ASA Failover State showing Failed after switch stacking/reboot

Re: ASA Failover State showing Failed after switch stacking/reboot

Re: ASA Failover State showing Failed after switch stacking/reboot