about stateful active/standby failover

[Sun Flower]

Hello guys.

I have two ASA's, same model and hardware. Asa have configured stateful active/standby failover by someone, few years ago. It was working normally until recently and no one have changed this configuration. Then Secondary unit is failed. Ping between 2 interfaces is ok. Please help me to resolve this problem.

on Primary site

interface Management0/0

description STATE Failover Interface

management-only

interface GigabitEthernet1/1

description LAN Failover Interface

failover

failover lan unit primary

failover lan interface failover GigabitEthernet1/1

failover link state Management0/0

failover interface ip failover 172.16.1.1 255.255.255.0 standby 172.16.1.2

failover interface ip state 172.16.0.1 255.255.255.0 standby 172.16.0.2

on Secondary site

interface Management0/0

description STATE Failover Interface

management-only

interface GigabitEthernet1/1

description LAN Failover Interface

output of show failover on PRIMARY

show run failover

failover

failover lan unit primary

failover lan interface failover GigabitEthernet1/1

failover link state Management0/0

failover interface ip failover 172.16.1.1 255.255.255.0 standby 172.16.1.2

failover interface ip state 172.16.0.1 255.255.255.0 standby 172.16.0.2

F1# show failover

Failover On

Failover unit Primary

Failover LAN Interface: failover GigabitEthernet1/1 (up)

Unit Poll frequency 1 seconds, holdtime 15 seconds

Interface Poll frequency 5 seconds, holdtime 25 seconds

Interface Policy 1

Monitored Interfaces 5 of 256 maximum

Version: Ours 8.2(2), Mate 8.2(2)

Last Failover at: 08:03:11 ULAST Jan 1 2003

        This host: Primary - Active

                Active time: 5755203 (sec)

                slot 0: ASA5550 hw/sw rev (2.0/8.2(2)) status (Up Sys)

                  Interface Backup2 (10.2.5.1): Normal (Waiting)

                  Interface Internet (202.131.225.90): No Link (Waiting)

                  Interface Backup1 (10.3.5.1): Normal (Waiting)

                  Interface Server (192.168.227.1): Normal (Waiting)

                  Interface Bank (10.20.1.1): Normal (Waiting)

                slot 1: ASA-SSM-4GE-INC hw/sw rev (1.0/1.0(0)10) status (Up)

        Other host: Secondary - Failed

                Active time: 0 (sec)

                slot 0: ASA5550 hw/sw rev (2.0/8.2(2)) status (Up Sys)

                  Interface Backup2 (0.0.0.0): No Link (Waiting)

                  Interface Internet (0.0.0.0): No Link (Waiting)

                  Interface Backup1 (0.0.0.0): Normal (Waiting)

                  Interface Server (0.0.0.0): Normal (Waiting)

                  Interface Bank (0.0.0.0): Normal (Waiting)

                slot 1: ASA-SSM-4GE-INC hw/sw rev (1.0/1.0(0)10) status (Up)

Stateful Failover Logical Update Statistics

        Link : state Management0/0 (up)

        Stateful Obj    xmit       xerr       rcv        rerr

        General         76184539   0          767513     6

        sys cmd         767328     0          767326     1

        up time         0          0          0          0

        RPC services    0          0          0          0

        TCP conn        25878669   0          11         5

        UDP conn        40545710   0          40         0

        ARP tbl         8987688    0          136        0

        Xlate_Timeout   0          0          0          0

        IPv6 ND tbl     0          0          0          0

        VPN IKE upd     1140       0          0          0

        VPN IPSEC upd   4004       0          0          0

        VPN CTCP upd    0          0          0          0

        VPN SDI upd     0          0          0          0

        VPN DHCP upd    0          0          0          0

        SIP Session     0          0          0          0

        Logical Update Queue Information

                        Cur     Max     Total

        Recv Q:         0       7       6522961

        Xmit Q:         0       34      106685671

output of show failover on SECONDARY

F1#  show failover

Failover On

Failover unit Secondary

Failover LAN Interface: failover GigabitEthernet1/1 (up)

Unit Poll frequency 1 seconds, holdtime 15 seconds

Interface Poll frequency 5 seconds, holdtime 25 seconds

Interface Policy 1

Monitored Interfaces 5 of 256 maximum

Version: Ours 8.2(2), Mate 8.2(2)

Last Failover at: 03:36:23 ULAST Dec 15 2013

       This host: Secondary - Failed

                Active time: 0 (sec)

                slot 0: ASA5550 hw/sw rev (2.0/8.2(2)) status (Up Sys)

                  Interface Backup2 (0.0.0.0): No Link (Waiting)

                  Interface Internet (0.0.0.0): No Link (Waiting)

                  Interface Backup1 (0.0.0.0): Normal (Waiting)

                  Interface Server (0.0.0.0): Normal (Waiting)

                  Interface Bank (0.0.0.0): Normal (Waiting)

                slot 1: ASA-SSM-4GE-INC hw/sw rev (1.0/1.0(0)10) status (Up)

        Other host: Primary - Active

                Active time: 5743217 (sec)

                slot 0: ASA5550 hw/sw rev (2.0/8.2(2)) status (Up Sys)

                  Interface Backup2 (10.2.5.1): Normal (Waiting)

                  Interface Internet (202.131.225.90): No Link (Waiting)

                  Interface Backup1 (10.3.5.1): Normal (Waiting)

                  Interface Server (192.168.227.1): Normal (Waiting)

                  Interface Bank (10.20.1.1): Normal (Waiting)

                slot 1: ASA-SSM-4GE-INC hw/sw rev (1.0/1.0(0)10) status (Up)

Stateful Failover Logical Update Statistics

        Link : state Management0/0 (up)

        Stateful Obj    xmit       xerr       rcv        rerr

        General         765518     0          35843181   874

        sys cmd         765518     0          765516     0

        up time         0          0          0          0

        RPC services    0          0          0          0

        TCP conn        0          0          12671303   80

        UDP conn        0          0          13432853   133

        ARP tbl         0          0          8968384    661

        Xlate_Timeout   0          0          0          0

        IPv6 ND tbl     0          0          0          0

        VPN IKE upd     0          0          1137       0

        VPN IPSEC upd   0          0          3988       0

        VPN CTCP upd    0          0          0          0

        VPN SDI upd     0          0          0          0

        VPN DHCP upd    0          0          0          0

        SIP Session     0          0          0          0

        Logical Update Queue Information

                        Cur     Max     Total

        Recv Q:         0       9       72011189

        Xmit Q:         0       1       765518

[Marius Gunnerud]

please be more specific with what you have tested.  "Ping between 2 interfaces is ok" doesn't tell us much. 

Which interfaces are you pinging between?

have you tested between other interfaces as well?

Is the ASA that shows as failed the ASA that used to be the primary?

Have you logged in via consol on both ASAs and checked the actual status of the ASAs (are they both active or has one of them truely failed)?

As the show output indicates is that either one of the ASAs has failed, or there is a communication issue between them.  this could very well be the result of a failed interface or a faulty cable.  By default it only takes one of the monitored interfaces to fail (or lose connectivity) for a failover to happen.

--
Please remember to rate and select a correct answer

[Sun Flower]

- ping is ok between 172.16.1.1 and 172.16.1.2, 172.16.0.1 and 172.16.0.2

- ASA that shows as failed the ASA that didn't use to be the primary , it used to be secondary.

- Yes, i logged via console on both ASAs and checked status of the ASAs. Primary is active and Secondary is failed.

- I have changed cable. Primary ASA indicates below as soon as cable changed.

Beginning configuration replication: Sending to mate.

End Configuration Replication to mate

Then output of SHOW FAILOVER on PRIMARY ASA :

F1# show failover

Failover On

Failover unit Primary

Failover LAN Interface: failover GigabitEthernet1/1 (up)

Unit Poll frequency 1 seconds, holdtime 15 seconds

Interface Poll frequency 5 seconds, holdtime 25 seconds

Interface Policy 1

Monitored Interfaces 5 of 256 maximum

Version: Ours 8.2(2), Mate 8.2(2)

Last Failover at: 08:03:11 ULAST Jan 1 2003

        This host: Primary - Active

                Active time: 5812656 (sec)

                slot 0: ASA5550 hw/sw rev (2.0/8.2(2)) status (Up Sys)

                  Interface Backup2 (10.2.5.1): Normal (Waiting)

                  Interface Internet (202.131.225.90): No Link (Waiting)

                  Interface Backup1 (10.3.5.1): Normal (Waiting)

                  Interface Server (192.168.227.1): Normal (Waiting)

                  Interface Bank (10.20.1.1): Normal (Waiting)

                slot 1: ASA-SSM-4GE-INC hw/sw rev (1.0/1.0(0)10) status (Up)

       Other host: Secondary - Standby Ready

                Active time: 9 (sec)

                slot 0: ASA5550 hw/sw rev (2.0/8.2(2)) status (Up Sys)

                  Interface Backup2 (0.0.0.0): No Link (Waiting)

                  Interface Internet (0.0.0.0): No Link (Waiting)

                  Interface Backup1 (0.0.0.0): Normal (Waiting)

                  Interface Server (0.0.0.0): Normal (Waiting)

                  Interface Bank (0.0.0.0): Normal (Waiting)

                slot 1: ASA-SSM-4GE-INC hw/sw rev (1.0/1.0(0)10) status (Up)

Stateful Failover Logical Update Statistics

        Link : state Management0/0 (up)

        Stateful Obj    xmit       xerr       rcv        rerr

        General         76940782   0          775168     6

        sys cmd         774983     0          774981     1

        up time         0          0          0          0

        RPC services    0          0          0          0

        TCP conn        26125140   0          11         5

        UDP conn        40971274   0          40         0

        ARP tbl         9064174    0          136        0

        Xlate_Timeout   0          0          0          0

        IPv6 ND tbl     0          0          0          0

        VPN IKE upd     1155       0          0          0

        VPN IPSEC upd   4056       0          0          0

        VPN CTCP upd    0          0          0          0

        VPN SDI upd     0          0          0          0

        VPN DHCP upd    0          0          0          0

        SIP Session     0          0          0          0

        Logical Update Queue Information

                        Cur     Max     Total

        Recv Q:         0       7       6588043

        Xmit Q:         0       34      107757911

But few seconds later Secondary ASA become FAILED.

And i also did FAILOVER RESET  command. After this command, secondary ASA became Standby Ready then few seconds later it became Failed again. Why does it become Failed again ?

[Marius Gunnerud]

Is this a new Active/Standby setup?

If it is not a new setup, has it ever worked and for how long was it working?

do you have the command monitor-interface configured (where interface name is the name of the interface you want to monitor and trigger a failover?  this command needs to be issued for each interface that you want to be monitored and can trigger a failover if it fails.

--
Please remember to rate and select a correct answer

[Sun Flower]

Thank you for your reply Marius....

It is not new Active?stanby setup. It was working for 3 years.

I haven't configured monitor-interface . But it was working without this command.

[Marius Gunnerud · Accepted solution]

You have a couple no link messages on your secondary as well as a no link on your primary.

       Interface Backup2 (0.0.0.0): No Link (Waiting)

       Interface Internet (0.0.0.0): No Link (Waiting)

I suggest checking these cables.  Remember that unless you have changed the default configuration a single interface failure, or even connectivity problems between an interface on the two ASAs will cause a failure.

If that doesn't help, try entering the monitor-interface command for the interfaces.

--
Please remember to rate and select a correct answer

[Mizanul Islam · Accepted solution]

Hi,

I have faced the same problem, I suggest to please check your configuration where Primary firewall interface (active & standby IP address configured correctlly). Also you check this command using console port. #show failover .

It could be show like that:

---------------------------------------------------------------------------

Interface dmz1 (10.98.57.3): Normal (Monitored)

Interface inside (10.98.8.97): Normal (Monitored)

---------------------------------------------------------------------------

Regards

Parosh

[Sun Flower]

Thanks guys... I checked interfaces and found out fault... I have solved...