topic Hi, in Network Security

Failover issue

Mark Cavendish — Tue, 12 Mar 2019 06:57:38 GMT

I am hoping someone can help me with this issue I am having. On Friday I noticed I lost the ability to telnet and ssh to my Cisco box which is a 5545, ASA version 9.1(1) and ASDM version 7.3(1).

I read it could be a bug and I tried to remove the telnet and SSH commands and reissue them, but it still just times out and it worked perfectly before.

When I logged on I saw the failover state had the secondary firewall as the active. So I thought I would reload the standby Primary from the ASDM to see if that would force the synching of commands across. Yet now the failover state is showing the primary as sync config instead of standby.

Here is the commands from the standby active I am logged onto:

Result of the command: "show failover"

Failover On
Failover unit Secondary
Failover LAN Interface: FAILOVER GigabitEthernet0/3 (up)
Unit Poll frequency 1 seconds, holdtime 15 seconds
Interface Poll frequency 5 seconds, holdtime 25 seconds
Interface Policy 1
Monitored Interfaces 4 of 316 maximum
Version: Ours 9.1(1), Mate 9.1(1)
Last Failover at: 15:59:36 GMT/BDT May 17 2015
This host: Secondary - Active
Active time: 16853397 (sec)
slot 0: ASA5545 hw/sw rev (1.0/9.1(1)) status (Up Sys)
Interface Private (10.2.1.4): Unknown (Waiting)
Interface DMZ (10.99.14.1): Unknown (Waiting)
Interface Public (193.63.212.2): Unknown (Waiting)
Interface management (192.168.1.1): No Link (Waiting)
Other host: Primary - Sync Config
Active time: 0 (sec)
slot 0: ASA5545 hw/sw rev (1.0/9.1(1)) status (Up Sys)
Interface Private (0.0.0.0): Unknown (Waiting)
Interface DMZ (0.0.0.0): Unknown (Waiting)
Interface Public (0.0.0.0): Unknown (Waiting)
Interface management (0.0.0.0): Unknown (Waiting)

Stateful Failover Logical Update Statistics
Link : Unconfigured.

__________________

Result of the command: "show failover state"

State Last Failure Reason Date/Time
This host - Secondary
Active Ifc Failure 15:03:13 GMT/BDT May 17 2015
Other host - Primary
Sync Config Comm Failure 14:01:09 GMT/BST Nov 28 2015

====Configuration State===
Config Syncing
Sync Done - STANDBY
====Communication State===

____________________

Result of the command: "show failover history"

==========================================================================
From State To State Reason
==========================================================================
15:26:19 GMT/BDT May 17 2015
Just Active Active Drain HELLO not heard from mate

15:26:19 GMT/BDT May 17 2015
Active Drain Active Applying Config HELLO not heard from mate

15:26:19 GMT/BDT May 17 2015
Active Applying Config Active Config Applied HELLO not heard from mate

15:26:19 GMT/BDT May 17 2015
Active Config Applied Active HELLO not heard from mate

15:34:30 GMT/BDT May 17 2015
Active Cold Standby Failover state check

15:34:32 GMT/BDT May 17 2015
Cold Standby Sync Config Failover state check

15:34:42 GMT/BDT May 17 2015
Sync Config Sync File System Failover state check

15:34:42 GMT/BDT May 17 2015
Sync File System Bulk Sync Failover state check

15:34:42 GMT/BDT May 17 2015
Bulk Sync Standby Ready Failover state check

15:45:47 GMT/BDT May 17 2015
Standby Ready Just Active Other unit wants me Active

15:45:47 GMT/BDT May 17 2015
Just Active Active Drain Other unit wants me Active

15:45:47 GMT/BDT May 17 2015
Active Drain Active Applying Config Other unit wants me Active

15:45:47 GMT/BDT May 17 2015
Active Applying Config Active Config Applied Other unit wants me Active

15:45:47 GMT/BDT May 17 2015
Active Config Applied Active Other unit wants me Active

15:57:08 GMT/BDT May 17 2015
Active Standby Ready Set by the config command

15:59:36 GMT/BDT May 17 2015
Standby Ready Just Active Other unit wants me Active

15:59:36 GMT/BDT May 17 2015
Just Active Active Drain Other unit wants me Active

15:59:36 GMT/BDT May 17 2015
Active Drain Active Applying Config Other unit wants me Active

15:59:36 GMT/BDT May 17 2015
Active Applying Config Active Config Applied Other unit wants me Active

15:59:36 GMT/BDT May 17 2015
Active Config Applied Active Other unit wants me Active

==========================================================================

Now I still have the original issue where I can't telnet or ssh and I am now worried why the failover is stuck in sync config. Does this mean it is broken or still working? Can I issue a no failover active on the current primary secondary unit to make the other host the primary which it should be and will that solve the issue? There must be no downtime from both units being down without prior approval, so I can't just restart them both.

Many thanks in advance,

Mark

Hi Mark,

Akshay Rastogi — Mon, 30 Nov 2015 05:27:59 GMT

Hi Mark,

From the description it looks like ASA stuck in sync state. In this state, you can not perform failover,'failover write-standby', ' no failover active', 'no failover' these will not work. It would throw an error saying failover is in sync transition state.

From my experience so far, reloading the 'active' device resolves the issues always. Sometime it takes a lot time if the configuration is big and comes back normally after sometime.

If it is still in sync configuration state then go for a maintenace window and reload the active device. It is possible that this might be the issue why you are not able to access the ASA boxes.

Hope it helps.

Regards,

Akshay Rastogi

Remeber to rate helpful posts.

Hi,

Rishabh Seth — Mon, 30 Nov 2015 06:06:50 GMT

Hi Mark,

Can you share output of show blocks?

Thanks,

Thanks both. I guessed a

Mark Cavendish — Mon, 30 Nov 2015 09:20:13 GMT

Thanks both. I guessed a reload would probably be the only thing to fix this after much more googling and issued a reload noconfirm from the ASDM command line interface but it didn't take the command for some reason. I now cannot reload until this coming w/e where I hope to be onsite to schedule a maintenance window and fingers crossed it solves the issue.

If I make any changes on the active this week (which won't get sent properly to the secondary), will they sync properly after rebooting the active or is there anything else I should do?

This is the result of the sh block command Rishabh:

Result of the command: "sh blocks"

SIZE MAX LOW CNT
0 4200 4188 4200
4 500 499 499
80 3504 3429 3504
256 3224 3137 3218
1550 13874 13673 13861
2048 2100 2092 2100
2560 3732 3729 3732
4096 100 99 100
8192 100 99 100
9344 100 100 100
16384 182 182 182
65536 16 16 16

Many thanks,

Mark

Hi Mark,

Rishabh Seth — Mon, 30 Nov 2015 09:24:08 GMT

Hi Mark,

So the reload did not work after running it from ASDM. Can you check the output of show reload for the firewall which was supposed to be reloaded.

Thanks,

Hi Mark,

Akshay Rastogi — Mon, 30 Nov 2015 11:13:02 GMT

Hi Mark,

Blocks looks fine. Therefore, there is no block depletion which could have cause loss of ssh or telnet access to ASA.

Reload of current Active ASA awould be enough. After the reload they must come up fine.

Regards,

Akshay Rastogi

Hi all

Mark Cavendish — Mon, 07 Dec 2015 08:53:51 GMT

Hi all

Just to confirm a reload after the w/e sorted both issues with the failover and telnet.

Many thanks for all your advice again.

Mark