Firepower (ISA3k) lost FMC connection after power cut

[Oleksii Maksymov]

Hello all!

I have a couple of ISA3k devices, which have lost FMC connection after losing power.

More than that, they show no FMC configured at all:

> show managers
No managers configured.

But if I try to add one it reports DB loss:

> configure manager add 1.1.1.1 asdasdasdasdasd
getPeersByRole: unable to connect to db at /usr/local/sf/lib/perl/5.10.1/SF/PeerManager/Peers.pm line 238.
Unable to access DetectionEngine::bulkLoad

Same if I try to use manage_procs.pl:

/home/admin# manage_procs.pl

**************** Configuration Utility **************

1 Reconfigure Correlator
2 Reconfigure and flush Correlator
3 Restart Comm. channel
4 Update routes
5 Reset all routes
6 Validate Network
0 Exit

**************************************************************
Enter choice: 3
Unable to connect to database: at /ngfw/usr/local/sf/lib/perl/5.10.1/SF/PeerManager/PeerInfo.pm line 211.
Disabling sfipproxy... at /ngfw/usr/local/sf/lib/perl/5.10.1/SF/PeerManager/ControlState.pm line 2407.
Unable to connect to database: at /ngfw/usr/local/sf/lib/perl/5.10.1/SF/PeerManager/PeerInfo.pm line 211.
Unable to connect to database: at /ngfw/usr/local/sf/lib/perl/5.10.1/SF/PeerManager/PeerInfo.pm line 211.
1
**************** Configuration Utility **************

1 Reconfigure Correlator
2 Reconfigure and flush Correlator
3 Restart Comm. channel
4 Update routes
5 Reset all routes
6 Validate Network
0 Exit

**************************************************************
Enter choice: 2
Unable to connect to database: at /ngfw/usr/local/sf/lib/perl/5.10.1/SF/PeerManager/PeerInfo.pm line 211.
Couldn't connect to DB at /ngfw/usr/local/sf/lib/perl/5.10.1/SF/RNA/EnterpriseConfig.pm line 906.

Printing stack trace:
called from /ngfw/usr/local/sf/lib/perl/5.10.1/SF/RNA/EnterpriseConfig.pm (906)
called from /ngfw/usr/local/sf/lib/perl/5.10.1/SF/PeerManager/ControlState.pm (321)
called from /ngfw/usr/local/sf/bin/manage_procs.pl (107)
called from /ngfw/usr/local/sf/bin/manage_procs.pl (54)
called from /ngfw/usr/local/sf/bin/manage_procs.pl (100)

I'm using version 6.6.1. I'm ready to do re-image and restore from backup, but I don't have a console connection to the device. So the question is:

How can I reboot the device into the boot-mode, so I can do a re-image via SSH?

If there's no "soft" way, maybe I can just rm -rf /  ? Will FTD then boot into boot mode or in rommon?

PS: re-image via SSH is possible, just will have to configure the DHCP server and use default credentials, I've done it before.

[Oleksii Maksymov]

!!! This is a highly unrecommended way of doing things, please proceed with caution

In case someone finds this later:

1. Setup a DHCP server in FTD MGMT segment (or relay to a remote server)

2. Execute "rm -rf /ngfw" from Linux shell on ISA3k

3. reboot it

After some time (~10 mins for me) I saw that the device received an IP from the DHCP server, and I was able to log in with default "admin/Admin123" credentials.

After that just proceed with the re-image. Somewhy it worked only after the second install init every time (did it 7+ times).

!!! You must re-image to the same SW version. Otherwise boot version may be incompatible and the re-image will fail.

PS: 7.0.1 is much more stable on ISA3k, so I highly recommend upgrading from 6.x