Good morning, I have two FTP 2110s in a HA pair managed by FMC. I am using the KVM version of the virtual FMC software. My upgrade from 6.6.5.2 to 7.0.4 failed (although the Readiness Check ran successfully), the VM became unresponsive. When I restored the VM file from just before the upgrade attempt it follows a consistent path (same thing three tries in a row so far):
The VM starts, no errors. I can ping and log in to the cli. I get the message "You have logged in while system startup is in progress. Please wait, some feature may be unavailable until startup is complete." At this time, the FMC web interface shows the "System processes are starting" animation. I can interact somewhat with the cli but I get a lot of errors of the following type:
System> show version
Failed get_sybase_password $VAR1 = bless( {
'-stacktrace' => 'System (/var/jre/bin/java -cp /var/opt/CSCOpx/MDC/tomcat/shared/lib/cs-lib-4.2.2.b.jar:/var/sf/lib64/:/lib64/:.: DBP) Failed at /usr/local/sf/lib/perl/5.10.1/SF/System/Privileged.pm line 5811
SF::System::Privileged::DecryptSybasePassword() called at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 138
SF::SFDBI::get_sybase_password() called at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 374
SF::SFDBI::connect() called at /usr/local/sf/lib/perl/5.10.1/SF/EODataHandler/Default.pm line 127
SF::EODataHandler::Default::bulkLoad(\'DE_Config\', undef, undef) called at /usr/local/sf/lib/perl/5.10.1/SF/EODataHandler/DetectionEngine.pm line 21
SF::EODataHandler::DetectionEngine::AUTOLOAD(\'DE_Config\', undef, undef) called at /usr/local/sf/lib/perl/5.10.1/SF/EOHandler.pm line 2466
SF::EOHandler::bulkLoad(\'DE_Config\') called at /usr/local/sf/lib/perl/5.10.1/SF/DetectionEngine.pm line 5936
SF::DetectionEngine::getPrimaryDetectionEngineUUID() called at /usr/local/sf/lib/perl/5.10.1/SF/CLI/version.pm line 120
SF::CLI::version::getSRUVersion() called at /usr/local/sf/lib/perl/5.10.1/SF/CLI/version.pm line 27
SF::CLI::version::show() called at /usr/local/sf/lib/perl/5.10.1/SF/CLI.pm line 88
SF::CLI::process_command(\'show\', \'version\') called at /usr/local/sf/bin/sfcli.pl line 7
',
'-stdout' => 'Error occurred during initialization of VM
java/lang/ClassNotFoundException: error in opening JAR file /var/jre/lib/rt.jar
',
'-line' => 5811,
'-file' => '/usr/local/sf/lib/perl/5.10.1/SF/System/Privileged.pm',
'-value' => 256,
'-text' => 'System (/var/jre/bin/java -cp /var/opt/CSCOpx/MDC/tomcat/shared/lib/cs-lib-4.2.2.b.jar:/var/sf/lib64/:/lib64/:.: DBP) Failed',
'-stderr' => '',
'-ui_message' => '',
'-package' => 'SF::System::Privileged'
}, 'Error::SFSystem' );
DBI connect('DBN=vms;Host=127.0.0.1:10033;CharSet=utf8','DBA',...) failed: Invalid user ID or password (DBD: login failed) at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 375
$VAR1 = [
'Connection failed',
bless( {
'-stacktrace' => 'Can\'t call method "prepare" on an undefined value at /usr/lib/perl5/site_perl/5.10.1/Error.pm line 273
Error::subs::run_clauses(\'HASH(0x43af680)\', \'Can\\\'t call method "prepare" on an undefined value at /usr/lo...\', undef, \'ARRAY(0x436dc48)\') called at /usr/lib/perl5/site_perl/5.10.1/Error.pm line 390
Error::subs::try(\'CODE(0x43b5b58)\', \'HASH(0x43af680)\') called at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 401
SF::SFDBI::connect() called at /usr/local/sf/lib/perl/5.10.1/SF/EODataHandler/Default.pm line 127
SF::EODataHandler::Default::bulkLoad(\'DE_Config\', undef, undef) called at /usr/local/sf/lib/perl/5.10.1/SF/EODataHandler/DetectionEngine.pm line 21
SF::EODataHandler::DetectionEngine::AUTOLOAD(\'DE_Config\', undef, undef) called at /usr/local/sf/lib/perl/5.10.1/SF/EOHandler.pm line 2466
SF::EOHandler::bulkLoad(\'DE_Config\') called at /usr/local/sf/lib/perl/5.10.1/SF/DetectionEngine.pm line 5936
SF::DetectionEngine::getPrimaryDetectionEngineUUID() called at /usr/local/sf/lib/perl/5.10.1/SF/CLI/version.pm line 120
SF::CLI::version::getSRUVersion() called at /usr/local/sf/lib/perl/5.10.1/SF/CLI/version.pm line 27
SF::CLI::version::show() called at /usr/local/sf/lib/perl/5.10.1/SF/CLI.pm line 88
SF::CLI::process_command(\'show\', \'version\') called at /usr/local/sf/bin/sfcli.pl line 7
',
'-file' => '/usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm',
'-text' => 'Can\'t call method "prepare" on an undefined value',
'-line' => '379',
'-package' => 'Error::subs'
}, 'Error::Simple' )
];
etc. After some time, both the cli and the website become unresponsive although the VM still answers pings.
Cisco TAC is looking to see if my backup VM is irretrievably corrupted. I am guessing given the above that the answer will be yes. If I set up a brand new FMC VM, will I be able to retrieve the config from the currently working firewalls, or is my only way forward a manual step by step rebuild? I can still access the cli on the firewalls. Thanks in advance for any advice or guidance.
