cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
288
Views
0
Helpful
0
Replies
ida71
Beginner

CSCvv87495 - FMC unresponsive no SSH, no GUI - Error 500, Console init: cannot fork, retry.. add 6.6.1 to affected versions

I encountered this bug on Friday 5th March 2021, after upgrading a HA pair of FMC1600's from 6.4.0.10 to 6.6.1-91, approx 15hrs after the upgrade was complete & everything was working fine, the HA sync failed due to loss of SSL between units caused by this bug.

 

Having discovered the loss of GUI & SSH access, we sent an engineer to site as Cisco TAC wanted to diagnose the fault, as did I. Console access when we were finally able to get an engineer to the lights out DC showed the following error

init: cannot fork, retry..

This was repeated on the screen pretty much continuously stopping us logging in. When I searched for this error, the bug CSCvv87495 was the first entry in the search, which matched our symptoms exactly & the only thing that changed during the upgrade was the new login page & GUI look, also offered at login to activate "Cisco Support Diagnostics" which I thought would be a good idea, turned out I was wrong

 

Cisco need to update the 6.6.1 release notes !  Located here

https://www.cisco.com/c/en/us/td/docs/security/firepower/660/66x/relnotes/firepower-release-notes-66x/features.html?bookSearch=true

 

This is the only reference to the Diagnostics

"Cisco Support Diagnostics

In Version 6.5.0+, Cisco Support Diagnostics (sometimes called Cisco Proactive Support) sends configuration and operational health data to Cisco, and processes that data through our automated problem detection system, allowing us to proactively notify you of issues. This feature also allows Cisco TAC to collect essential information from your devices during the course of a TAC case.

During initial setup and upgrades, you may be asked to accept or decline participation. You can also opt in or out at any time."

 

I asked TAC to check this twice, but did NOT get a response. During the whole issue (approx 24hrs) the Primary FMC was still communicating with Cisco, was just not accessible locally. The fix was to reboot the Primary FMC, await its long return (circa 25minutes) to an operational state, plus 16minutes for a DB sync with the Secondary FMC, make it active. Then disable the Cisco Support  Diagnostics under SYSTEM>Licences>Smart Licenses.

 

Hope this helps someone else. 

0 REPLIES 0