Cisco FirePower 4110 Flash initiation issue

ave7ome — Thu, 12 Jan 2023 18:33:15 GMT

Hello,

Some time ago both FPR 4110 appliances located on the same site went down...L3-wise, i guess. At the same time.
Physical interfaces of core devices that were connected to appliances remained up/up, but contexts on Virtual ASA stopped responding. GUI became unavailable.
After the lookup, i could see that DME process went down on the FPR:
Firepower4110# scope firmware
Software Error: Exception during execution: [Error: Timed out communicating with DME]
Firepower4110(local-mgmt)# show pmon state
{output omitted}
svc_sam_dme failed 4(4) 0 6 no
{output omitted}

Additionally it appears that there are some issues with the file system:
Firepower4110(local-mgmt)# ls
/bin/ls: reading directory '/workspace/': Input/output error
Firepower4110(fxos)# show startup-config
No startup configuration
Firepower4110(fxos)# sh logg nvram
2023 Jan 11 10:08:31 %$ VDC-1 %$ (none) kernel: [38370840.345670] EXT4-fs error (device sda7): ext4_find_entry:1442: inode #2: comm svc_sam_dcosAG: reading directory lblock 0
2023 Jan 11 10:08:31 %$ VDC-1 %$ (none) kernel: [38370840.345786] EXT4-fs error (device sda7): ext4_find_entry:1442: inode #2: comm svc_sam_dcosAG: reading directory lblock 0
2023 Jan 11 10:08:32 %$ VDC-1 %$ (none) kernel: [38370841.487203] EXT4-fs error (device sda7): ext4_find_entry:1442: inode #2: comm httpd.sh: reading directory lblock 0
2023 Jan 11 10:08:34 %$ VDC-1 %$ (none) kernel: [38370843.048717] EXT4-fs error (device sda3): ext4_find_entry:1442: inode #2: comm lacp: reading directory lblock 0
2023 Jan 11 10:08:35 %$ VDC-1 %$ (none) kernel: [38370844.050561] EXT4-fs error (device sda3): ext4_find_entry:1442: inode #2: comm lacp: reading directory lblock 0
2023 Jan 11 10:08:35 %$ VDC-1 %$ (none) kernel: [38370844.050855] EXT4-fs error (device sda3): ext4_find_entry:1442: inode #2: comm lacp: reading directory lblock 0

I attempted to reboot the secondary node. It went straight to ROMMON mode. Commands like "dir" or "ls" returned an error saying "No mapping".

I decided to go according to Cisco FPR4100 Troubleshooting article, section "Recovering from corrupted file system":
Link

First two steps went well. Between 2nd and 3rd steps, i could see the following:
Checking all filesystems.r.r.r.r.rrr done.
##############################################################
Boot has been interrupted or internal disk is not initialized!
##############################################################
##############################################################
Do you want to initialize it? (y/n) y
fdisk: cannot open /dev/sda: No

And when i tried to do the 3rd step (initialyze the flash):
switch(boot)# init system
This command is going to erase your startup-config, licenses as well as the contents of your bootflash:.
Do you want to continue? (y/n) [n] y
fdisk: cannot open /dev/sda: No such device or address
/isanboot/sbin/init-system: line 144: [: -gt: unary operator expected
There is a problem with the flash device. It could not be initialized.

So my question is:
Is there any way to go further with recovery attempts? Is there a hope?

Thank you and hope for some hint going forward.

Re: Cisco FirePower 4110 Flash initiation issue

balaji.bandi — Thu, 12 Jan 2023 19:42:34 GMT

i do not believe that step 2 success as per your logs.

Can you post complete logs attached as text. (how are you loading boot image ? USB or TFTP ?

Re: Cisco FirePower 4110 Flash initiation issue

ave7ome — Fri, 13 Jan 2023 09:21:07 GMT

Hello,

Thank you for the response.

I'm loading a kickstart image from the tftp server.
Please see the logs attached starting from the execution of "boot" command from ROMMON.

Thank you!

topic Re: Cisco FirePower 4110 Flash initiation issue in Network Security

Cisco FirePower 4110 Flash initiation issue

Re: Cisco FirePower 4110 Flash initiation issue

Re: Cisco FirePower 4110 Flash initiation issue