FXOS Upgrade

fatalXerror · ‎12-07-2020

Hi Guys,

Please help, I am trying to upgrade my FXOS but I am failing. the Chassis 1: is stuck in Upgrading Status for days already.

FP4100 /system # show firmware monitor
FPRM:
Package-Vers: 2.8(1.125)
Upgrade-Status: Ready

Fabric Interconnect A:
Package-Vers: 2.8(1.125)
Upgrade-Status: Ready

Chassis 1:
Server 1:
Package-Vers: 2.8(1.125)
Upgrade-Status: Upgrading

I tried also to re-image the appliance but still the same issue.

When I checked the events, I noticed these logs;

[FSM:STAGE:STALE-FAIL]: Check for the Rommon first response status on blade 1/1(FSM-STAGE:sam:dme:OsControllerDeployOS:HostCheckRommonReady)

[FSM:STAGE:REMOTE-ERROR]: Result: timeout Code: unspecified Message: ROMMON Status 0(sam:dme:OsControllerDeployOS:HostCheckRommonReady)

I also tried to console in to the SSP using the command connect module 1 console, and I noticed these logs.

xfs_quota: cannot setup path for mount /opt/cisco/csp: No such device or address
xfs_quota: cannot find mount point for path `/opt/cisco/csp/cores': No such file or directory
xfs_quota: cannot setup path for mount /opt/cisco/csp: No such device or address
xfs_quota: cannot find mount point for path `/opt/cisco/csp/cores': No such file or directory
Start_CSP_RcInit in Progress...........

Continue to proceed Normal Mode start-up.

System Mode Check: MULTI-INSTANCE mode assigned
System Mode Check: MULTI-INSTANCE mode assigned
INFO: System Disk /dev/sda present. Status: Inoperable.
System Disk /dev/sda is not found. Moving blade into fail-safe mode

error(2, 'No such file or directory')
Cisco Firepower Extensible Operating System (FX-OS) Software. TAC support: http://www.cisco.com/tac Copyright (c) 2009-2016, Cisco Systems, Inc. All rights reserved. The copyrights to certain works contained in this software are owned by other third parties and used and distributed under license. Certain components of this software are licensed under the 'GNU General Public License, version 3' provided with ABSOLUTELY NO WARRANTY under the terms of 'GNU General Public License, Version 3', available here: http://www.gnu.org/licenses/gpl.html. See User Manual (''Licensing'') for details. Certain components of this software are licensed under the 'GNU General Public License, version 2' provided with ABSOLUTELY NO WARRANTY under the terms of 'GNU General Public License, version 2', available here: http://www.gnu.org/licenses/old-licenses/gpl-2.0.html. See User Manual (''Licensing'') for details. Certain components of this software are licensed under the 'GNU LESSER GENERAL PUBLIC LICENSE, version 3' provided with ABSOLUTELY NO WARRANTY under the terms of 'GNU LESSER GENERAL PUBLIC LICENSE' Version 3, available here: http://www.gnu.org/licenses/lgpl.html. See User Manual (''Licensing'') for details. Certain components of this software are licensed under the 'GNU Lesser General Public License, version 2.1' provided with ABSOLUTELY NO WARRANTY under the terms of 'GNU Lesser General Public License, version 2', available here: http://www.gnu.org/licenses/old-licenses/lgpl-2.1.html. See User Manual (''Licensing'') for details. Certain components of this software are licensed under the 'GNU Library General Public License, version 2' provided with ABSOLUTELY NO WARRANTY under the terms of 'GNU Library General Public License, version 2', available here: http://www.gnu.org/licenses/old-licenses/lgpl-2.0.html. See User Manual (''Licensing'') for details.
error(2, 'No such file or directory')
error(2, 'No such file or directory')
error(2, 'No such file or directory')
error(2, 'No such file or directory')
error(2, 'No such file or directory')
error(2, 'No such file or directory')
error(2, 'No such file or directory')
error(2, 'No such file or directory')
error(2, 'No such file or directory')
error(2, 'No such file or directory')

Cisco Security Services Platform
Type ? for list of commands
System Mode Check: MULTI-INSTANCE mode assigned
Firepower-module1-failed>
Waiting for Application infrastructure to be ready...
Verifying the signature of the Application image...
System Mode Check: MULTI-INSTANCE mode assigned
Creating swap file ...
/bin/pyos.sh: li/bin/dd: error w
Creating FXOS swap file ...

/bin/dd: error writing '/opt/cisco/csp/swapfile2': Cannot allocate memory
9581+0 records in
9580+0 records out
9809920 bytes (9.8 MB, 9.4 MiB) copied, 0.0325805 s, 301 MB/s
Done with swap file ...
/opt/ssp-mgmt-scripts/ssp_heimdall_init: line 733: 6630 Killed /opt/bootcli/cisco/cli/bin/ssp_vnic_mvlogs.sh (wd: /)
/opt/ssp-mgmt-scripts/ssp_heimdall_init: line 733: 6783 Killed /usr/bin/heimdall ${HEIMDALL_CONFIG} >> ${CISCO_LOGS_DIR}/stdout_heimdall.log 2>&1
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 6792 Killed sleep 2
/opt/ssp-mgmt-scripts/auth_agent_init: fork: Cannot allocate memory
INIT: cannot fork, retry..
/opt/ssp-mgmt-scripts/ssp_system_storage.sh: line 50: 6801 Killed ${GET_FEATURE_CONFIG} -f ${HDPARM_LOG_STR} -p ${PLTFM} -v ${HDPARM_LOG_SUPP_VALUE}

/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 6856 Killed sleep 2
/opt/ssp-mgmt-scripts/auth_agent_init: fork: Cannot allocate memory
/opt/ssp-mgmt-scripts/ssp_system_storage.sh: line 50: 6857 Killed ${GET_FEATURE_CONFIG} -f ${HDPARM_LOG_STR} -p ${PLTFM} -v ${HDPARM_LOG_SUPP_VALUE}
/opt/ssp-mgmt-scripts/asa_cmd_init: line 81: 6842 Killed sleep 7
/opt/ssp-mgmt-scripts/asa_cmd_init: fork: Cannot allocate memory
INIT: cannot fork, retry..

/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 6871 Killed sleep 2
/opt/ssp-mgmt-scripts/ssp_heimdall_init: line 89: /usr/bin/expr: Argument list too long
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 6884 Killed sleep 2
/opt/ssp-mgmt-scripts/ssp_heimdall_init: line 90: echo: write error: Invalid argument
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 6886 Killed sleep 2
System Mode Check: MULTI-INSTANCE mode assigned
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 6891 Killed sleep 2
System Mode Check: MULTI-INSTANCE mode assigned
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 6904 Killed sleep 2
/opt/ssp-mgmt-scripts/auth_agent_init: fork: Cannot allocate memory
/opt/ssp-mgmt-scripts/ssp_mi_turbo.sh: line 75: 6916 Killed /bin/sleep 2
/opt/ssp-mgmt-scripts/ssp_heimdall_init: line 139: 6907 Killed /opt/ssp-mgmt-scripts/ssp_mi_turbo.sh

Cisco Firepower Extensible Operating System (FX-OS) Software. TAC support: http://www.cisco.com/tac Copyright (c) 2009-2016, Cisco Systems, Inc. All rights reserved. The copyrights to certain works contained in this software are owned by other third parties and used and distributed under license. Certain components of this software are licensed under the 'GNU General Public License, version 3' provided with ABSOLUTELY NO WARRANTY under the terms of 'GNU General Public License, Version 3', available here: http://www.gnu.org/licenses/gpl.html. See User Manual (''Licensing'') for details. Certain components of this software are licensed under the 'GNU General Public License, version 2' provided with ABSOLUTELY NO WARRANTY under the terms of 'GNU General Public License, version 2', available here: http://www.gnu.org/licenses/old-licenses/gpl-2.0.html. See User Manual (''Licensing'') for details. Certain components of this software are licensed under the 'GNU LESSER GENERAL PUBLIC LICENSE, version 3' provided with ABSOLUTELY NO WARRANTY under the terms of 'GNU LESSER GENERAL PUBLIC LICENSE' Version 3, available here: http://www.gnu.org/licenses/lgpl.html. See User Manual (''Licensing'') for details. Certain components of this software are licensed under the 'GNU Lesser General Public License, version 2.1' provided with ABSOLUTELY NO WARRANTY under the terms of 'GNU Lesser General Public License, version 2', available here: http://www.gnu.org/licenses/old-licenses/lgpl-2.1.html. See User Manual (''Licensing'') for details. Certain components of this software are licensed under the 'GNU Library General Public License, version 2' provided with ABSOLUTELY NO WARRANTY under the terms of 'GNU Library General Public License, version 2', available here: http://www.gnu.org/licenses/old-licenses/lgpl-2.0.html. See User Manual (''Licensing'') for details.
/bi

/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 6971 Killed sleep 2

/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 6973 Killed sleep 2
/opt/ssp-mgmt-scripts/ssp_system_storage.sh: line 50: 6977 Killed ${GET_FEATURE_CONFIG} -f ${HDPARM_LOG_STR} -p ${PLTFM} -v ${HDPARM_LOG_SUPP_VALUE}
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 6978 Killed sleep 2
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 6994 Killed sleep 2
/opt/ssp-mgmt-scripts/ssp_heimdall_init: fork: Cannot allocate memory
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 6997 Killed sleep 2
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 6999 Killed sleep 2
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 7002 Killed sleep 2
/opt/ssp-mgmt-scripts/ssp_system_storage.sh: line 50: 7009 Killed ${GET_FEATURE_CONFIG} -f ${HDPARM_LOG_STR} -p ${PLTFM} -v ${HDPARM_LOG_SUPP_VALUE}
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 7003 Killed sleep 2
/opt/ssp-mgmt-scripts/ssp_heimdall_init: line 91: /usr/bin/awk: Argument list too long
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 7022 Killed sleep 2
/opt/ssp-mgmt-scripts/ssp_heimdall_init: line 91: /bin/cat: Argument list too long
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 7027 Killed sleep 2
System Mode Check: MULTI-INSTANCE mode assigned
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 7028 Killed sleep 2
System Mode Check: MULTI-INSTANCE mode assigned
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 7040 Killed sleep 2
/opt/ssp-mgmt-scripts/ssp_heimdall_init: fork: Cannot allocate memory
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 7047 Killed sleep 2
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 7051 Killed sleep 2
/opt/ssp-mgmt-scripts/ssp_system_mgmt_startup.sh: fork: Cannot allocate memory
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 7059 Killed sleep 2
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 7061 Killed sleep 2
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 7067 Killed sleep 2
/opt/ssp-mgmt-scripts/ssp_heimdall_init: line 206: 7076 Killed ${GET_FEATURE_CONFIG} -f ${FAILSAFE_STR} -p ${PLTFM} -v ${FAILSAFE_SUPP_VALUE}
/opt/ssp-mgmt-scripts/ssp_heimdall_init: line 299: 7044 Killed /opt/ssp-mgmt-scripts/ssp_mi_turbo.sh
/opt/ssp-mgmt-scripts/ssp_heimdall_init: line 299: 7085 Killed $CISCO_BOOTCLI/cisco/cli/bin/coredump.sh 1 "donot_enable_asa_coredump" >> ${CISCO_LOGS_DIR}/stdout_heimdall.log 2>&1
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 7083 Killed sleep 2
/opt/ssp-mgmt-scripts/ssp_system_storage.sh: fork: Cannot allocate memory
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 7095 Killed sleep 2
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 7102 Killed sleep 2
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 7109 Killed sleep 2
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 7114 Killed sleep 2
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 7115 Killed sleep 2
System Mode Check: MULTI-INSTANCE mode assigned
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 7122 Killed sleep 2
System Mode Check: MULTI-INSTANCE mode assigned
/opt/ssp-mgmt-scripts/auth_agent_init: line 194: 7129 Killed sleep 2
/opt/ssp-mgmt-scripts/ssp_heimdall_init: fork: Cannot allocate memory
/opt/ssp-mgmt-scripts/ssp_mi_turbo.sh: line 75: 7138 Killed /bin/sleep 2
/opt/ssp-mgmt-scripts/ssp_mi_turbo.sh: line 75: 7146 Killed /bin/sleep 2
/opt/ssp-mgmt-scripts/ssp_mi_turbo.sh: fork: Cannot allocate memory

Not sure what are these logs, maybe someone can help me about this matter.

Thank you so much.