Unable to recover a 9300LM switch

[rasmus.elmholt]

Hi all,

I have by mistake tried to install version 17.06.05 on a 9300LM switch with the result that it is not able to bootup.

I have now tried to make a recovery using USB and FTP but none of it works.

The strange part is that whenever I try to boot from FTP the switch gives me an error and says the file checksum is incorrect.

But when I make a SHA512 on the file on the FTP server it is the same as from software.cisco.com.

switch: dir bootflash:

Size       Attributes Name
 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
2097152      -rw-     nvram_config
2097152      -rw-     nvram_config_bkup
4096         drw-     sss
381          -rw-     boothelper.log
1027574083   -rw-     cat9k_iosxe.17.06.05.SPA.bin
7488         -rw-     cat9k_iosxe.17.06.05.SPA.conf
19760140     -rw-     cat9k-cc_srdriver.17.06.05.SPA.pkg
110638088    -rw-     cat9k-espbase.17.06.05.SPA.pkg
2098184      -rw-     cat9k-guestshell.17.06.05.SPA.pkg
9220         -rw-     cat9k-lni.17.06.05.SPA.pkg
698438660    -rw-     cat9k-rpbase.17.06.05.SPA.pkg
37864456     -rw-     cat9k-sipbase.17.06.05.SPA.pkg
17523716     -rw-     cat9k-webui.17.06.05.SPA.pkg
7488         -rw-     packages.conf
61326340     -rw-     cat9k-sipspa.17.06.05.SPA.pkg
4096         drw-     SHARED-IOX
0            -rw-     mode_event_log
4096         drw-     .installer
4096         drw-     core
4096         drw-     .USWAP
4096         drw-     .CRFT
350          -rw-     mcelog.txt
962          -rw-     bootloader_evt_handle.log
4096         drw-     .prst_sync
4096         drw-     .rollback_timer
4096         drw-     gs_script
5242880      -rw-     .ssd
5242880      -rw-     ssd
4096         drw-     tech_support
4096         drw-     dc_profile_dir
4096         drw-     EDScisco
4096         drw-     sys_report
4096         drw-     fp_cc_crash
134899       -rw-     memleak.tcl
2049         -rw-     svl_ipc.tcl
4096         drw-     Tbot
0            -rw-     NVRAM
1338         -rw-     DigiCertGlobalRootCA.ca
1924         -rw-     ISRGRootX1CrossSignDST.ca
1606         -rw-     StarfieldServicesRootCA.ca
1261         -rw-     Baltimore.ca
4096         drw-     .rommon_sync
4096         drw-     .dbpersist
14707        -rw-     rdope_out.txt
89           -rw-     rdope.log
4096         drw-     pnp-info
4096         drw-     onep
4096         drw-     guest-share
0            -rw-     dope_hist
4096         drw-     pnp-tech
9220         -rw-     cat9k-wlc.17.06.05.SPA.pkg
2328         -rw-     2023_10_26T06_11_46.994533.xml
4096         drw-     .images
32056328     -rw-     cat9k-srdriver.17.06.05.SPA.pkg
48649463     -rw-     cat9k-rpboot.17.06.05.SPA.pkg
1198         -rw-     .packages.verify.log
151          -rw-     .iox_alt_hdd_cleanup.log
308          -rw-     .bootpackages.verify.log
1048         -rw-     preboothelper.log
116          -rw-     .mb_fpgapackages.log
2554         -rw-     fpgahelper.log
1165         -rw-     mcu_debug.txt
152          -rw-     .fpga_upg_run.log
27255820     -rw-     cat9k-cc_srdriver.17.09.04a.SPA.pkg
173560840    -rw-     cat9k-espbase.17.09.04a.SPA.pkg
2155532      -rw-     cat9k-guestshell.17.09.04a.SPA.pkg
9220         -rw-     cat9k-lni.17.09.04a.SPA.pkg
817918984    -rw-     cat9k-rpbase.17.09.04a.SPA.pkg
51749896     -rw-     cat9k-sipbase.17.09.04a.SPA.pkg
65844232     -rw-     cat9k-sipspa.17.09.04a.SPA.pkg
41329672     -rw-     cat9k-srdriver.17.09.04a.SPA.pkg
18818052     -rw-     cat9k-webui.17.09.04a.SPA.pkg
9220         -rw-     cat9k-wlc.17.09.04a.SPA.pkg
7594         -rw-     cat9k_iosxe.17.09.04a.SPA.conf
53866919     -rw-     cat9k-rpboot.17.09.04a.SPA.pkg
 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
switch: dir usbflash0:
media drive usbflash0: not present.
switch:
switch: IP_ADDRESS=10.36.22.10
switch: IP_SUBNET_MASK=255.255.255.0
switch: DEFAULT_GATEWAY=10.36.22.1
switch: ping 10.4.4.44
Pinging 10.4.4.44, 4 time(s), with packet-size 16
 service type   : 0
 total length   : 9216 bytes
 identification : 46369
 fragmentation  : 0



 time to live   : 59
 protocol       : 1
 source         : 10.4.4.44
 destination    : 10.36.22.10
16 bytes from 10.4.4.44: ICMPv4 seq#1 RTT=218 ms

switch: boot ftp://10.4.4.44/Cisco-IOS/9300LM/cat9k_iosxe.17.09.04a.SPA2.bin
boot: attempting to boot from [ftp://10.4.4.44/Cisco-IOS/9300LM/cat9k_iosxe.17.09.04a.SPA2.bin]
h/w (environment):
 mac       : E8:0A:B9:28:6A:00
n/w (environment):
 ip        : 10.36.22.10
 mask      : 255.255.255.0
 gateway   : 10.36.22.1

h/w:
 interface : eth0 (Ethernet)
 mac       : E8:0A:XX:XX:XX:XX
n/w (ip v4):
 ip        : 10.36.22.10
 mask      : 255.255.255.0
 route(s)  : 0.0.0.0 -> 10.36.22.0/255.255.255.0
           : 10.36.22.1 -> 0.0.0.0/0.0.0.0
n/w (ip v6):
 ip(s)     : FE80::EA0A:XXFF:FEXX:XXXX/64
 route(s)  : :: -> FE80::/64

 server    : 10.4.4.44
 file      : Cisco-IOS/9300LM/cat9k_iosxe.17.09.04a.SPA2.bin
 blocksize : 4096
network: static IPv4 address 10.36.22.10
network: static IPv4 gateway 10.36.22.1
Error Success
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Integrity Check Failed:
        calculated DA71D33F:3F4F64AC:90AA62B8:8B71B3ED:719ECA65
        expected   F5CCF256:B2F4BE65:ED6A414D:9BFA4D8D:4605553F
ERROR: failed to boot from ftp://10.4.4.44/Cisco-IOS/9300LM/cat9k_iosxe.17.09.04a.SPA2.bin (Aborted)
switch:

 I am unable to get the file over from an USB stick as well as it will not recognise my 32GB FAT32 formatted USB key in ROMMON.

[rasmus.elmholt]

And I have been through most of this: https://www.cisco.com/c/en/us/support/docs/switches/catalyst-9300-series-switches/217607-troubleshoot-and-recover-catalyst-9000-s.html

[balaji.bandi]

So you looking to Switch to boot 17.9.4a ?

Check what is the content of

- Move the File packages.conf to packages.back

copy  cat9k_iosxe.17.09.04a.SPA.conf to packages.conf

:boot packages.conf  (post complete boot log here)

[rasmus.elmholt]

The flash and bootflash filesystems are Read-Only in ROMMON, so it is not possible to move or change the files.

switch: dir
usage:
        dir <path-to-directory>
supported filesystem(s):
         flash: ro flash
     bootflash: ro system partition
     usbflash0: rw front-panel USB
     usbflash2: rw front-panel USB C

[Leo Laohoo]

While in ROMMON, enter the following command: 

boot flash:cat9k_iosxe.17.09.04a.SPA.conf

[rasmus.elmholt]

I tried this but it just hang during boot. It loaded the image and showed me a lot of #-signs but it newer booted into the software. I have two 9300LM switches with the same issue, and none of them are booting.

[Leo Laohoo]

---

@rasmus.elmholt wrote:
I have two 9300LM switches with the same issue, and none of them are booting.

---

One not booting, understandable.  Two or more? 

Post the complete output to the ROMMON command "set".

[rasmus.elmholt · Accepted solution]

I have two switches with the same issue and error. I think my issue with this is that it is not able to boot from ftp because of some error that does not exist as the file has been verified:

Integrity Check Failed:
        calculated DA71D33F:3F4F64AC:90AA62B8:8B71B3ED:719ECA65
        expected   F5CCF256:B2F4BE65:ED6A414D:9BFA4D8D:4605553F
ERROR: failed to boot from ftp://10.4.4.44/Cisco-IOS/9300LM/cat9k_iosxe.17.09.04a.SPA2.bin (Aborted)

 Today I managed to find a 8GB USB key and boot from the .bin file on there, and then one-shot CLI into install mode.