Cisco 3850 not booting with USB in ROMMON mode

[hammadbilal]

Hi Experts,

My Switch is currently in ROMMON Mode. I am trying to boot the 3850 switch using usb flash drive but I it is not working.

Below are the configuration details in ROMMON Mode:

dir flash:

switch: dir flash:
Directory of flash:/

7745 drwx 4096 .
2 drwx 4096 ..
7746 -rw- 2097152 nvram_config
7747 -rw- 2097152 nvram_config_bkup
7748 -rw- 20097668 cat3k_caa-guestshell.16.12.11.SPA.pkg
7749 -rw- 34849830 cat3k_caa-rpbase.16.12.11.SPA.pkg
7750 -rw- 401218172 cat3k_caa-rpcore.16.12.11.SPA.pkg
7751 -rw- 3418756 cat3k_caa-srdriver.16.12.11.SPA.pkg
7752 -rw- 23022208 cat3k_caa-webui.16.12.11.SPA.pkg
7753 -rw- 4780 packages.conf
38721 drwx 4096 .installer
7754 -rw- 148 bootloader_evt_handle.log
46465 drwx 4096 core
54209 drwx 4096 .prst_sync
69697 drwx 4096 .rollback_timer
54212 drwx 4096 gs_script
77441 drwx 4096 tech_support
77467 drwx 4096 dc_profile_dir
7757 -rw- 76 boothelper.log
7756 -rw- 132108 memleak.tcl
7755 -rw- 76 boothelper.old

1130311680 bytes available (517582848 bytes used)

switch:

dir usbflash0:

switch: dir usbflash0:
Directory of usbflash0:/

4 -rw- 482611915 cat3k_caa-universalk9.16.12.11.SPA.bin

15018516480 bytes available (482648064 bytes used)

switch:

set

switch: set
?=
ABNORMAL_RESET_COUNT=2
ASIC_PCI_RESET=1
AUTOREBOOT_RESTORE=0
BOOT=usbflash0:cat3k_caa-universalk9.16.12.11.SPA.bin
BOOT_LOADER_UPGRADE_DISABLE=0
BSI=0
CFG_MODEL_NUM=WS-C3850-24P-S
CLEI_CODE_NUMBER=IPM8A00ARC
CSR_PCIERST_DISCONNECTED=yes
DEFAULT_ROUTER=192.0.2.1
ECI_CODE_NUMBER=470023
IP_ADDR=192.0.2.123/255.255.255.0
LICENSE_BOOT_LEVEL=ipbasek9,all:ngwc;
MAC_ADDR=cc:8e:71:86:07:00
MANUAL_BOOT=no
MODEL_NUM=WS-C3850-24P
MODEL_REVISION_NUM=AD0
MOTHERBOARD_ASSEMBLY_NUM=73-15805-05
MOTHERBOARD_REVISION_NUM=B0
MOTHERBOARD_SERIAL_NUM=FOC21503GXP
POE1_ASSEMBLY_NUM=73-16439-02
POE1_REVISION_NUM=A0
POE1_SERIAL_NUM=FOC21486HJ0
RANDOM_NUM=276587724
RECOVERY_BUNDLE=sda9:cat3k_caa-recovery.bin
STACK_1_1=2_0
STKPWR_ASSEMBLY_NUM=73-11956-08
STKPWR_REVISION_NUM=B0
STKPWR_SERIAL_NUM=FOC2147A9EE
SWITCH_IGNORE_STARTUP_CFG=0
SWITCH_IGNORE_STRATUP_CFG=0
SWITCH_NUMBER=1
SYSTEM_SERIAL_NUM=FCW2151C091
TAN_NUM=800-43046-03
TAN_REVISION_NUMBER=D0
TEMPLATE=advanced
TERMLINES=0
USB_ASSEMBLY_NUM=73-16576-01
USB_REVISION_NUM=A0
USB_SERIAL_NUM=FOC21476M9W
VERSION_ID=V08
boot=usbflash0:cat3k_caa-universalk9.16.12.11.SPA.bin
switch_ignore_startup_cfg=0

Once I start to boot from usb, I am getting the below output.

switch: boot usbflash0:cat3k_caa-universalk9.16.12.11.SPA.bin
Reading full image into memory................................................................................................................................................................................................................................................................................................................................................................................................................................................................................done
Bundle Image
--------------------------------------
Kernel Address : 0x558d56e8
Kernel Size : 0x43857f/4425087
Initramfs Address : 0x55d0dc67
Initramfs Size : 0x1cab2e3/30061283
Compression Format: mzip

Bootable image at @ ram:0x558d56e8
Bootable image segment 0 address range [0x81100000, 0x81da6400] is in range [0x80180000, 0x90000000].
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
File "usbflash0:cat3k_caa-universalk9.16.12.11.SPA.bin" uncompressed and installed, entry point: 0x818968e0
Loading Linux kernel with entry point 0x818968e0 ...
Bootloader: Done loading app on core_mask: 0xf

### Launching Linux Kernel (flags = 0x5)

Linux version 4.9.187 (deeratho@sjc-ads-7586) (gcc version 5.3.0 (GCC) ) #1 SMP Sun Jul 2 23:04:41 PDT 2023
CVMSEG size: 2 cache lines (256 bytes)
Cavium Inc. SDK-5.1.0
bootconsole [early0] enabled
CPU0 revision is: 000d900a (Cavium Octeon II)
Checking for the multiply/shift bug... no.
Checking for the daddiu bug... no.
%IOSXEBOOT-65259848eff01324a205b969dcebb486-new_cksum: (rp/0): 4
%IOSXEBOOT-65259848eff01324a205b969dcebb486-saved_cksum: (rp/0): 4

Final tar file: mcu_ucode_bundle_6_2_0.tar

Both links down, not waiting for other switches
Switch number is 1

Chassis 1 reloading, reason - Active/standby selection failed in 1+1 Mode
reload fp action requested
rp processes exit with reload switch code

% FACTORYRESET - Exiting without calling factory_reset
watchdog: watchdog0: watchdog did not stop!
reboot: Restarting system

Booting...
Interface GE 0 link down***ERROR: PHY link is down
Reading full image into memory........

and this continues until I stop this by pressing and hold the mode button

I also tried with Emergency -install method but the result is same.

Any suggestions, what could be the issue and solution to this.

[Leo Laohoo]

Post the complete output to the following ROMMON command:  

unset ALL
boot flash:packages.conf

[hammadbilal]

Below is the complete output:

switch: boot flash:packages.conf

Getting rest of image
Reading full image into memory...Check base package header ...: done = 16384
Getting rest of image
Reading full image into memory....done
Reading full base package into memory...: done = 34849830
Bundle Image
--------------------------------------
Kernel Address : 0x703da3c4
Kernel Size : 0x43857f/4425087
Initramfs Address : 0x70812943
Initramfs Size : 0x1cab2e3/30061283
Compression Format: mzip

Bootable image at @ ram:0x703da3c4
Bootable image segment 0 address range [0x81100000, 0x81da6400] is in range [0x80180000, 0x90000000].
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@boot_system: 623
Loading Linux kernel with entry point 0x818968e0 ...
Bootloader: Done loading app on core_mask: 0xf

### Launching Linux Kernel (flags = 0x5)

Linux version 4.9.187 (deeratho@sjc-ads-7586) (gcc version 5.3.0 (GCC) ) #1 SMP Sun Jul 2 23:04:41 PDT 2023
CVMSEG size: 2 cache lines (256 bytes)
Cavium Inc. SDK-5.1.0
bootconsole [early0] enabled
CPU0 revision is: 000d900a (Cavium Octeon II)
Checking for the multiply/shift bug... no.
Checking for the daddiu bug... no.
%IOSXEBOOT-65259848eff01324a205b969dcebb486-new_cksum: (rp/0): 4
%IOSXEBOOT-65259848eff01324a205b969dcebb486-saved_cksum: (rp/0): 4

Final tar file: mcu_ucode_bundle_6_2_0.tar

Both links down, not waiting for other switches
Switch number is 1

Chassis 1 reloading, reason - Active/standby selection failed in 1+1 Mode
reload fp action requested
rp processes exit with reload switch code

% FACTORYRESET - Exiting without calling factory_reset
watchdog: watchdog0: watchdog did not stop!
reboot: Restarting system

Booting...
Interface GE 0 link down***ERROR: PHY link is down
Reading full image into memory................................................................................................................................................................................................................................................................................................................................................................................................................................................................................done
Bundle Image
--------------------------------------
Kernel Address : 0x537946e8
Kernel Size : 0x43857f/4425087
Initramfs Address : 0x53bccc67
Initramfs Size : 0x1cab2e3/30061283
Compression Format: mzip

Bootable image at @ ram:0x537946e8
Bootable image segment 0 address range [0x81100000, 0x81da6400] is in range [0x80180000, 0x90000000].
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
File "usbflash0:cat3k_caa-universalk9.16.12.11.SPA.bin" uncompressed and installed, entry point: 0x818968e0
Loading Linux kernel with entry point 0x818968e0 ...
Bootloader: Done loading app on core_mask: 0xf

### Launching Linux Kernel (flags = 0x5)

Linux version 4.9.187 (deeratho@sjc-ads-7586) (gcc version 5.3.0 (GCC) ) #1 SMP Sun Jul 2 23:04:41 PDT 2023
CVMSEG size: 2 cache lines (256 bytes)
Cavium Inc. SDK-5.1.0
bootconsole [early0] enabled
CPU0 revision is: 000d900a (Cavium Octeon II)
Checking for the multiply/shift bug... no.
Checking for the daddiu bug... no.
%IOSXEBOOT-65259848eff01324a205b969dcebb486-new_cksum: (rp/0): 4
%IOSXEBOOT-65259848eff01324a205b969dcebb486-saved_cksum: (rp/0): 4

Final tar file: mcu_ucode_bundle_6_2_0.tar

Both links down, not waiting for other switches
Switch number is 1

Chassis 1 reloading, reason - Active/standby selection failed in 1+1 Mode
reload fp action requested
rp processes exit with reload switch code

% FACTORYRESET - Exiting without calling factory_reset
watchdog: watchdog0: watchdog did not stop!
reboot: Restarting system

Booting...
Interface GE 0 link down***ERROR: PHY link is down
Reading full image into memory................................................................................................................................................................................................................................................................................................................................................................................................................................................................................done
Bundle Image
--------------------------------------
Kernel Address : 0x537946e8
Kernel Size : 0x43857f/4425087
Initramfs Address : 0x53bccc67
Initramfs Size : 0x1cab2e3/30061283
Compression Format: mzip

Bootable image at @ ram:0x537946e8
Bootable image segment 0 address range [0x81100000, 0x81da6400] is in range [0x80180000, 0x90000000].
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
File "usbflash0:cat3k_caa-universalk9.16.12.11.SPA.bin" uncompressed and installed, entry point: 0x818968e0
Loading Linux kernel with entry point 0x818968e0 ...
Bootloader: Done loading app on core_mask: 0xf

### Launching Linux Kernel (flags = 0x5)

Linux version 4.9.187 (deeratho@sjc-ads-7586) (gcc version 5.3.0 (GCC) ) #1 SMP Sun Jul 2 23:04:41 PDT 2023
CVMSEG size: 2 cache lines (256 bytes)
Cavium Inc. SDK-5.1.0
bootconsole [early0] enabled
CPU0 revision is: 000d900a (Cavium Octeon II)
Checking for the multiply/shift bug... no.
Checking for the daddiu bug... no.
%IOSXEBOOT-65259848eff01324a205b969dcebb486-new_cksum: (rp/0): 4
%IOSXEBOOT-65259848eff01324a205b969dcebb486-saved_cksum: (rp/0): 4

Final tar file: mcu_ucode_bundle_6_2_0.tar

Both links down, not waiting for other switches
Switch number is 1

Chassis 1 reloading, reason - Active/standby selection failed in 1+1 Mode
Switch rebooted continuously for atleast 5 times. Disabling autoreboot
reload fp action requested
rp processes exit with reload switch code

% FACTORYRESET - Exiting without calling factory_reset
watchdog: watchdog0: watchdog did not stop!
reboot: Restarting system

Booting...
Interface GE 0 link down***ERROR: PHY link is down

The system is not configured to boot automatically. The
following command will finish loading the operating system
software:

boot

switch:

[Leo Laohoo]

Try the ROMMON command: 

unset ALL

If this does not work, perform an emergency install again.

[hammadbilal]

I did the steps again but same result.

Check the below output....

switch: unset All
Variable All" not set, or is readonly.

switch: emergency-install usbflash0:cat3k_caa-universalk9.16.12.11.SPA.bin
The bootflash will be erased during install operation, continue (y/n)?y
Starting emergency recovery (usbflash0:cat3k_caa-universalk9.16.12.11.SPA.bin)...
Reading full image into memory........................done
Bundle Image
--------------------------------------
Kernel Address : 0x537795d4
Kernel Size : 0x3faba9/4172713
Initramfs Address : 0x53b74180
Initramfs Size : 0xdf2d3b/14626107
Compression Format: .mzip

Bootable image at @ ram:0x537795d4
Bootable image segment 0 address range [0x81100000, 0x820a0000] is in range [0x80180000, 0x90000000].
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
File "sda9:cat3k_caa-recovery.bin" uncompressed and installed, entry point: 0x816733a0
Loading Linux kernel with entry point 0x816733a0 ...
Bootloader: Done loading app on core_mask: 0xf

### Launching Linux Kernel (flags = 0x5)

Initiating Emergency Installation of bundle usbflash0:cat3k_caa-universalk9.16.12.11.SPA.bin

Reading bundle usbflash0:cat3k_caa-universalk9.16.12.11.SPA.bin...

Validating bundle usbflash0:cat3k_caa-universalk9.16.12.11.SPA.bin...
Installing bundle usbflash0:cat3k_caa-universalk9.16.12.11.SPA.bin...
Verifying bundle usbflash0:cat3k_caa-universalk9.16.12.11.SPA.bin...
Package cat3k_caa-guestshell.16.12.11.SPA.pkg is Digitally Signed
Package cat3k_caa-rpbase.16.12.11.SPA.pkg is Digitally Signed
Package cat3k_caa-rpcore.16.12.11.SPA.pkg is Digitally Signed
Package cat3k_caa-srdriver.16.12.11.SPA.pkg is Digitally Signed
Package cat3k_caa-webui.16.12.11.SPA.pkg is Digitally Signed
Preparing flash...
Syncing device...
Emergency Install successful... Rebooting
Restarting system.

Booting...
Interface GE 0 link down***ERROR: PHY link is down

The system is not configured to boot automatically. The
following command will finish loading the operating system
software:

boot

switch:

Now the switch configuration is changed after applying emergency install.

switch: set
?=
ABNORMAL_RESET_COUNT=0
ASIC_PCI_RESET=1
AUTOREBOOT_RESTORE=1
BOOT=flash:packages.conf
BOOT_LOADER_UPGRADE_DISABLE=0
BSI=0
CFG_MODEL_NUM=WS-C3850-24P-S
CLEI_CODE_NUMBER=IPM8A00ARC
CSR_PCIERST_DISCONNECTED=yes
DEFAULT_ROUTER=192.0.2.1
ECI_CODE_NUMBER=470023
IP_ADDR=192.0.2.123/255.255.255.0
LICENSE_BOOT_LEVEL=ipbasek9,all:ngwc;
MAC_ADDR=cc:8e:71:86:07:00
MANUAL_BOOT=yes
MODEL_NUM=WS-C3850-24P
MODEL_REVISION_NUM=AD0
MOTHERBOARD_ASSEMBLY_NUM=73-15805-05
MOTHERBOARD_REVISION_NUM=B0
MOTHERBOARD_SERIAL_NUM=FOC21503GXP
POE1_ASSEMBLY_NUM=73-16439-02
POE1_REVISION_NUM=A0
POE1_SERIAL_NUM=FOC21486HJ0
RANDOM_NUM=276587724
RECOVERY_BUNDLE=sda9:cat3k_caa-recovery.bin
STACK_1_1=2_0
STKPWR_ASSEMBLY_NUM=73-11956-08
STKPWR_REVISION_NUM=B0
STKPWR_SERIAL_NUM=FOC2147A9EE
SWITCH_IGNORE_STARTUP_CFG=0
SWITCH_IGNORE_STRATUP_CFG=0
SWITCH_NUMBER=1
SYSTEM_SERIAL_NUM=FCW2151C091
TAN_NUM=800-43046-03
TAN_REVISION_NUMBER=D0
TEMPLATE=advanced
TERMLINES=0
USB_ASSEMBLY_NUM=73-16576-01
USB_REVISION_NUM=A0
USB_SERIAL_NUM=FOC21476M9W
VERSION_ID=V08
boot=usbflash0:cat3k_caa-universalk9.16.12.11.SPA.bin
switch_ignore_startup_cfg=0

switch:

[Leo Laohoo · Accepted solution]

set MANUAL_BOOT=no
unset SWITCH_IGNORE_STRATUP_CFG=0
unset boot=usbflash0:cat3k_caa-universalk9.16.12.11.SPA.bin
unset switch_ignore_startup_cfg=0
unset STACK_1_1=2_0

And then reboot again.

[hammadbilal]

That worked for me. The switch reboot with IOS was successful.

Thanks for the help.

Appreciated.