Cisco 9300 Password Recovery not working

[Omar1212]

Hi All,

I am currently stuck in a situation where I am locked out of a stack of 9300 switches.
I have copied the username and the hashed password from a backup config that we had as this is a new switch model than the current one we have.
Switching, Catalyst 9000, Other Switches, LAN SwitchingSwitching, Catalyst 9000, Other Switches, LAN SwitchingSwitching, Catalyst 9000, Other Switches, LAN Switching
The problem I have a the moment and I am not sure how it happened is that I have been locked out of the switch as the password is not working.

I have tried rommon access through pressing the mode button a couple of time after the system light reboots and that gets me into the bootloader.

With "switch:" prompt  have tried as suggested "SWITCH_IGNORE_STARTUP_CFG=1" and copied and pasted that command then ran "boot" to reboot the stack, however the stack still boots with the start up config like I never even put the command in.

I am a bit lost on what to do at this point and I cant seem to find anyone who has had the same issue 

Any help is appreciated . Thanks

[balaji.bandi]

the steps  should work :

https://medium.com/@jennb0111/password-recovery-for-cisco-catalyst-9300-switch-fca21e88b6d

http://www.jay-miah.co.uk/password-recovery-on-cisco-9300-series-switch/

[Omar1212]

Thanks for the reply. As mentioned in the post I have tried this and the command ""SWITCH_IGNORE_STARTUP_CFG=1" does not seem to take any effect.

[balaji.bandi]

May be there is small command missing while doing, can you capture the console output full and post us for reference, what is going wrong to identify (right from boot)

[Omar1212]

Thanks I am accessing the switch via RDP and console so I have no one on site at the moment to further reboot and get back into ROMMON .  However I was able to extract this from the putty terminal session as I left it open. Please find below:

Press RETURN to get started.

▒
Initializing Hardware...

Correcting the primary Nvram
Correcting the primary Nvram
System Bootstrap, Version 16.12.2r, RELEASE SOFTWARE (P)
Compiled Fri 10/25/2019 11:48:55.48 by rel

Current ROMMON image : Primary
Last reset cause : PowerOn
C9300L-24T-4X platform with 8388608 Kbytes of main memory

boot: attempting to boot from [flash:packages.conf] (interrupted)
switch: SWITCH_IGNORE_STARTUP_CFG=1
switch: SWITCH_IGNORE_STARTUP_CFG=1
switch: dir flash:

Size Attributes Name
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
2097152 -rw- nvram_config
2097152 -rw- nvram_config_bkup
14464008 -rw- cat9k-cc_srdriver.16.12.04.SPA.pkg
89134084 -rw- cat9k-espbase.16.12.04.SPA.pkg
1684484 -rw- cat9k-guestshell.16.12.04.SPA.pkg
531063808 -rw- cat9k-rpbase.16.12.04.SPA.pkg
31478788 -rw- cat9k-sipbase.16.12.04.SPA.pkg
60392448 -rw- cat9k-sipspa.16.12.04.SPA.pkg
23217156 -rw- cat9k-srdriver.16.12.04.SPA.pkg
12907520 -rw- cat9k-webui.16.12.04.SPA.pkg
9216 -rw- cat9k-wlc.16.12.04.SPA.pkg
6648 -rw- packages.conf
41799673 -rw- cat9k-rpboot.16.12.04.SPA.pkg
4096 drw- .installer
444 -rw- bootloader_evt_handle.log
4096 drw- core
4096 drw- .prst_sync
4096 drw- .rollback_timer
4096 drw- gs_script
4096 drw- tech_support
4096 drw- dc_profile_dir
395 -rw- boothelper.log
132108 -rw- memleak.tcl
1953 -rw- svl_ipc.tcl
0 -rw- NVRAM
4096 drw- .dbpersist
4096 drw- onep
16089 -rw- rdope_out.txt
89 -rw- rdope.log
0 -rw- dope_hist
35 -rw- pnp-tech-time
108342 -rw- pnp-tech-discovery-summary
1036 -rw- vlan.dat
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

switch:
switch: SWITCH_IGNORE_STARTUP_CFG=1
switch: boot
boot: attempting to boot from [flash:packages.conf]
boot: reading file packages.conf
#
#########################################################################################################################################################################################################################################################################################################################################################################################################################

Waiting for 120 seconds for other switches to boot

Switch number is 2
All switches in the stack have been discovered. Accelerating discovery

[Leo Laohoo]

---

@Omar1212 wrote:

All switches in the stack have been discovered. Accelerating discovery

---

Is this switch still connected with a stacking cable?

[Omar1212]

yes currently they are still in the stack. I did read online and it mentioned as long as I do the same procedure on the master it should go be the same. Do you think I need to do every single switch on its own ?

[Leo Laohoo]

---

@Omar1212 wrote:

yes currently they are still in the stack.

---

OMfG.  <FACEPALM>

That is why password recovery will not work because they are still in a stack!

If the master has the complete config, it does not matter if the slave switch has password-recovery configured.  As soon the switch detects the master through the stacking cable, the switch will download the config and rendering the password-recovery settings moot.

[Omar1212]

right !

So do you recommend removing them from the stack and doing each switch on their own.

[Leo Laohoo · Accepted solution]

Remove the stacking cable. 

Perform the password-recovery. 

[Omar1212]

Thanks you are right . consoling onto the other switch and I found out that its up with the config .

I disconnected the stacking cables and done the password recovery on the master and that has worked.

Thanks for your help

[Leo Laohoo]

Happy to help.

[Leo Laohoo]

---

@Omar1212 wrote:

With "switch:" prompt  have tried as suggested "SWITCH_IGNORE_STARTUP_CFG=1" and copied and pasted that command then ran "boot" to reboot the stack, however the stack still boots with the start up config like I never even put the command in.

---

After entering this command, post the complete output to the ROMMON command of "set".

[Omar1212]

Thanks for the reply I will get the output as soon as possible.

I was getting access to the switch through RDP  to a PC on site while having someone reboot the switch for me. as soon as I get that I will update here.

[Omar1212]

Hi  Please find below the output of the "set" command : 

I noticed the switch number is saying 2 however as far as I know you cant console on the on master switch.

Do you think its related to that ?

Thanks.

BAUD=9600
BOARDID=24677
BOOT=flash:packages.conf;
BOOTLDR=
BSI=0
CONFIG_FILE=
MAC_ADDR=F8:7A:41:46:8A:00
MANUAL_BOOT=no
MODEL_NUM=C9300L-24T-4X
MODEL_REVISION_NUM=F0
MOTHERBOARD_ASSEMBLY_NUM=73-19168-04
MOTHERBOARD_REVISION_NUM=C0
MOTHERBOARD_SERIAL_NUM=FOC2509008P
RANDOM_NUM=1400769744
RET_2_RCALTS=
ROMMON_AUTOBOOT_ATTEMPT=3
SWITCH_IGNORE_STARTUP_CFG=1
SWITCH_NUMBER=2
SWITCH_PRIORITY=13
SYSTEM_SERIAL_NUM=FOC2509L7HD
TEMPLATE=access
VERSION_ID=V01