Can't login to privilege level 15 user after firmware upgrade

[Dox2163]

Before the upgrade, I was able to SSH into a level 15 user and it would land me directly to # without using enable.

After upgrading to from 16.3.8 to 16.9.4, my level 2 account can still SSH in but level 15 user account gets % login invalid. So I try going from level 2 and then enable, then it says % No password set. This is a cat 3850

Any help appreciated. I haven't had a chance to console yet.

[Richard Burts]

I am not clear whether the no password set error is generated when you attempt to login for your level 2 account or whether your login for level 2 is successful and the error is generated when you attempt to use the enable command. Can you clarify?

I hope your console access is successful when you try it. But I am concerned that it may have the same issue. If it is successful then please get a copy of the current running config and post it (after obscuring sensitive information such as passwords and public IP).

HTH

Rick

[Dox2163]

Login to level 2 is successful, the no password set error appears when I enter enable or enable 15.

[RicardoSN · Accepted solution]

Hello Dox2163,

According to Cisco's Release notes for 16.9.4 there are no new software features.

That's a long versions jump though... It's possible that in between versions 16.3.8 to 16.9.4 there was an update on some management configurations that could cause this.

Anyways, getting back to your issue. The "% login invalid" will prompt when the user or password typed are not configured on the Switch you're trying to access. Being that the case, it would depend on which authentication method you are using why is this happening, it may be a Tacacs+ or Radius server related issue or something else.

If you are using Local users the most probable scenario is that for some reason your level 15 user got lost in the limbo (deleted) and you most likely will have to perform a Password Recovery to re configure it.

Here is a procedure for Cisco Catalyst 3850.

https://www.rogerperkin.co.uk/cisco/switching/3850/password-recovery-cisco-catalyst-3850/

If you're not that near to the Switch to perform a Password Recovery locally, you could also try this procedure. (Only if the Switch has SNMP configured).

https://www.cisco.com/c/en/us/support/docs/ip/simple-network-management-protocol-snmp/15217-copy-configs-snmp.html

Good Luck!

[Dox2163 · Accepted solution]

Thank you for everyone's help.

I was able to get in via console. I immediately checked the show run and all users were gone except the one level 2 user.

I recreated my users and was able to downgrade the firmware.

I was going to perform a password recovery as ricardoSN suggested if console didn't work. Can I accept multiple solutions?

[Richard Burts]

Thanks for the update letting us know that your console access was successful and allowed you to identify and correct the problem. That is good to know. 

Yes it is possible to accept more than one answer as a solution.

HTH

Rick

[mnandira@cisco.com]

switch: boot

boot: attempting to boot from [flash:cat9k_iosxe.17.03.04.SPA.bin]

boot: reading file cat9k_iosxe.17.03.04.SPA.bin

switch: 

switch: 

switch: boot

boot: attempting to boot from [flash:cat9k_iosxe.17.03.04.SPA.bin]

boot: reading file cat9k_iosxe.17.03.04.SPA.bin

#########################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################

Both links down, not waiting for other switches

Switch number is 1

              Restricted Rights Legend

Use, duplication, or disclosure by the Government is

subject to restrictions as set forth in subparagraph

(c) of the Commercial Computer Software - Restricted

Rights clause at FAR sec. 52.227-19 and subparagraph

(c) (1) (ii) of the Rights in Technical Data and Computer

Software clause at DFARS sec. 252.227-7013.

           Cisco Systems, Inc.

           170 West Tasman Drive

           San Jose, California 95134-1706

Cisco IOS Software [Amsterdam], Catalyst L3 Switch Software (CAT9K_IOSXE), Version 17.3.4, RELEASE SOFTWARE (fc3)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2021 by Cisco Systems, Inc.

Compiled Sat 03-Jul-21 01:55 by mcpre

This software version supports only Smart Licensing as the software licensing mechanism.

PLEASE READ THE FOLLOWING TERMS CAREFULLY. INSTALLING THE LICENSE OR

LICENSE KEY PROVIDED FOR ANY CISCO SOFTWARE PRODUCT, PRODUCT FEATURE,

AND/OR SUBSEQUENTLY PROVIDED SOFTWARE FEATURES (COLLECTIVELY, THE

"SOFTWARE"), AND/OR USING SUCH SOFTWARE CONSTITUTES YOUR FULL

ACCEPTANCE OF THE FOLLOWING TERMS. YOU MUST NOT PROCEED FURTHER IF YOU

ARE NOT WILLING TO BE BOUND BY ALL THE TERMS SET FORTH HEREIN.

Your use of the Software is subject to the Cisco End User License Agreement

(EULA) and any relevant supplemental terms (SEULA) found at

http://www.cisco.com/c/en/us/about/legal/cloud-and-software/software-terms.html.

You hereby acknowledge and agree that certain Software and/or features are

licensed for a particular term, that the license to such Software and/or

features is valid only for the applicable term and that such Software and/or

features may be shut down or otherwise terminated by Cisco after expiration

of the applicable license term (e.g., 90-day trial period). Cisco reserves

the right to terminate any such Software feature electronically or by any

other means available. While Cisco may provide alerts, it is your sole

responsibility to monitor your usage of any such term Software feature to

ensure that your systems and networks are prepared for a shutdown of the

Software feature.

FIPS: Flash Key Check : Key Not Found, FIPS Mode Not Enabled

All TCP AO KDF Tests Pass

cisco C9300-48UXM (X86) processor with 1331366K/6147K bytes of memory.

Processor board ID FOC2225U0RE

2048K bytes of non-volatile configuration memory.

8388608K bytes of physical memory.

1638400K bytes of Crash Files at crashinfo:.

11264000K bytes of Flash at flash:.

Base Ethernet MAC Address          : 6c:6c:d3:d1:e9:00

Motherboard Assembly Number        : 73-17959-06

Motherboard Serial Number          : FOC222218PZ

Model Revision Number              : D0

Motherboard Revision Number        : A0

Model Number                       : C9300-48UXM

System Serial Number               : FOC2225U0RE

CLEI Code Number                   : 

Press RETURN to get started!

######## KEY IS TO WAIT HERE TILL THE IOS IMAGE GETS LOADED TILL YOU GET THE BELOW SWITCH PROMPT #####

*Switch>en

Switch#

Switch#

Switch#

Switch#

Switch#

Switch#copy start

Switch#copy startup-config runn

Switch#copy startup-config running-config

Washinton-SR-c9300#. ===>>.

[paul driver]

Hello
Sounds like your exec authorization hasnt been appended correctly either at line level or using AAA
So are you using local authentication and if so is this with AAA or without?

Can you post you run configuration please

sh run | in username|aaa|line

[Dox2163]

This is the result from above command. After the upgrade, it seems all my users were wiped out except the that one level 2 user.

no aaa new-model
username xxxxxxx privilege 2 secret 5 xxxxxxxxxxxxxxxxxx
line con 0
line aux 0
line vty 0 4
line vty 5 15

[Georg Pauwen]

Hello,

on a side note, and you might have already tried this, you could delete and reenter the entire SSH related information including the domain name, and zeroizing and reapplying the RSA key. After that, reload the device...

[iamrobbormai]

Maybe this will help someone else in the future. I saw something similar upgrading 16.3 to 16.9.4. We had passwords saved as secret 4. The stack I updated at this level removed the user during the upgrade process. I tested on a different stack and found if I recreated the user and the password was at secret 5, the user was not removed during the upgrade process.

I too was able to access the switch via the console to add the users after they were removed during upgrade.

[Richard Burts]

That is an interesting observation, that the behavior might be related to users with secret 4. Thanks for sharing it. +5 for the contribution.

HTH

Rick

[vishal.c07]

Hello everyone,

Problem: Not able to login into Cisco Switch after firmware upgrade

Symptoms:

I've faced same issue, after troubleshooting I've found that user or enable password with "secret 4" encryption are getting vanished after firmware upgradation
Example : username admin privilege 15 secret 4 *******

Solution: Reconfigure username password before upgradation and make sure it is not "secrete 4"

Note: SNMP configuration also vanish in my case.

Regards

Vishal

[vishal.c07]

Hello everyone,

Problem: Not able to login into Cisco Switch after firmware upgrade

Symptoms:

I've faced same issue, after troubleshooting I've found that user or enable password with "secret 4" encryption are getting vanished after firmware upgradation
Example : username admin privilege 15 secret 4 *******

Solution: Reconfigure username password before upgradation and make sure it is not "secrete 4"

Note: SNMP configuration also vanish in my case.

Regards

Vishal