Username and password no longer working

[streaming]

Hi experts!

I have a stack of two Cisco SG550XG 10G Switch 24 Port and everything worked fine during install and setup a year ago and also for a while after that.

I have lost access since about half a year and the configured username and password has stopped working - on browser, SSH and the serial console.

Any idea what might be behind that and how to regain access?

Thanks,

Stephan

[Richard Burts]

Stephan

It is interesting that the configured user name and password no longer work. Is that the only thing that is not working?  Or are there other aspects of the configuration that also do not work?

Are you authenticating the user locally on the SG550XG or is it using an authentication server?

Are you sending log messages from SG550XG to a logging server? If so are there any log messages indicating any issue on the SG550XG?

[streaming]

Hi Rick!

I am authenticating locally and unfortunately no external logging is configured.

Everything else is working fine, which is mainly switching. We are not using routing.

ATM I have two possible causes in mind: password aging or a broken flash filesystem on the master.

thanks,

Stephan

[Richard Burts]

Stephan

Thanks for the additional information. I do not have experience with this model of switch and can not comment on the possibility of password aging. Perhaps someone with experience with this might join the discussion?

As for how to recover I have a couple of thoughts:

- if you connect to the serial console and then attempt access using SSH or browser, are there any messages sent to the console? This might shed some light on the issue and suggest a recovery.

- do you have snmp access to the switch? If so perhaps it might shed some light on what is going on. And if the snmp supported on line write access you might be able to configure a new password.

- you could power cycle the switch. It is possible that this might clear whatever is the issue. But it is also possible that it might break some things that are now working. Do you have a copy of the config that could be used if you need to recover/rebuild the switch?

[Jens Albrecht]

First of all, you should check whether one of the steps suggested by @Richard Burts allow you to regain access.

If not, then the last resort is to do a password recovery on the master switch according to this reference:

https://www.cisco.com/c/en/us/support/docs/smb/switches/cisco-small-business-300-series-managed-switches/smb4985-administrator-password-recovery-for-300-and-500-series-manag.html

Since you have a stack and the config is stored on the master the steps are:

• power down the entire stack, i.e. both switches in your case
• restart the master switch only and carefully follow the step-by-step reference
  in step 5 the "copy start run" command allows you to retain the existing configuration
• after reloading the master switch in step 11, you can power on the second switch

So this procedure allows you to configure a new username and password while retaining the existing configuraton of your stack.

HTH!

[streaming]

Hi Richard!

Unfortunately the stack is not configured for SNMP, so no luck with that.

I checked your other recommendation to watch the serial console while logging in on web or SSH but also no luck, no additional log messages came up on the serial console.

To do the power cycle I will have to find a suitable maintenance window.

ATM switch1 is master and switch2 is a member.

My plan is to move all traffic away from switch1 to switch2 using host-side features. We have ESX hosts and NetApp storage - all of them are capable to relocate the traffic through specific interfaces non-disruptively. I will also uncable everything.

Then I will power down switch1 which should result in switch2 becoming the new master and hopefully no traffic interruption will occur. Ideally my login problem will already be solved then ...

If the login problem persists I will have to use the password recovery procedure in a stack - which means that I will have to schedule another maintenance window because then I will have to shut down all hosts and VMs.

But before doing that I will power up switch1 (which is disconnected from switch2) to see if any hardware and esp flash issues come up during boot. If necessary I will replace it by a spare switch from stock.

If everything fails I will have to wipe both switches and build them up from scratch which will result in quite a long downtime as I have no backup from the config.

Will let you know about the progress.

Thanks,

Stephan

[Richard Burts]

Stephan

Thanks for the update. Your plan sounds good. I hope it goes well.

[streaming · Accepted solution]

Hi Richard!

Had the maintenance window yesterday and here are my findings:

- switching all the traffic from master to slave worked flawlessly including the uplink (LACP)

- unplugged everything and the took the power away from the master

- the slave took over and became the new master however access was not restored

- rebooted the old master and I was able to login with my old username and password on the serial console!

- after some minutes I received a message on the serial console that my password has expired and if I want to change it. I did so.

- had to power down the slave and reconnect it. came up after reboot without any problems.

So it was password aging which prevented access.

The option is only available in the web interface after switching to advanced:

Not used to that feature on Cisco switches, I was working with 65xx in the past.

Anyway thanks for all the help,

Stephan

[Richard Burts]

Stephan

Thanks for the update. You are welcome. It is interesting that the issue was indeed password aging. Glad to know that you have resolved your issue.