CSPC upgrade from 2.8.1.4 to 2.8.1.7 hung at apply-in-progress

[Benjamin Bartsch]

Hi - I was recently tasked with upgrading the CSPC server from 2.8.1.4 to 2.8.1.7.  I believe I clicked the right buttons in the CSPC GUI to kickoff the installation.  However, it has been hung at 'apply-in-progress' for days now.  I can't get a TAC engineer to assist - SR 688388242 due to some entitlement issues.

Anybody run into this or have any ideas on what I can do to fix it?  I'm thinking I may need my server guy to just power cycle the VM but I don't know if that's the correct course of action.  I can't seem to find any buttons to click to cancel the install and try again.

Any help would be appreciated.

[adias]

In deploying the  image  you would have created the cli admin password?  (java based shell authentication is not the same as the GUI)

If so have you also created/activated  the users collectorlogin and root ? (real linux shell)

If not do the following for the cli admin  shell

admin# pwdreset root 90

And

admin# pwdreset collectorlogin 90

it will activate those accounts and generate their passwords  ( keep them where you can refer to them in the future)

after that

CONNECT TO admin user via SSH and send me the following commands

admin#  show version -d

admin#  show apply

admin# show download

then

Via SSH Login to collector as user collectorlogin and then use command below to connect as root (cannot connect directly)

# su root

and then the following commands and send me the outputs of all of the commands requested in private message :

# cat /opt/LCM/logs/install

# cat /opt/LCM/logs/apply

# find / -name sp-30*

[Benjamin Bartsch]

Adias - I don't know the admin credentials (the guy who set up this server no longer works for my customer).  I can get to the SSH login page but it says something about 'account locked out for too many wrong passwords'

[adias]

Are you using the same password as the admin GUI account, it may not be the same they are different authentications. If you confident you have the CLI admin or collectorlogin and root passwords.. you can try again in about one hour.  If not likely you will need to redeploy the server.

[Benjamin Bartsch]

We are trying several different username and password combinations as we do not know the exact Linux admin user or password.

It does not appear the Lixux CLI is allowing anything for user admin.  Even after waiting over an hour the CLI is returning 'account locked due to X failed logins' where X is still incrementing.

Is there a password reset procedure we can try?

[adias]

It only could be user admin and the password that was created on installation of the ova or for it modified afterwards on CLI. There is not exactly a way to reset the password since you cannot get to anywhere to reset, the closest to a reset is to do it via the VMware single user mode which would require the grub password which you do not have.. It would be more expedient to redeploy

[Benjamin Bartsch]

Understood.  We are going to try a reboot first to see if that doesn't clear the admin credential lock so we can try a few new passwords we found.

I still have access to the CSPC collector GUI and it seems to be working despite the hung upgrade.  Is there a method to export the settings that might make re-deployment easier (stuff like managed node list, discovery / collection / upload settings)

[chris.cornwell]

Will the backup and restore backup under the administration tab in the GUI allow us to redeploy all of the devices that are configured?

[adias]

Unfortunately those backup features are version specific and will not work for higher versions.. but since the 2.8.1.4 ova is at the same version it may work.  However what you can also do is on credentials list - export credentials  and on managed device list export manage devices list.