Upgrade from CSPC 2.7.4 stuck on "Apply-in-progress"

[danielbarri]

A CSPC run 2.7.4 with scheduled upgrade. Now it stuck on "Apply-in-progress" for more than a month(see below outputs). A reboot did not make any difference. Also tried unsuccessfully the steps in:

https://community.cisco.com/t5/smart-net-total-care-portal-and/upgrade-to-cspc-2-8-stuck-on-quot-apply-in-progress-quot/td-p/3736801

admin# sh version -d
        Build-name   : Collection Platform Software 2.7.4
        Package-type : ServicePack
        Version      : sp-30.0.4-0-0-lnx64
             Component : CSPC Base
             Version   : 2.7.4
        Package-type : JeOS
        Version      : jeos-30.0.4-0-lnx64
             Component : AdminShell
             Version   : 1.7
             Component : LCM
             Version   : 1.7
             Component : CASP
             Version   : 2.7.4
             Component : ConcsoTgw
             Version   : 1.7.2
             Component : Hardened CentOS
             Version   : 6.8 patch#0

admin# show apply
........
        Version Number     : sp-30.1.0-0-0-lnx64
        Status             : Apply-in-progress
        Start Time         : Oct/22/2018 14:55:39
        End Time           : -

===========================
# pwd
/opt/LCM/tmp
# ls -alrth
total 6.0G
-rw-r--r--. 1 casuser casusers 877M Oct 24  2017 sp-30.0.2-0-0-lnx64-package.zip
-rw-r--r--. 1 casuser casusers 1.1G Apr 11  2018 sp-30.0.4-0-0-lnx64-package.zip
drwxr-xr-x. 9 casuser casusers 4.0K Apr 11  2018 ..
-rw-r--r--. 1 root    root     340M Apr 12  2018 sp-30.0.4-0-0-lnx64.zip
-rw-r--r--. 1 root    root     720M Apr 12  2018 jeos-30.0.4-0-lnx64.zip
-rw-r--r--. 1 root    root     767M Oct 20 12:18 jeos-30.1.0-0-lnx64.zip
-rw-r--r--. 1 root    root     787M Oct 20 12:20 sp-30.1.0-0-0-lnx64.zip
-rw-r--r--. 1 root    root     1.6G Oct 20 12:20 sp-30.1.0-0-0-lnx64-package.zip
drwxr-xr-x. 2 root    root     4.0K Oct 21 11:00 downloads
drwxr-xr-x. 3 casuser casusers 4.0K Dec  3 13:28 .

================
# tail -f /opt/LCM/logs/apply
[Thu Apr 12 07:47:06 CEST 2018] Values from new apply wrapper
[Thu Apr 12 07:47:06 CEST 2018] /opt/LCM/tmp/sp-30.0.4-0-0-lnx64
[Thu Apr 12 07:47:06 CEST 2018] ---------
EST 2018] Completed update of Apply Count DB
Inside Apply script
Package Name : /opt/LCM/tmp/sp-30.0.4-0-0-lnx64
Timestamp :
Count : 0
[Mon Oct 22 14:55:39 CEST 2018] Wrapper TCL Called with arguments "SP", "/opt/LCM/tmp/sp-30.1.0-0-0-lnx64-package.zip" and "false"
[Mon Dec  3 13:32:10 CET 2018] Apply TCL Called with arguments "SP" and "/opt/LCM/tmp/sp-30.1.0-0-0-lnx64.zip"

[danielbarri]

Attached is the GUI Software Update status

[Supriya Patel]

Hi User,

Please try below and let me know:

Please login to root and use below single line command, it will place apply-in-progress status to failed one. Once you use below command, please check 'show apply' and make sure that it is failed status.

/bin/sh /opt/LCM/bin/sqliteToDerbyScripts_shell.sh "connect 'jdbc:derby://localhost:1527/opt/LCM/info/updates; create=true;user=app;password=password';" "update updates set PACKAGE_STATUS = 'Apply-failed' where PACKAGE_STATUS = 'Apply-in-progress';"

Once done, please follow below guide for manual upgrade process (page 20 - Apply JeOS package 2.8 and then page 21 for SP):

https://www.cisco.com/c/dam/en/us/support/docs/cloud-systems-management/common-services-platform-collector-cspc/CSPC-Upgrade-Guide.pdf

Please do let me know if you have any question.

Thanks,

Supriya

[danielbarri]

Hi,

I performed the suggested steps and CSPC is now running 2.8. But the apply status stayed at "Apply-failed" :(

admin# show version -d
        Build-name   : Collection Platform Software 2.8
        Package-type : ServicePack
        Version      : sp-30.1.0-0-0-lnx64
             Component : CSPC Base
             Version   : 2.8
        Package-type : JeOS
        Version      : jeos-30.1.0-0-lnx64
             Component : AdminShell
             Version   : 1.8
             Component : LCM
             Version   : 1.8
             Component : CASP
             Version   : 2.8
             Component : ConcsoTgw
             Version   : 1.7.3
             Component : Hardened CentOS
             Version   : 6.10 patch#0
--------------------
admin# show apply
.................
        Version Number     : sp-30.1.0-0-0-lnx64
        Status             : Apply-failed
        Start Time         : Oct/22/2018 14:55:39
        End Time           : -
        Reason             : null

[Bharath S]

Hi Daniel, 

Database which stores the download and apply status will not be modified when upgrades are performed manually. It has to be changed manually again to "Applied" in this case.

Please login to CSPC UI verify Software Updates page is displaying proper data and u have option to upgrade to any new version available.

Thanks

[danielbarri · Accepted solution]

Hi,

On the GUI there was 2.8.0.1 available and I did the upgrade which worked fine. Now the 'sh apply' status seems correct.

admin# sh apply

......................
        Version Number     : sp-30.1.0-0-0-lnx64
        Status             : Apply-failed
        Start Time         : Oct/22/2018 14:55:39
        End Time           : -
        Reason             : null

        Version Number     : sp-30.1.0-1-0-lnx64
        Status             : Applied
        Start Time         : Dec/06/2018 11:11:27
        End Time           : Dec/06/2018 12:06:18

Thanks for your assistance!