Buy or Renew
Buy or Renew
Cisco + Splunk: It’s a new day for your data. Learn more.
 
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
cancel
Start a conversation
1486
Views
0
Helpful
9
Replies

CSPC Stuck in Apply-in-progress

RicTodd89
Level 1
Level 1

‎03-20-2019 09:27 AM

Hello,

I am trying to patch our CSPC to version 8.1.1.2.  It downloaded successfully, but it is still currently stuck in apply-in-progress.

Here is the output after running #show apply:

admin# show apply
Version Number : sp-30.0.4-0-0-lnx64
Status : Applied
Start Time : May/01/2018 18:02:19
End Time : -

Version Number : sp-30.1.0-1-0-lnx64
Status : Apply-in-progress
Start Time : Dec/13/2018 19:33:56
End Time : -

Version Number : sp-30.1.1-2-0-lnx64
Status : Apply-in-progress
Start Time : Mar/19/2019 22:49:23
End Time : -

 

I see that the last update I attempted was also stuck in apply-in-progress.  I have root access to the device.

 

Can someone please assist?

 

0 Helpful
9 Replies 9

Ye Liang
Cisco Employee
Cisco Employee

‎03-20-2019 10:03 AM

Hello,

 

Under "admin" user, please provide output of show version -d.

 

Under "root" user, please go to /opt/LCM/logs and show output of "tail -50 install" and "tail -50 apply".

 

Thanks,

Ye

0 Helpful

RicTodd89
Level 1
Level 1
In response to Ye Liang

‎03-22-2019 12:24 PM

admin# show version -d
Build-name : Collection Platform Software 2.8

Package-type : ServicePack
Version : sp-30.1.0-0-0-lnx64
Component : CSPC Base
Version : 2.8
Package-type : JeOS
Version : jeos-30.1.0-1-lnx64
Component : AdminShell
Version : 1.8.1
Component : LCM
Version : 1.8.1
Component : CASP
Version : 2.8.0.1
Component : ConcsoTgw
Version : 1.7.3
Component : Hardened CentOS
Version : 6.10 patch#0

 

keytool error: java.lang.Exception: Certificate not imported, alias <concsoweb3-prd> already exists
keytool error: java.lang.Exception: Certificate not imported, alias <concsoweb3-prd> already exists

 

Service Name : COLLECTOR
empty
Local DB and SchedulerDB are not in sycn so jobs scheduled will be lost

 

 


Changing log_group value in auditd
Stopping auditd: [ OK ]
Starting auditd: [ OK ]
Archive: LCM_DERBY_Scripts.zip
inflating: /opt/LCM/bin/apply.sh
inflating: /opt/LCM/bin/asd-download.sh
inflating: /opt/LCM/bin/autoupdate-init.exp
inflating: /opt/LCM/bin/cmd-exec.tcl
inflating: /opt/LCM/bin/decryptor.sh
inflating: /opt/LCM/bin/downloadResultUpdate.sh
inflating: /opt/LCM/bin/getDependencyList.sh
inflating: /opt/LCM/bin/ida-download.sh
inflating: /opt/LCM/bin/lcmagent-apply-check.tcl
inflating: /opt/LCM/bin/lcmagent-apply-count.tcl
inflating: /opt/LCM/bin/lcmagent-apply.tcl
inflating: /opt/LCM/bin/lcmagent-apply-wrapper.tcl
inflating: /opt/LCM/bin/lcmagent-download.tcl
inflating: /opt/LCM/bin/lcmagent-system.tcl
inflating: /opt/LCM/bin/LCMDecryptor.jar
inflating: /opt/LCM/bin/packaging.sh
inflating: /opt/LCM/bin/restart.sh
inflating: /opt/LCM/bin/Resume.sh
inflating: /opt/LCM/bin/status_update.sh
inflating: /opt/LCM/bin/updateApplyCheckData.sh
inflating: /opt/LCM/bin/updator.sh
lcmuptime.sh cronjob doesnot exit , so creating lcmuptime.sh cronjob
remotelogin exists!
removing remotelogin from ssh config
============================
Rebooting the appliance now
============================

============================================================================
Finished execution of post-install scripts in package jeos-30.1.1-2-lnx64 ...
============================================================================

 

Upgrading Derby DB
Installing Derby DB 10.14.2.0
Derby DB Upgraded
Inside Preinstall
[Tue Mar 19 18:53:13 EDT 2019] Executed pre-install scripts
[Tue Mar 19 18:53:13 EDT 2019] Executing install script for "Hardened CentOS"
[Tue Mar 19 23:11:52 UTC 2019] Executed install script for "Hardened CentOS"
[Tue Mar 19 23:11:52 UTC 2019] Install script for "Hardened CentOS" returned success
[Tue Mar 19 23:11:52 UTC 2019] Executing install script for "LCM"
[Tue Mar 19 23:12:08 UTC 2019] Executed install script for "LCM"
[Tue Mar 19 23:12:08 UTC 2019] Install script for "LCM" returned success
[Tue Mar 19 23:12:09 UTC 2019] Executing install script for "AdminShell"
[Tue Mar 19 23:18:33 UTC 2019] Executed install script for "AdminShell"
[Tue Mar 19 23:18:33 UTC 2019] Install script for "AdminShell" returned success
[Tue Mar 19 23:18:34 UTC 2019] Executing install script for "ConcsoTgw"
[Tue Mar 19 23:19:20 UTC 2019] Executed install script for "ConcsoTgw"
[Tue Mar 19 23:19:21 UTC 2019] Install script for "ConcsoTgw" returned success
[Tue Mar 19 23:19:21 UTC 2019] Executing install script for "CASP"
[Tue Mar 19 23:19:22 UTC 2019] Executed install script for "CASP"
[Tue Mar 19 23:19:22 UTC 2019] Install script for "CASP" returned success
[Tue Mar 19 23:19:22 UTC 2019] Calling Procedure
[Tue Mar 19 23:19:23 UTC 2019] Calling updator.sh
Inside update script
Order Id :
Package Name :
Count :
update next package to install
[Tue Mar 19 23:19:24 UTC 2019] End of Script Execution
[Tue Mar 19 23:19:24 UTC 2019] End of Procedure
[Tue Mar 19 23:19:24 UTC 2019] Executing post-install scripts
Current SP :
Current JeOS :
Service Name : COLLECTOR
Hostname : CSPC01.corp.local
Timezone : UTC
Replaced scripts with Derby statements
[Tue Mar 19 23:20:07 UTC 2019] Executed post-install scripts
[Tue Mar 19 23:20:07 UTC 2019] Installation of software package of type "JeOS" completed successfully
[Tue Mar 19 23:20:08 UTC 2019] --Component--CASP----jeos-30.1.0-1-lnx64--
[Tue Mar 19 23:20:08 UTC 2019] --Component--ConcsoTgw----jeos-30.1.0-1-lnx64--
[Tue Mar 19 23:20:09 UTC 2019] --Component--AdminShell----jeos-30.1.0-1-lnx64--
[Tue Mar 19 23:20:10 UTC 2019] --Component--LCM----jeos-30.1.0-1-lnx64--
[Tue Mar 19 23:20:10 UTC 2019] --Component--Hardened CentOS----jeos-30.1.0-1-lnx64--
[Tue Mar 19 23:20:11 UTC 2019] --JeOS--JeOS----jeos-30.1.0-1-lnx64--
[Tue Mar 19 23:20:11 UTC 2019] status db updation completed
apply.sh - 0------exit status
[Wed Mar 20 16:20:38 UTC 2019] Apply TCL Called with arguments "SP" and "/opt/LCM/tmp/sp-30.1.1-2-0-lnx64"
[Wed Mar 20 16:20:38 UTC 2019] Inside existence of status db
[Wed Mar 20 16:20:40 UTC 2019] Package file "/opt/LCM/tmp/sp-30.1.1-2-0-lnx64" not found
You have new mail in /var/spool/mail/root

0 Helpful

jofrumki
Cisco Employee
Cisco Employee
In response to RicTodd89

‎03-25-2019 05:11 PM

From the CLI please run the command and make note of the file location to use in the next command;
find / -name sp-30.1.1-2-0-lnx64.zip
Then run the command;
screen /opt/LCM/bin/lcmagent-apply.tcl SP /path/to/file/sp-30.1.1-2-0-lnx64.zip
You can then press CRTL+A+D and monitor the installation using the command;
tailf /opt/LCM/logs/install
This may take some time to start initially so please give it plenty of room to run.
0 Helpful

RicTodd89
Level 1
Level 1
In response to jofrumki

‎03-27-2019 08:31 AM

I ran the command you mentioned and waited about a day.

Here are the results when going back to the collectorlogin:

[collectorlogin@CSPC01 ~]$ tailf /opt/LCM/logs/install
remotelogin exists!
removing remotelogin from ssh config
============================
Rebooting the appliance now
============================

============================================================================
Finished execution of post-install scripts in package jeos-30.1.1-2-lnx64 ...
============================================================================

 

Then from the regular admin account login:

 

admin# show version
Build-name : Collection Platform Software 2.8

Version : sp-30.1.0-0-0-lnx64
admin# show apply
Version Number : sp-30.0.4-0-0-lnx64
Status : Applied
Start Time : May/01/2018 18:02:19
End Time : -

Version Number : sp-30.1.0-1-0-lnx64
Status : Apply-in-progress
Start Time : Dec/13/2018 19:33:56
End Time : -

Version Number : sp-30.1.1-2-0-lnx64
Status : Apply-in-progress
Start Time : Mar/19/2019 22:49:23
End Time : -

0 Helpful

jofrumki
Cisco Employee
Cisco Employee
In response to RicTodd89

‎03-27-2019 09:45 AM

Please find the Troubleshooting guide here, https://www.cisco.com/c/dam/en/us/support/docs/cloud-systems-management/common-services-platform-collector-cspc/CSPC_Troubleshooting_Guide.pdf
and reference the section on Apply-in-progress Hunged. After following these steps you should be capable of attempting the upgrade again.
0 Helpful

RicTodd89
Level 1
Level 1
In response to jofrumki

‎03-27-2019 11:50 AM

Okay, I'm following the guide.  

Not sure which PID needs to be entered.  Also the update script listed is unavailable to me.

Here is the output after running the command listed in the troubling shooting guide:

[root@CSPC01 ~]# ps -aef | grep apply
root 19842 19825 0 18:44 pts/1 00:00:00 grep apply

0 Helpful

jofrumki
Cisco Employee
Cisco Employee
In response to RicTodd89

‎03-27-2019 01:41 PM

Based on the output there is no installation taking place.

I will send the script in a private message and let our SMEs know the link is not accessible from the guide. 

0 Helpful

RicTodd89
Level 1
Level 1
In response to jofrumki

‎04-03-2019 09:24 AM

Hi,

Just following up that I have not received the private message.

0 Helpful

jofrumki
Cisco Employee
Cisco Employee
In response to RicTodd89

‎04-03-2019 10:13 AM

The community and private message option will not allow me to send files. Please private message me your email so that I can send the script.

0 Helpful
Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Customers Also Viewed These Support Documents