ISE Restore fails after 80% (Segmentation fault)

[Thomas Buergi]

Hi All

After a system crash I'm trying to restore. I first did a application reset-config ise and then a restore over an ftp repository.

After 80% it fails with at DB Syncup with a segmentation fault. Tried this with different Backup versions all having the latest patch level.

Any Ideas ?

Thanks

Initiating restore.  Please wait...
% restore in progress: Starting Restore...10% completed
% restore in progress: Retrieving backup file from Repository...20% completed
% restore in progress: Decrypting backup data...25% completed
% restore in progress: Extracting backup data...30% completed
 Leaving the currently connected AD domain
 Please rejoin the AD domain from the administrative GUI
% restore in progress: Stopping ISE processes required for restore...35% completed
% restore in progress: Restoring ISE configuration database...40% completed
% restore in progress: Updating Database metadata...70% completed
% restore in progress: Restoring logs...75% completed
% restore in progress: Performing ISE Database synchup...80% completed
/opt/CSCOcpm/bin/isecfgrestore.sh: line 1248: 11162 Segmentation fault      (core dumped) curl -q -k --tlsv1 --connect-timeout 10 https://$MYIP/admin/index.jsp > /dev/null 2>&1
/opt/CSCOcpm/bin/isecfgrestore.sh: line 1248: 11177 Segmentation fault      (core dumped) curl -q -k --tlsv1 --connect-timeout 10 https://$MYIP/admin/index.jsp > /dev/null 2>&1
/opt/CSCOcpm/bin/isecfgrestore.sh: line 1248: 11187 Segmentation fault      (core dumped) curl -q -k --tlsv1 --connect-timeout 10 https://$MYIP/admin/index.jsp > /dev/null 2>&1

......and so on 

**************************************************************************************************************

Cisco Application Deployment Engine OS Release: 2.2
ADE-OS Build Version: 2.2.0.162
ADE-OS System Architecture: x86_64

Copyright (c) 2005-2014 by Cisco Systems, Inc.
All rights reserved.
 

Version information of installed applications
---------------------------------------------

Cisco Identity Services Engine
---------------------------------------------
Version      : 1.3.0.876
Build Date   : Tue Oct 28 04:02:29 2014
Install Date : Fri Dec  5 08:31:15 2014     

Cisco Identity Services Engine Patch 
---------------------------------------------
Version      : 1
Install Date : Mon Jan 26 10:21:50 2015

[mverbon]

Hi All,

I see there is no follow up on this issue.

I have got exactelly the same issue, ISE version 1.3 with Patch2.

Are there any knows issues with Restore and ISE version 1.3?

Thanks.

Martin

[Thomas Buergi]

I had to reimage ISE completly, after that the restore worked without issues.

An application reset-config before the restore was not enough.

Cheers Thomas

[rhub]

Hi all,

i faced exactly the same problem yesterday. The restore-process stuck at 80%, so it is completely useless to do backups, if we are not able to restore them in a timely manner

I really would appreciate if someone could follow on this

Cheers

Roman

[Charlie Moreton]

I believe this needs to be raised to TAC for documentation and a fix to be created and published.

Please Rate Helpful posts and mark this question as answered if, in fact, this does answer your question.  Otherwise, feel free to post follow-up questions.

Charles Moreton

[mcdermott_greg]

I just submitted this issue to TAC.  Thankfully we haven't put ISE into production yet.  So we'll see what happens.  I'll keep you posted.

[mcdermott_greg · Accepted solution]

After sending the appropriate logs to the TAC I learned that even though the restore looked to have failed, at 80% the actual restore has been completed and what was failing was the node synchronization.   With this knowledge I was able to access the admin GUI and manually sync up the other nodes and have been running without issue.

As it turns out this is  documented issue, CSCur36983, and will be fixed in release 1.4.

Here is the reply I received from the TAC:

"Based on the logs sent, we're definitely hitting CSCur36983 which will be
fixed in ISE 1.4:
https://tools.cisco.com/bugsearch/bug/CSCur36983/

According to the bug details, when we reach 80% the restoration part itself
has already completed. What is failing is a sync with the rest of the nodes
in the deployment. Therefore, you can reload the ISE at this point and you
should have access to the admin GUI again. From there, you can force a sync up  with the other nodes."

[rhub]

Hi Greg,

many thanks for your reply, which I appreciate very much

Roman Huber