Cisco ISE can not restore backup configuration it show "Error Backup of existing DB failed Hence Restore failed"

[alexander kobayashi]

Dear All 

I did restore backupconfiguration on cisco ise verion 2.7 p2 but not work it show message "Cisco ISE can not restore backup configuration it show "Error Backup of existing DB failed Hence Restore failed"

anyone have experience about this 

thank you

[poongarg · Accepted solution]

This error is generally seen when wrong encryption key is used while restoring the backup or corrupted download from repository.
Check the details in below logs
show logging system restore.log
show logging system ade/ADE.log

[Arne Bier]

Hello @alexander kobayashi and @poongarg 

any updates on this issue?

I ran into the same issue today. I built a fresh ISE 2.7 from Cisco OVA (600GB) and then applied patch 2. No other config was done to this node. It was in Standalone.  On the CLI I attempted the restore. Twice. Each time the same error. I checked the gpg encryption password and it's correct. I also checked the MD5 checksum of the backup file and it's correct.

ise-01/admin# restore CFG10-201002-1355.tar.gpg repository arne encryption-key plain SomeSecretKey
% Warning: Do not use Ctrl-C or close this terminal window until the restore completes.
Initiating restore.  Please wait...
% restore in progress: Starting Restore...10% completed
% restore in progress: Retrieving backup file from Repository...20% completed
% restore in progress: Decrypting backup data...25% completed
% restore in progress: Extracting backup data...30% completed
 Leaving the currently connected AD domain
 Please rejoin the AD domain from the administrative GUI
% restore in progress: Stopping ISE processes required for restore...35% completed
Cleaning up TC-NAC docker configuration...
% restore in progress: Restoring ISE configuration database...40% completed
% Application restore failed

ise-01/admin#

ise-01/admin# show restore status
%% Configuration restore status
%% ----------------------------
%      backup name: CFG10-201002-1355.tar.gpg
%       repository: arne
%       start date: Sat Oct 17 06:50:18 AEST 2020
%        scheduled: no
%   triggered from: CLI
%             host:
%           status: Error: Backup of existing DB failed. Hence Restore is aborted

%% Operation restore status
%% ------------------------
%  No data found. Try 'show restore history' or ISE operation audit report


ise-01/admin# show restore history
Sat Oct 17 06:58:42 AEST 2020: restore CFG10-201002-1355.tar.gpg from repository arne: failed - Error: Backup of existing DB failed. Hence Restore is aborted

The ADE Log shows the following

2020-10-17T07:34:02.180456+10:00 ise-01 admin: Connected to:
2020-10-17T07:34:02.180465+10:00 ise-01 admin: Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
2020-10-17T07:34:02.180474+10:00 ise-01 admin: With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
2020-10-17T07:34:02.180480+10:00 ise-01 admin:
2020-10-17T07:34:02.432907+10:00 ise-01 admin:
2020-10-17T07:34:02.432928+10:00 ise-01 admin: #011#011      0
2020-10-17T07:34:02.434538+10:00 ise-01 admin: Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
2020-10-17T07:34:02.434556+10:00 ise-01 admin: With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
2020-10-17T07:34:02.445587+10:00 ise-01 admin: notice:[backup-restore:restore:isecfgrestore.sh] Flash recovery area is % 1 full
2020-10-17T07:34:07.038241+10:00 ise-01 admin: info:[backup-restore:restore:isecfgrestore.sh] Starting DB export backup for SCN 5074636 and host ise-01
2020-10-17T07:34:27.282994+10:00 ise-01 admin: error:[backup-restore:restore:isecfgrestore.sh] DB BACKUP FAILED :
2020-10-17T07:34:27.316040+10:00 ise-01 ADE-SERVICE[6608]: [30506]:[info] locks:file: lock.c[356] [admin]: deleted progress file /tmp/adeos-backuprestore-inprogress
2020-10-17T07:34:27.316524+10:00 ise-01 ADE-SERVICE[6608]: [30506]:[info] locks:file: lock.c[456] [admin]: deleted backup-restore progress file: /tmp/adeos-backuprestore-inprogress
2020-10-17T07:34:27.317839+10:00 ise-01 ADE-SERVICE[6608]: [30506]:[info] locks:file: lock.c[356] [admin]: deleted progress file /tmp/adeos-restore-inprogress
2020-10-17T07:34:27.318264+10:00 ise-01 ADE-SERVICE[6608]: [30506]:[info] locks:file: lock.c[466] [admin]: deleted restore progress file: /tmp/adeos-restore-inprogress
2020-10-17T07:34:27.319063+10:00 ise-01 ADE-SERVICE[6608]: [30506]:[error] backup-restore:backup: br_restore.c[1308] [admin]: Error: Backup of existing DB failed. Hence Restore is aborted
2020-10-17T07:34:27.325476+10:00 ise-01 ADEOSShell[30506]: ADEAUDIT 2018, type=RESTORE, name=RESTORE FAILED, username=admin, cause=A restore has failed, adminipaddress=127.0.0.1, interface=CLI, detail=Error: Backup of existing DB failed. Hence Restore is aborted
2020-10-17T07:34:27.335413+10:00 ise-01 ADEOSShell[30506]: ADEAUDIT 3014, type=RESTORE, name=RESTORE FAILED, username=admin, cause=Error during restore, adminipaddress=127.0.0.1, interface=CLI, detail=Error: Backup of existing DB failed. Hence Restore is aborted
2020-10-17T07:34:27.336208+10:00 ise-01 ADE-SERVICE[6608]: [30506]:[info] backup-restore:backup: br_history.c[487] [admin]: updating /tmp/ise-cfg-br-flags with status: complete and message: (null)
2020-10-17T07:34:27.343669+10:00 ise-01 ADE-SERVICE[6608]: [30506]:[info] backup-restore:restore: br_cli.c[209] [admin]: error message: Application restore failed

The other log file was not informative

ise-01/admin# show logging system restore.log
Sat Oct 17 07:34:27 AEST 2020: restore CFG10-201002-1355.tar.gpg from repository arne: failed - Error: Backup of existing DB failed. Hence Restore is aborted

Sat Oct 17 06:58:42 AEST 2020: restore CFG10-201002-1355.tar.gpg from repository arne: failed - Error: Backup of existing DB failed. Hence Restore is aborted

I had to reboot the node after the first failed restore.

I don't understand why ISE is trying to backup the existing database.

[Mike.Cifelli]

@Arne Bier I had the same issue last Friday when migrating a 4 node 2.4p9 cluster to a brand new 2.7p2 cluster (all virtual).  I ended up having to issue an application reset-config ise, and then on the first attempt to restore config it worked like a charm.  When discussing with my Cisco reps the theory was since I had left the VMs running for a while the issue was self-inflicted.  Not 100% sure that is accurate, but I spent roughly 2 hours troubleshooting prior to the reset.

[Arne Bier]

Hey @Mike.Cifelli - thanks for sharing that - I had another ISE node that was also 2.7p2 brand new fresh install and I was able to restore the config to that! Can you believe it! The only difference between ISE01 and ISE02 is that ISE01 was possibly not shut down well and perhaps the Oracle got some "issues". I will config reset ISE01 and then register it to ISE02 - I am too far down the trail now to try and restore back on ISE01.

Again, I appreciate the feedback - good to know there is a common pattern here

[Damien Miller]

Inline upgrade > backup/restore. I just inline upgraded a 24 node deployment from 2.4 to 2.7 without issue on Wednesday