ISE backup hung

[muradhattar]

It appears like the backup is hung on the primary ISE 1.3 node. 

- I created a repository in the GUI and started a backup to it. The repository was not accessible so the backup got stuck at 75%, it wasn't actually doing anything for the last week. 

ISE01# show backup status
%% Configuration backup status
%% ----------------------------
% backup name: pre_2.0_upgrade
% repository: Shared_Network_Drive
% start date: Mon Nov 30 11:35:59 PST 2015
% scheduled: no
% triggered from: Admin web UI
% host: pwise01.dpw.co.la.ca.us
% status: Backup is in progress...
% progress %: 75
% progress message: Moving Backup file to the repository

%% Operation backup status
%% ------------------------
% No data found. Try 'show backup history' or ISE operation audit report

- created an FTP repository in the command line , verified that the ISE server can access it, then started a backup to it. 

ISE01#backup configuration_1 repository upgrade ise-config encryption-key plain SOAndSO
% Warning: Waiting up to 1200 seconds for APP_BACKUP to finish...

- if I try stopping the ise application I get database locked 

application stop ise

Waiting up to 20 seconds for lock: APP_BACKUP APP_BACKUP to complete
Database is still locked by lock: APP_BACKUP APP_BACKUP. Aborting. Please try it later
% Error: Another ISE DB process (APP_BACKUP APP_BACKUP) is in progress, cannot perform Application Stop at this time

- tried cancelling the job from the GUI I get this message 

[nspasov · Accepted solution]

Have you given a reload via the CLI?

[andres.picos]

Reload is a solution? Really?

[Damien Miller]

If you have a back up issue of your own it would be better to start a discussion. Reloads are not usually required to clear a stuck backup anymore, but in 1.3 that or TAC were the only options.

[andres.picos]

I've arrived here searching for a solution in a similar issue in ISE 2.4 p6. and coincidentally I found this post.

I found a bug for ISE 2.4 p6 (CSCvq13817) where a workaround is "Reload the ISE PAN server". 

I think it is absurd this kind of solutions that Cisco provides and the large number of Bugs that Cisco platforms have.

[Ruud.Hoba]

[smirk]

Have you tried turning it off and on again?!?!

[/smirk]

Command:

backup backupname repository repository-name ise-config encryption-key plain hashkey

solves as well.. even in the GUI the progress bar starts showing the progress :)

[Ruud.Hoba · Accepted solution]

Oh. By the way. The bug remains, so its very likely the next time your scheduled backup runs, it gets stuck again. And can be solved by the CLI command.
Upgrading is recommended, more recent versions are available...
Note as well that after a reload the backup schedule needs to be reconfigured. The old one is bad and won't work. Haven't tried it but I guess a cron job or similar is also plausible. And when using HPNA for example, a command line script can be configured and scheduled to be ran from the HPNA server to the ISE.

[Mat Barry]

This worked for me.   Bravo, Sir.

[rene_braun]

Bug is still there on ISE 2.6 P3, but the workaround is working fine. 

[Ruud.Hoba]

To bad the bug is still there.. But hey, they can't fix all at once..
I haven't been upgrading recently to test other versions/patches.

Good to hear the workaround is still valid.. 🙂

[Darkmatter]

This bug was still present in ISE 2.7 but it's fixed since the install of Patch 1

I can confirm i experienced the same issue but backups are now properly being done on ISE 2.7 P1

[DMog85286]

This bug just reared its ugly head here on 2.7 Patch 4.

Went to test a sftp backup after creating the repository and the GUI went to displaying Configuration Data Backup 0% .... for over 24 hours with no progress. Went to Stop it and there is a delay** of being able to click Stop; not sure why that is happening...but when the Stop is active, I click Stop and nothing happens. Logged in to CLI and checked backup status; and shows my config "Backup is in progress..." 

I used the 'workaround'; performing a manual backup via CLI and even though it made it to 75% and then failed; it did clear up the hung GUI session.

**Does anyone know what causes this delay?  Whenever I attempt to work at all in the Backup & Restore tab within the GUI it happens. I attempt to modify a scheduled backup or do a manual one, I get the Page Unresponsive | Wait or Exit Page option window. Using Chrome or Edge doesn't matter. 

[Marcelo Morais]

Hi @DMog85286 ,

 please use the following command to check for any new info about the issue:

ise/admin# debug backup-restore all

Hope this helps !!!

[DannyDulin]

This bug appeared for us on 2.7 patch 3.

We experience that delay too. I'm not sure what it is but I wonder if it has anything to do with reaching out to the repositories.

Sure would like to find out this answer.