
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-29-2024 10:38 PM
Hello,
I am wondering, what the root cause could for failing to send generated backup to remote repository.
Running the back on ISE box itself, all phase can be traced.
And it seems that the configuring is completed, but the transfer is failing - here I was using TFTP.
% backup in progress: Moving Backup file to the repository...75% completed
% Transfer timed out.
% File transfer error
% Internal CA Store is not included in this backup. It is recommended to export it using "application configure ise" CLI command
% Creating backup with timestamped filename: ConfigBackup-CLI-CFG10-200326-0705.tar.gpg
% backup in progress: Starting Backup...10% completed
% backup in progress: Validating ISE Node Role...15% completed
% backup in progress: Backing up ISE Configuration Data...20% completed
% backup in progress: Backing up ISE Indexing Engine Data...45% completed
% backup in progress: Backing up ISE Logs...50% completed
% backup in progress: Completing ISE Backup Staging...55% completed
% backup in progress: Backing up ADEOS configuration...55% completed
% backup in progress: Moving Backup file to the repository...75% completed
% Transfer timed out.
% File transfer error
Solved! Go to Solution.
- Labels:
-
Identity Services Engine (ISE)
Accepted Solutions
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-05-2024 03:32 PM
Eventually, I was able to fix the sftp upload issue by removing the configured from GUI at:
Admin > System > Maintenance > Repository
And creating the repository via CLI.
% Internal CA Store is not included in this backup. It is recommended to export it using "application configure ise" CLI command
% Creating backup with timestamped filename: test-sftp-CFG10-240705-1724.tar.gpg
% backup in progress: Starting Backup...10% completed
% backup in progress: Validating ISE Node Role...15% completed
% backup in progress: Backing up ISE Configuration Data...20% completed
% backup in progress: Backing up ISE Indexing Engine Data...45% completed
% backup in progress: Backing up ISE Logs...50% completed
% backup in progress: Completing ISE Backup Staging...55% completed
% backup in progress: Backing up ADEOS configuration...55% completed
% backup in progress: Moving Backup file to the repository...75% completed
% backup in progress: Completing Backup...100% completed
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-29-2024 10:54 PM
what ISE version ?
what Model of Remote backup method ? SCP/FTP/SFTP ?
% backup in progress: Moving Backup file to the repository...75% completed
as per this error looks for me far end folder permission (repository) issue to write the files on the backup destination.
You can also run debug on ISE and check what is the error :
# debug backup-restore backup

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-30-2024 02:35 PM
@Netmart - ISE allows you to configure a repository using various protocols (including tftp) - but what you do with that repository is important - for storing data, you can't use tftp as a protocol - you can use tftp repo for other things that do not involve writing data.
The only supported protocols for an ISE repo that involve storing data are ftp, SFTP, NSF and local disk.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-30-2024 10:16 PM - edited 06-30-2024 10:31 PM
Thank you very much Arne.
I set up sftp, also tested the privilege access to ISE destination folder by running a SFTP session, downloading a test file with the credentials configured in ISE:
Version:3.1.0.518
repository server-sftp
url sftp://server-sftp/data/sftp/ISE
user cisco password hash ******
# backup test-server-sftp repository server-sftp ise-config encryption-key plain *****
% Internal CA Store is not included in this backup. It is recommended to export it using "application configure ise" CLI command
% Creating backup with timestamped filename: test-brutus-sftp-CFG10-240701-0018.tar.gpg
% backup in progress: Starting Backup...10% completed
% backup in progress: Validating ISE Node Role...15% completed
% backup in progress: Backing up ISE Configuration Data...20% completed
% backup in progress: Backing up ISE Indexing Engine Data...45% completed
% backup in progress: Backing up ISE Logs...50% completed
% backup in progress: Completing ISE Backup Staging...55% completed
% backup in progress: Backing up ADEOS configuration...55% completed
% backup in progress: Moving Backup file to the repository...75% completed
% Failure occurred during request
I hope we do not hit: https://bst.cisco.com/bugsearch/bug/CSCwd63717?rfs=qvlogin

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-30-2024 10:28 PM
Are you able to view the directory contents of that SFTP repository, from the vantage of the ISE CLI? Put a simple file in directory /server-sftp/data/sftp/ISE and then check if you can view the file:
show repo server-sftp
if that doesn't work, then I suspect that you haven't created the crypto host key on the CLI - if your repo URL is
sftp://myserver.com/ then your command would be
crypto host_key add host myserver.com
if your repo URL contains an IP address, then use the IP address in the command above.
A useful debugging command for seeing what ISE is doing when you test those show/backup commands:
debug transfer 7

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-30-2024 10:39 PM
Hello Arne, please see output below.
Since this is a production environment, does running "debug transfer 7" has any impact on ISE application services?
sh repository server-sftp
% Error: Repository server-sftp could not be accessed. In case Backup was Restored on different setup, Please reconfigure the repository passwords (expected behaviour).
% Failure occurred during reques
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-30-2024 10:49 PM
It doesn't break to enable the debugs - just disable them once you're done.
Do you have the plain text password of the username "cisco" ? If so, then log into the ISE Admin GUI, and just overwrite the password for that repo config. If you have done a config restore, then ISE will complain and force you to overwrite the password (even if the password hasn't changed)
Can you ping the SFTP server?

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-01-2024 07:08 PM
I am able to ping the sftp server.
I also run from a different linux box sftp commnands to this repository/sftp-server by using same cisco user credentials - no problem.
/admin# show repository server-sftp
6 [520055]:[info] transfer: cars_xfer.c[225] [admin]: sftp dir of repository server-sftp requested
6 [520055]:[info] transfer: cars_xfer_util.c[2297] [admin]: Server validation successful brutus
7 [520055]:[debug] transfer: sftp_handler.c[1095] [admin]: Running sftp command: brutus cisco *** /data/sftp/ISE/ ls -l /data/sftp/ISE/
6 [520055]:[info] transfer: sftp_handler.c[585] [admin]: DEBUG: local user: admin UID: 0 sftp_run_parent FD: 8 remote host: brutus remote user: cisco command: ls -l /data/sftp/ISE/
7 [520055]:[debug] transfer: sftp_handler.c[594] [admin]: fd is:8
7 [520061]:[debug] transfer: sftp_handler.c[292] [admin]: Executing SFTP command: 0 admin /usr/bin/sftp -oIdentityFile=/home/admin/.ssh/id_rsa -oUserKnownHostsFile=/home/admin/.ssh/known_hosts -oPasswordAuthentication=yes cisco@brutus
3 [520055]:[error] transfer: sftp_handler.c[365] [admin]: sftp_select Error: timeout!
7 [520055]:[debug] transfer: sftp_handler.c[964] [admin]: sftp parent status -999
% Error: Repository server-sftp could not be accessed. In case Backup was Restored on different setup, Please reconfigure the repository passwords (expected behaviour).
% Failure occurred during request

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-30-2024 10:42 PM
...yes, host key has been added:
# crypto host_key add host server-sftp
host key fingerprint added
Operating in CiscoSSL FIPS mode
# Host server-sftp found:line 1
server-sftp RSA SHA256:*******

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-01-2024 07:30 PM
It looks like TCP/22 is not allowed (blocked by firewall / ACL) between ISE and that SFTP server.
Instead of ping, see if you can get a response from doing an SSH from the ISE CLI, to the SFTP server (SSH/SFTP normally default to TCP/22)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-02-2024 04:23 AM
There could be several things such as:
#1: Firewalls/ACL between the ISE and sFTP server,
#2: iptables on the sFTP server itself,
#3: /etc/hosts.allow or /etc/hosts/deny on the sFTP server that prevents your ISE server to connect. Yes, it is there, in addition to the iptables itself,
The best thing to do is to create a dummy sFTP on the ISE with the same hostname/IP address as the actual sFTP server (named it dummy or something like that) and gives it the same username/pw of the sFTP server. After that, on the command, add the host key like "crypto host_key add host dummy.cisco.com" or "crypto host_key add host X.X.X.X". Once you confirmed that the key is successfully added, do a "show repository dummy" and you should see a listing of all the file in that directory of the username you specified when creating the "dummy" repository.
If you can't get the host key added in ISE, it means tcp/22 is being blocked somewhere. If you're able to successfully add the host key but can not view the repository, it means the sFTP server is likely implementing the /etc/hosts.allow or /etc/hosts.deny (assuming the username and pw is valid). Remember, tcpdump is your friend....

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-02-2024 10:03 PM
Based on TCPdump and manual SSH from ISE box into server of Repository, it seems that port TCP22 is allowed.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-02-2024 09:56 AM - edited 07-02-2024 09:59 AM
All those restrictions can be be ruled out (!)
I was even able to SSH into server of repository by using same credentials.
I also took a tcpdump and monitored incoming ssh connections:
Able to confirm the SSH connection between ISE box and repository has been established. However, ISE sends Finish and closes TCP connection without having any data sent.
12:40:10.817610 IP sftp-server.ssh > ise-box.37912: Flags [S.], seq 1378995211, ack 1513259488, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 8], length 0
12:40:10.820623 IP ise-box.37912 > sftp-server.ssh: Flags [.], ack 1, win 229, length 0
12:40:10.820996 IP ise-box.37912 > sftp-server.ssh: Flags [P.], seq 1:35, ack 1, win 229, length 34
12:40:10.821005 IP sftp-server.ssh > ise-box.37912: Flags [.], ack 35, win 58, length 0
12:40:11.026064 IP sftp-server.ssh > ise-box.37912: Flags [P.], seq 1:22, ack 35, win 58, length 21
12:40:11.029028 IP ise-box.37912 > sftp-server.ssh: Flags [.], ack 22, win 229, length 0
12:40:11.029860 IP ise-box.37912 > sftp-server.ssh: Flags [P.], seq 35:579, ack 22, win 229, length 544
12:40:11.029866 IP sftp-server.ssh > ise-box.37912: Flags [.], ack 579, win 62, length 0
12:40:11.032480 IP sftp-server.ssh > ise-box.37912: Flags [P.], seq 22:534, ack 579, win 62, length 512
12:40:11.042211 IP ise-box.37912 > sftp-server.ssh: Flags [P.], seq 579:851, ack 534, win 237, length 272
12:40:11.050075 IP sftp-server.ssh > ise-box.37912: Flags [P.], seq 534:1382, ack 851, win 66, length 848
12:40:11.093316 IP ise-box.37912 > sftp-server.ssh: Flags [.], ack 1382, win 250, length 0
12:41:11.089153 IP ise-box.37912 > sftp-server.ssh: Flags [F.], seq 851, ack 1382, win 250, length 0
12:41:11.095743 IP sftp-server.ssh > ise-box.37912: Flags [F.], seq 1382, ack 852, win 66, length 0
12:41:11.098674 IP ise-box.37912 > sftp-server.ssh: Flags [.], ack 1383, win 250, length 0
I hope we do not hit the following bug, though we do run Service Pack3:
Host: **
Personas: Administration, Monitoring, Policy Service (SESSION,PROFILER,DEVICE ADMIN)
Role: PRI(A), SEC(M)
System Time: Jul 02 2024 12:24:18 PM******
FIPS Mode: Disabled
Version:3.1.0.518
Patch Information: 3
ISE 3.1 certain SFTP servers stopped working after upgrade to patch 4/5
CSCwd89657
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-05-2024 03:32 PM
Eventually, I was able to fix the sftp upload issue by removing the configured from GUI at:
Admin > System > Maintenance > Repository
And creating the repository via CLI.
% Internal CA Store is not included in this backup. It is recommended to export it using "application configure ise" CLI command
% Creating backup with timestamped filename: test-sftp-CFG10-240705-1724.tar.gpg
% backup in progress: Starting Backup...10% completed
% backup in progress: Validating ISE Node Role...15% completed
% backup in progress: Backing up ISE Configuration Data...20% completed
% backup in progress: Backing up ISE Indexing Engine Data...45% completed
% backup in progress: Backing up ISE Logs...50% completed
% backup in progress: Completing ISE Backup Staging...55% completed
% backup in progress: Backing up ADEOS configuration...55% completed
% backup in progress: Moving Backup file to the repository...75% completed
% backup in progress: Completing Backup...100% completed
