Backup of ACI fails with Fault F0053 Upload Failed, Remote file not found

[joeharb]

Trying to backup newly deployed ACI fabric and it fails will an error code:

Fault delegate: Configuration backup/restore job 2020-09-16T09-42-04 failed with error: Upload failed, Remote file not found

I can upload files to the sftp server without issue.

Any help would be appreciated.

Thanks,

Joe

[Robert Burns · Accepted solution]

Likely a syntax or path issue.  Can you provide the following CLI Output:

moquery -c fileRemotePath

moquery -c configExportP

moquery -c configRsExportDestination

Identify the name of the 'Remote Location' (if you have more than one configured), as well as the name of the Config Export policy you're using (if more than one)

Robert

[joeharb]

Spot on Robert, the path had a typo in it that I didn't notice.

Thanks,

Joe

[rjakabcisco]

Hello,

I have similar problem but it doesn't look like a path issue. I'm getting the same error when I try to upload the snapshot manually  using GUI either through Admin > Config Rollbacks > Actions - Create Snapshot now or through Admin > Import/Export > Snapshot Management > Policy - Start Now: Upload failed, Remote file not found

When I connect to SFTP server from APIC using the same path manually everything works, I'm also able to upload files:

apic2# sftp sftp://svc_backup_aci@10.141.32.9:22//Backups/Network/aci/
svc_backup_aci@10.141.32.9's password: 
Connected to 10.141.32.9.
Changing to: /Backups/Network/aci/
sftp> pwd
Remote working directory: /C:/Backups/Network/aci/
sftp> put /tmp/tar_verify.log
Uploading /tmp/tar_verify.log to /C:/Backups/Network/aci/tar_verify.log
tar_verify.log
sftp> ls -l
-rw-******    1 -        -            1221 Aug 14 11:48 tar_verify.log
sftp> rm tar_verify.log
Removing /C:/Backups/Network/aci/tar_verify.log
sftp> exit
apic2#

CLI output of required commands (only the relevant part):

apic2# moquery -c fileRemotePath

# file.RemotePath
name : backup-server
annotation :
authType : usePassword
childAction :
descr :
dn : uni/fabric/path-backup-server
epgDn : uni/tn-mgmt/mgmtp-default/oob-default
extMngdBy :
host : 10.141.32.9
identityPrivateKeyContents :
identityPrivateKeyPassphrase :
identityPublicKeyContents :
lcOwn : local
modTs : 2024-08-14T11:53:10.425+02:00
monPolDn : uni/fabric/monfab-default
nameAlias :
protocol : sftp
remotePath : /Backups/Network/aci/
remotePort : 22
rn : path-backup-server
status :
uid : 15374
uri : sftp://10.141.32.9:22//Backups/Network/aci/
userName : svc_backup_aci
userPasswd :
userdom : :all:
vrfName :

apic2# moquery -c configExportP

# config.ExportP
name : SFTP-backup
adminSt : untriggered
annotation :
childAction :
descr : test
dn : uni/fabric/configexp-SFTP-backup
extMngdBy :
format : json
includeSecureFields : yes
internalSource : no
lcOwn : local
maxSnapshotCount : global-limit
modTs : 2024-08-14T11:48:41.963+02:00
monPolDn : uni/fabric/monfab-default
nameAlias :
rn : configexp-SFTP-backup
seqNum : 2
snapshot : no
status :
targetDn :
triggerTime : 2024-08-14T11:48:38.864+02:00
uid : 15374
userName : admin
userdom : :all:

apic2# moquery -c configRsExportDestination
No Mos found

Do you have any suggestions?

Thanks,

Robert

[rjakabcisco]

Found a solution,

this is a bug related to Windows server with OpenSSH.

The APIC uses libcurl to push the files where an extra "/" is added to the sftp URL between the server IP-or-hostname and the file path. It looks like Linux servers are fine with handling the "//" whilst Windows/OpenSSH servers are not. You can find the URL constructed by the APIC by looking at the backup destination config page in the GUI. For example:

sftp://10.141.32.9:22//Backups/Network/aci/

When we configure the backup destination in the APIC GUI, the GUI enforces us to put a leading "/" in the destination path which creates "//" in the URL. The good news is that we can put a path without the leading slash via API, and that's the workaround for this issue.

After creating the backup destination in APIC GUI we need to modify the path to not include the leading "/".

POST example:

• backup-server is the name of the Remote Location in APIC and needs to be specified as part of dn attribute in XML body - path-backup-sever
  
• backup-server password needs to be included in the POST body otherwise the change will not be accepted by APIC

POST https://10.141.32.9/api/mo.xml

<fileRemotePath dn='uni/fabric/path-backup-server' remotePath='Backups/Network/aci' userPasswd='backup-server-password'/>

After changing the path via API "//" is replaced with "/" in constructed URL and the connection starts to work:

sftp://10.141.32.9:22/Backups/Network/aci/

Robert