DNA Backup houskeeping

[KristofB]

Hi,

Our DNA backups are scheduled daily.

Analytics (NFS based) backup seem to be incremental, but automation data (SFTP based) backup are not. Those backups add 15GB to our server every day… How can I keep control of the used space?   

I could create a script to delete backups on the server, but what do I have to delete? Backup seem to be divided over 8 folders? And would DNA sync to that removal? 

Or can I script something from DNA to delete old backups? (via API's?)  

Thank you!

Kristof

[Mariusz Kazmierski · Accepted solution]

Hi Kristof, 

The current logic is that indeed we create multiple directories, but then in these directories we create dedicated folders for each backup (identified by backup-id, below I have three backups with IDs:438213ea-3d5e-4428-8602-2440136d8478, 1b9a37ab-6329-4037-84c1-fd8734cbf41b, 8b5ac05f-5e7b-43a5-a801-013ba7e0f6de):, e.g.:

$ find -maxdepth 2 -type d -ls
   392470      4 drwxrwxr-x   9 dnac     dnac         4096 Jan  8 09:00 .
   392533      4 drwxrwxr-x   5 dnac     dnac         4096 Jan  8 09:00 ./maglev-system.credentialmanager
  1570337      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  8 09:00 ./maglev-system.credentialmanager/438213ea-3d5e-4428-8602-2440136d8478
  1578782      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  7 22:46 ./maglev-system.credentialmanager/1b9a37ab-6329-4037-84c1-fd8734cbf41b
  1578801      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  7 23:30 ./maglev-system.credentialmanager/8b5ac05f-5e7b-43a5-a801-013ba7e0f6de
   392473      4 drwxrwxr-x   5 dnac     dnac         4096 Jan  8 09:00 ./app-hosting.postgres
   785642      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  8 09:00 ./app-hosting.postgres/438213ea-3d5e-4428-8602-2440136d8478
   790341      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  7 22:46 ./app-hosting.postgres/1b9a37ab-6329-4037-84c1-fd8734cbf41b
   790344      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  7 23:30 ./app-hosting.postgres/8b5ac05f-5e7b-43a5-a801-013ba7e0f6de
   392532      4 drwxrwxr-x   5 dnac     dnac         4096 Jan  8 09:00 ./maglev-system.glusterfs
  2095515      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  8 09:01 ./maglev-system.glusterfs/438213ea-3d5e-4428-8602-2440136d8478
  2095817      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  7 22:47 ./maglev-system.glusterfs/1b9a37ab-6329-4037-84c1-fd8734cbf41b
  2097485      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  7 23:31 ./maglev-system.glusterfs/8b5ac05f-5e7b-43a5-a801-013ba7e0f6de
   658620      4 drwxr-xr-x   2 dnac     dnac         4096 Jan  8 09:01 ./backup-metadata
   393413      4 drwxrwxr-x   5 dnac     dnac         4096 Jan  8 09:01 ./fusion.postgres
   393028      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  8 09:01 ./fusion.postgres/438213ea-3d5e-4428-8602-2440136d8478
   393422      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  7 22:47 ./fusion.postgres/1b9a37ab-6329-4037-84c1-fd8734cbf41b
   407472      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  7 23:31 ./fusion.postgres/8b5ac05f-5e7b-43a5-a801-013ba7e0f6de
   392531      4 drwxrwxr-x   5 dnac     dnac         4096 Jan  8 09:00 ./maglev-system.mongodb
   392534      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  8 09:01 ./maglev-system.mongodb/438213ea-3d5e-4428-8602-2440136d8478
   393419      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  7 22:47 ./maglev-system.mongodb/1b9a37ab-6329-4037-84c1-fd8734cbf41b
   393421      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  7 23:31 ./maglev-system.mongodb/8b5ac05f-5e7b-43a5-a801-013ba7e0f6de
   392471      4 drwxrwxr-x   5 dnac     dnac         4096 Jan  8 09:00 ./ndp.redis
  2095509      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  8 09:00 ./ndp.redis/438213ea-3d5e-4428-8602-2440136d8478
   658624      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  7 22:46 ./ndp.redis/1b9a37ab-6329-4037-84c1-fd8734cbf41b
   658638      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  7 23:30 ./ndp.redis/8b5ac05f-5e7b-43a5-a801-013ba7e0f6de

To check backup-id you need to hover over backup-name in UI. 

Currently there is no official API to handle backups but indeed you can handle it on the server side. 

By removing all folders + associated backup-metadata json file in directory ./backup-metadata via cmd:

rm -rf */1b9a37ab-6329-4037-84c1-fd8734cbf41b*

leading to:

$ find -maxdepth 2 -type d -ls
   392470      4 drwxrwxr-x   9 dnac     dnac         4096 Jan  8 09:00 .
   392533      4 drwxrwxr-x   4 dnac     dnac         4096 Jan  8 09:05 ./maglev-system.credentialmanager
  1570337      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  8 09:00 ./maglev-system.credentialmanager/438213ea-3d5e-4428-8602-2440136d8478
  1578801      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  7 23:30 ./maglev-system.credentialmanager/8b5ac05f-5e7b-43a5-a801-013ba7e0f6de
   392473      4 drwxrwxr-x   4 dnac     dnac         4096 Jan  8 09:05 ./app-hosting.postgres
   785642      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  8 09:00 ./app-hosting.postgres/438213ea-3d5e-4428-8602-2440136d8478
   790344      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  7 23:30 ./app-hosting.postgres/8b5ac05f-5e7b-43a5-a801-013ba7e0f6de
   392532      4 drwxrwxr-x   4 dnac     dnac         4096 Jan  8 09:05 ./maglev-system.glusterfs
  2095515      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  8 09:01 ./maglev-system.glusterfs/438213ea-3d5e-4428-8602-2440136d8478
  2097485      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  7 23:31 ./maglev-system.glusterfs/8b5ac05f-5e7b-43a5-a801-013ba7e0f6de
   658620      4 drwxr-xr-x   2 dnac     dnac         4096 Jan  8 09:05 ./backup-metadata
   393413      4 drwxrwxr-x   4 dnac     dnac         4096 Jan  8 09:05 ./fusion.postgres
   393028      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  8 09:01 ./fusion.postgres/438213ea-3d5e-4428-8602-2440136d8478
   407472      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  7 23:31 ./fusion.postgres/8b5ac05f-5e7b-43a5-a801-013ba7e0f6de
   392531      4 drwxrwxr-x   4 dnac     dnac         4096 Jan  8 09:05 ./maglev-system.mongodb
   392534      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  8 09:01 ./maglev-system.mongodb/438213ea-3d5e-4428-8602-2440136d8478
   393421      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  7 23:31 ./maglev-system.mongodb/8b5ac05f-5e7b-43a5-a801-013ba7e0f6de
   392471      4 drwxrwxr-x   4 dnac     dnac         4096 Jan  8 09:05 ./ndp.redis
  2095509      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  8 09:00 ./ndp.redis/438213ea-3d5e-4428-8602-2440136d8478
   658638      4 drwxrwxr-x   3 dnac     dnac         4096 Jan  7 23:30 ./ndp.redis/8b5ac05f-5e7b-43a5-a801-013ba7e0f6de

all specific data should be removed and Cisco DNA-Center will get it reflected in UI (you might need to wait a bit and/or click refresh on backup screen). 

Hope it helps! Let me know if you have any further queries. 

Regards,
Mariusz

[rasmus.elmholt]

But the recommendation is not to remove files from the backup server itself but through the DNAC GUI or API?

[t_s]

Hi,

Feel free to use the following script to automate this process:
tslenter/Cisco-DNA-Backup-Cleanup: Cisco DNA server side cleanup (github.com)

The backup location should run linux with a BASH shell.

Regards,

Tom

[rasmus.elmholt]

Hi Kristof

Some of the internal Cisco guys has developed this script I have been using with great success:

https://pypi.org/project/ciscodnacbackupctl/

[mjohnson3150]

Hi Rasmus,

That looks very promising. I installed it but I'm trying to figure out how to use it. There is scant little of instruction. Any tips? Thanks.

[rasmus.elmholt]

Hi,

Sorry about the late response. Follow the instructions on the page and it should would.

ciscodnacbackupctl config --help
Usage: ciscodnacbackupctl config [OPTIONS]

Options:
  --env / --no-env
  --hostname TEXT   [required]
  --username TEXT   [required]
  --password TEXT   [required]
  --secure          Secure HTTPS towards Cisco DNA Center
  --encode          Encode Cisco DNA Center config to Base64 string
  --overwrite       Writes over the existing config for Cisco DNA Center
  --help            Show this message and exit.

ciscodnacbackupctl daemon start --keep 30d
#This will keep backups for the last 30 days

If you have any issues just let the guys behind know on their github page.

[Sylvain_Che]

Hey @rasmus.elmholt 

This ciscodnacbackupctl script works pretty well but I'm facing a concern.

I started the daemon (--keep 3) and configured the schedule_purge (--keep 3 as well). But if I close my SSH session to the Linux server where the script runs, the script action seems to be aborted. The purge doesn't happen.

Any clue how to circumvent this?

Thanks,

Sylvain.

[mjohnson3150]

Hi @Sylvain_Che 

I may be misunderstanding what you're trying to do but my understanding is that if you're going to use the schedule_purge option is that you'd need to keep your session open until it completes. But if you've already setup the daemon to do that going forward, why would you need to use this option? The daemon would take care of it. I don't think it's documented but it looks like the daemon does its purge weekly. In my case, I have backups scheduled on our DNA box to run every Saturday and when I check things on Monday morning with the list option, I noticed that the purge has taken place. (I'm also using --keep 3 with my daemon). 

[Sylvain_Che]

Hi,

I didn't understand that the script was supposed to work this way.

Tried to run only the daemon and it does the job as expected.

Thanks for the tips.

Sylvain.

[rks113]

Hi,

Question: Does it keep for the last 3 backups?

ciscodnacbackupctl daemon start --keep 3

[rasmus.elmholt]

Hi,

I have not tried what you are doing but I think mjohnson3150 is correct. If you have any concerns or things you think is not working please open an issue on the github repo.

[Rajesh Kongath]

Hi @Sylvain_Che & @mjohnson3150 

I have a newbie question, sorry but I am new to Linux.

I would like to know where you are running the tool? on DNAC appliance itself or on the remote backup server. 

Appreciate your support. 

[t_s]

tslenter/Cisco-DNA-Backup-Cleanup: Cisco DNA server side cleanup (github.com) needs to be run on the remote server side (Backup server) (Not Cisco DNA)

[rasmus.elmholt]

In most installations we have done. We have added the tool on the same server we save the backup on.

DO NOT RUN IT ON THE DNAC.