Cisco Catalyst Center Backup Bug CSCwe31960

[Dominic Stalder]

Hi guys

we are hitting this bug right here: CSCwe31960

I would like to apply the workaround described, but I don't know how to get to this point, where I can edit the following:

> For 2.2.2.x release please do the following: kubectl -n ndp edit sts elasticsearch search for the SNAPSHOT and edit SNAPSHOT_MAX to 5 and SNAPSHOT_CORE 5

I can login with maglev to the Cisco Catalyst Center, but in which context I have to go to run this command above?

Thanks and best regards

Dominic

[Torbjørn]

If you are on 2.2.2.X or an earlier version you should be able to enter privileged shell with "_shell".

[maflesch]

"_shell" doesn't exist in anything prior to 2.3.x.x. There is no restricted shell in 2.2.3.x or below. From what it sounds like, you are on 2.3.x.x code which has the restricted code. 

This issue has also only been seen by one other customer. So I'm hesitant that you are actually hitting this. Can you provide the following please:

for node in 0 1 2 ;do curl -s elasticsearch-$node.elasticsearch.ndp:9200/_cluster/health?pretty;done
maglev release status
maglev system_updater update_info
maglev backup history

[Dominic Stalder]

Hi maflesch

where do I have to execute the commands you provided?

Best regards

Dominic

[Dominic Stalder]

Hi maflesch

I think you are correct, we did not hit this bug; unfortunately it is my bad: the NFS quota was full. After deleting the whole share, I was able to re-complete an adhoc backup:

But maybe you can answer me the following questions too:

1. would like to have an automated data cleanup on this NFS share initiated by Cisco Catalyst Center. But based on my knowledge, there is no way to accomplish that at the moment, correct?

2. I found this 4 years old post: https://community.cisco.com/t5/cisco-catalyst-center/dna-backup-houskeeping/td-p/4007804

There a solution is provided by manually delete specific folders on the NFS share, but I still don't get it a 100%...

- The first scheduled backup is a full backup, correct?

- The following backups are incremental, correct?

- Do I need to keep the original full backup the whole time?

- And when I delete an (intermediate) incremental backup, what happens with the integrity of the whole backup process?

- And would I need to get the backup Id manually from the GUI to be able to delete incremental backups?

I would very much appreciate any help on this topic, because I don't get it right now

Thanks a lot

Dominic

[maflesch · Accepted solution]

Hi Dominic,

Glad to hear you were able to take a backup. Yes, there is a feature for backup retention in place. This currently exists in our ESXi Virtual Appliance (VA) platform only. For our On-Prem (physcial appliance) and AWS product, this does not exist yet.

With that said, we will soon be transitioning our On-Prem and AWS product code to the same code base that the ESXi VA runs on. When that happens, the On-Prem and AWS will have the backup retention feature as well. This should start to happen around the beginning of the new year and be pushed out to everyone closer to March - April of next year, pending nothing critical stops it.

[Dominic Stalder]

Hi maflesch

Thanks a lot for this detailed feedback and your kind help! Looking forward to the feature

Best regards and good luck with the working process of getting there

Dominic