Purging the database does not purge completely

vaibhav.parlekar1 · ‎03-27-2017

Hi,

We are seeing a weird behaviour of information being retained on the FMC even after purging the database. E.g. We have purged all the connections & host information from the purge tab under system. I have also deleted all the logs from the context explorer.

Now in the context explorer the if the log view setting is set to past 1 hour we can see no data being loaded as expected. But if we set the setting to a 1 day or 1 week we can see the application data in the context explorer. But there are no logs pertaining to the applications if we drill into analysis of those applications.

Similar behaviour is seen on the summary dashboard with regards to the log view setting.

Is this a bug or is there a command on the FMC to purge the historical data completely ?

I could not find any reference to this behaviour in the documentation. Any help on the same would be great.

vaibhav.parlekar1 · ‎03-29-2017

hi team,

any take on this one. Tried purging the database multiple times from the FMC but the application data from the network analysis still persists. We are not able to troubleshoot the statistics of application data.

Mark Alley · ‎04-25-2017

I have the same problem. Ever come up with a solution?

vaibhav.parlekar1 · ‎05-03-2017

Hi Mark,

I tried the procedure multiple times and only find that the logs are purged from the context explorer and not from the dashboard. the statistics of the applications remain as it is.

This is really bad and there is no documentation around it and neither any notes about this behaviour nor any help on the community around it.

I think there is a Cli command in the linux shell to purge it completely but I guess only CiscoTac is aware of it.

vaibhav.parlekar1 · ‎06-28-2017

I think there is a Cli perl script called managed_pruning but I don't know the syntax of running this script on the fmc. Also there is a bug associated with it even in 6.2.0 with bug id CSCvc51459

Not sure if this issue is resolved yet.

Marvin Rhoads · ‎06-30-2017

The script is actually called "manage_pruning.pl". As a perl script, you can simply run it as-is from the cli. It is located and has options as follows:

admin@sfvdc:/var/sf/bin$ pwd
/var/sf/bin
admin@sfvdc:/var/sf/bin$ sudo manage_pruning.pl
**************** Configuration Utility **************
 1 Status
 2 Prune configured items
 3 Purge Event database & (2)
 4 Print Status File
 0 Exit
**************************************************************
Enter choice: 0
Thank you
admin@sfvdc:/var/sf/bin$

I don't know if or how one can customize the "configured items".

Selecting option 2 results in the following (and the dashboard retains its application history and statistics):

**************************************************************
Enter choice: 2
Deleted file /var/tmp/pruning_status.msg
EOStore Pruner: DNSListObject.DataHandler 345 deleted
EOStore Pruner: Dashboard.DataHandler 5 deleted
EOStore Pruner: DashboardWidget.DataHandler 2 deleted
EOStore Pruner: IDSRule.DataHandler 16 deleted
EOStore Pruner: IDSRuleImport.DataHandler 5 deleted
EOStore Pruner: IPListObject.DataHandler 345 deleted
EOStore Pruner: IntrusionPolicy.DataHandler 3 deleted
EOStore Pruner: NetworkAnalysisPolicy.DataHandler 2 deleted
EOStore Pruner: SnortAttribConfig.DataHandler 4 deleted
EOStore Pruner: URLListObject.DataHandler 322 deleted
EOStore Pruner: VariableSet.DataHandler 1 deleted
DBD::SQLAnywhere::db prepare failed: Syntax error near 'limit' on line 4 (DBD: prepare failed) at /usr/local/sf/lib/perl/5.10.1/SF/Pruning/ProcessDB.pm line 810.
ERROR: Unable to prepare HASH(0x136d7438) at /usr/local/sf/lib/perl/5.10.1/SF/Pruning/ProcessDB.pm line 812.
DBD::SQLAnywhere::db prepare failed: Syntax error near 'limit' on line 4 (DBD: prepare failed) at /usr/local/sf/lib/perl/5.10.1/SF/Pruning/ProcessDB.pm line 810.
ERROR: Unable to prepare HASH(0x135e6460) at /usr/local/sf/lib/perl/5.10.1/SF/Pruning/ProcessDB.pm line 812.
DBD::SQLAnywhere::db prepare failed: Syntax error near 'limit' on line 4 (DBD: prepare failed) at /usr/local/sf/lib/perl/5.10.1/SF/Pruning/ProcessDB.pm line 810.
ERROR: Unable to prepare HASH(0x137173c0) at /usr/local/sf/lib/perl/5.10.1/SF/Pruning/ProcessDB.pm line 812.
DBD::SQLAnywhere::db prepare failed: Syntax error near 'limit' on line 4 (DBD: prepare failed) at /usr/local/sf/lib/perl/5.10.1/SF/Pruning/ProcessDB.pm line 810.
ERROR: Unable to prepare HASH(0x136d7438) at /usr/local/sf/lib/perl/5.10.1/SF/Pruning/ProcessDB.pm line 812.
**************** Configuration Utility **************
 1 Status
 2 Prune configured items
 3 Purge Event database & (2)
 4 Print Status File
 0 Exit
**************************************************************
Enter choice:

vaibhav.parlekar1 · ‎06-30-2017

Thanks a lot for this Marvin,

I see there are issues when running this script.

Unable to execute SELECT count(*) as count FROM SRU_import_log () at /usr/local/sf/lib/perl/5.10.1/SF/Pruning/ProcessDB.pm

I didn't get the other errors you got when running the script.

I tried both the options 2 & 3 & yes you are right. the application & connection statistics remain as it is in the dashboard.

It's clear there is a bug with the purging of the database. If the dashboard still shows statistics it means there are recorded somewhere in the database.

My experience with the FMC is very bad too buggy & clunky software.

Marvin Rhoads · ‎07-01-2017

I suspect the script has not been updated as the database schema has been modified over several releases.

Which errors you get probably depends on how many versions you are past the one for which the script was created.

I agree the software is in need of a makeover. Under the covers there is a LOT of legacy code. The new FTD bits haven't helped as there are now pieces of the old Cisco Security Manager (CSM) embedded into FMC.

I keep hoping some of the Meraki (or Viptela) approach to UI design and functionality can be incorporated into the Cisco secuirty product line. We shall see....