Reviewing File Scans on v5.0+ AMP for Endpoints

[aledipas]

Hi Everyone,

In version 5.0 we made changes to the way files and paths are stored. Our existing articles will be updated soon and we apologize for the delay. 

The files and paths are now stored in an referential structure in the historyex.db file. Attached is a python script which will rebuild the paths and convert the file in to a CSV, so that you can review the scanned files for exclusions within your environments. Ideally you will want to use this in a linux environment. The Ubuntu on Windows 10 environment also works well if you choose to use that.

Script syntax is fairly simple:

./historyex_to_csv.py historyex.db > historyex.csv

Please note TAC will not support this script or any changes made to it but feel free to ask questions here in the discussion forum.

Please also review the new script for converting *debug* sfc.exe.log files for seeing even more verbose file activity on your Windows endpoints. The historyex.db method primarily only shows executable file types and may not be sufficient for your environment.

Place the handle_count.sh script in the same folder as your sfc.exe.log files and simply run the script without arguments. Please note that this is a bash script and should work on Ubuntu, OSX, and Ubuntu on Windows 10:

./handle_count.sh

After the script is run you will have several files. The most important file to review is the "data.csv" file. This will contain file operations as well as the source process.

Thanks!

[jplopper]

When running "./historyex_to_csv4.py historyex.db > historyex.csv", I am getting the error below.  Thoughts?

Traceback (most recent call last):
File "./historyex_to_csv4.py", line 20, in <module>
for row in c_path_history.execute("SELECT path, hash, lastref, type FROM path_history ORDER by lastref"):
sqlite3.OperationalError: no such table: path_history
jplopper@ubu-jplopper1:~/AmpTshoot$ ./historyex_to_csv4.py historyex.db > historyex.csv
Traceback (most recent call last):
File "./historyex_to_csv4.py", line 20, in <module>
for row in c_path_history.execute("SELECT path, hash, lastref, type FROM path_history ORDER by lastref"):
sqlite3.OperationalError: no such table: path_history

[jplopper]

I'm dumb.  https://stackoverflow.com/questions/12103244/no-such-table-error put me onto the answer.  When I extracted the files from the diagnostic file, it named them according to the path.  In my case it was "C:\Program Files\Cisco\AMP\historyex.db".  New to Linux.

[aledipas]

Your historyex.db may be corrupt/unreadable per those sqlite errors. It may be best to stop the A4E service, delete it, then re-do your testing.

[epwilson2]

I had to change the script for this to work with Python3. Python3 changes the print statement to a print() function. In order to fix this you will need to add parenthesis around the text that will be printed. Example below.

OLD: print "Missing historyex.db"

NEW: print ("Missing historyex.db")

[Troja007]

Hi all,

hay anyone a hint for me?? I´m getting no error message, but the *.csv file has always a size of 0kb.

Any Ideas?

Cheers

[David Janulik]

Hi

please open a ticket and get this escalated to Tier3. We can get backend logs filtered to see most frequent lookups per day/connector.

David