FTP log files

[jason.fletcher]

Good day, I am using the IDSM with 3.0(3)S13 in conjunction with CSPM 2.3.3i. I know that FTP can't be configured from the GUI, and I have manually edited the sapd.conf file for each sensor to force the log file FTP. If I wanted to FTP the log file from the IDSM say every 2 minutes, regardless of the current log file size, to populate an outside database in near real-time, how would I accomplish this? Is it possible to tune the time interval between FTPing the log files? Any info would be greatly appreciated.

[marcabal]

You can't directly tell the IDSM how often to ftp the log files because the ftp is triggered by the closing of the previously open log file.

But you can indirectly do this by having the log file close and new one opened every 2 minutes.

In the loggerd.conf file there is a token to designate how long a log file should be opened. You would need to edit the loggerd.conf template on CSPM and change the default value to two minutes.

Realize that there is also another configuration token for the maximum size of the log file. If the log file hits the max size before the max time is reached the log file will be closed and ftp'd before the max time is reached. So the log file could be ftp's when the max time is reached or ftp'd prior to that if the max size is reached.

BUT I warn you now that this could cause some issues for you. IDSM and sapd have been tested with the default log file times to ensure that there is no disk problems with high alarm rates and high log files counts. By changing the log file time to such a small amount you are adding potential for problems. It could happen that loggerd starts producing a large number of log files (1 every two minutes) that are really small, and sapd may not be able to keep up. Sapd mainly deletes log files when they start to fill up the disk space, and does not usually delete based on the number of files. So if the files are really small they could take up little disk space but could consume all of the available inodes and cause disk problems. (I believe we put in code in sapd to take care of this issue, but I cant remember for sure.)

One of the reasons CSPM does not allow the configuration of the max time and max size tokens is because of the potential problems and the time it would take to test all of the possible variations on the sensor.

If you try this you may not see the affect of this for several days, so I suggest trying this on only one IDSM initially if you can, and letting it run for a week or two and see if any problems develop.

If everything still looks good after a week or two then maybe deploy it to your other IDSMs.

Marco

[jason.fletcher]

Thanks Marco, I am going to give this a shot and maybe play around a little with the timing and what not. Basically we want to put a web frontend up in near real time and start tying some of our security pieces together on the backend so that we can get a better picture of what is happening. The best way I can think of to do this is to build our own database, hence the ftping of log files from the IDSM into flat files and importing them into a database that we can customize and tie to syslog and TCPdump info as well as system info, etc. I like the log files produced by the IDSM because I can also see the shun commands that are being used so we can maintain a nice list of shuned IP's and we can tie that IP to the traffic that caused the shuning on the backend as well as a history of other alarms generated by the IP. Thanks for the info and I'll let you know how it goes.