V4.X Sensors Stopped Reporting Events

[paulhignutt]

I have 4 4.X sensors, two IDSM2s and two 4235s. All of a sudden 3 of them stopped reporting alerts. I have reset them and still nothing. Event Monitor shows "Connected TLS" under connection status. I have issued the command "show event alerts past 01:00" from the CLI and still nothing. The only thing in common is that I upgraded the signatures to S78 the other day, but the one is still going at it just fine while the others are doing nothing. Any ideas what would cause this?

[marcabal · Accepted solution]

Have you tried running "show version" and ensured that AnalysisEngine is still running?

If AnalysisEngine is not running, then this would explain why no alerts are being seen, but points to a software issue of some kind. You would need to contact the TAC if AnalysisEngine is not running.

You could try downgrading back to the previous version while waiting for TAC assistance. There is always a small possibility that the new signatures introduced in S78 may be causing an issue we don't know about.

(You may even try the downgrade anyway to see if it starts monitoring again)

If AnalysisEngine is still running, then you will need to check a few other things:

Check the statistics for the interfaces "show interfaces". Run the command a few times and ensure that the packet counts on the monitoring interfaces and for the analysis are increasing. There is also a counter for the number of alarms that have been generated. See if this counter is increasing.

If the packet counts are not increasing then check your sensor configuration for the interfaces (make sure they are assigned to the interface group and are enabled), and check the physical connections to your hub/switch.

If you get an error about not being able to communicate with analysisEngine or sensorApp (a timeout) then you could be experiencing a software bug. Wait a few minutes and try again. If the error stays then contact the TAC.

If you get an error about no sensing interfaces or no virtual sensor or interface group, then check your configuration and make sure you have interfaces assigned to the group and enabled.

If the alarm count is increasing, but still no alarms in "show events" then there may be a communication problem between sensorApp and the eventStore that we have never seen before.

Other things to try:

Enable the 994 and 995 alarms in Engine OTHER. These alarms let you know when the sensor starts or stops seeing traffic.

Try resetting the sensor.

Then try the show events and see if either of these alarms showed up in show events.

Check the filter configuration on your sensor.

Every now and then we hear of a user that accidentally entered in a filter that filtered out ALL of their alarms.

By default the filter will wildcard all of the fields. So if you have created a filter but never put in specific data for a field, then you could have accidentally told the sensor to filter out ALL of your alarms. In which case you need to remove that filter to get things going again.

[paulhignutt]

I had to downgrade them to start the alerts again. I had previously reset the boxes before the downgrade, and that didn't work. The thing I think is strange about this is that I had 1 other box that was identical to one of the ones that quit working, and it worked fine witht he signature upgrade. My biggest complaint about this IDS of Cisco's is the gross lack of documentation of how it works at a nuts and bolts level, and how to implement it in a complex environment. At least the V4.X sensors actually somewhat work most of the time as opposed the the V3.X. Thanks for you help. Paul

[ulysses thomas]

If you are using IEV to monitor your upgraded 4.1 IDS sensors you will need to upgrade it using IEV-sig-4.1-1-S80.exe.

I had the same problem you described until I upgraded IEV. Clicking (About) shows IEV version. The tipoff was the red x next to the sensor name. Also sh interface showed the sensor was in fact working, IEV was not logging the alerts.

BTW

The Cisco IDS Course explains everything at an excruciating nuts & bolts level if you like. Multiple sensors, multiple interfaces, Campus configuration, internet configuration, SMB configuration, enterprise configuration and so forth.

[paulhignutt]

I use VMS Event Monitor and IDSMC.

As to the course, that'd be great and I'm looking forward to going. However, I'm sore at Cisco for not providing adequate documentation for their product. After you pay a bazillion dollars for the gear, adequate documentation should be available and you shouldn't have to go to a training course to get it. That is unless that course is free, which it should be given the cost of this equipment. I think it would be great if they would offer even an "E-Course" for free to those who have purchased the equipment.