IPS 5.0-4 on 4215 SSL Dropping Problem

[belenburg]

I have a 4215 running S188 monitoring two segments inline. I have been having problems with SSL connections. It seems that SSL connections are being lost or reset almost immediately. The browser acts like the session has timed out and we have to sign in to our web applications again. I don't see any events firing but when I remove the IPS, I don't have any problems.

[owillins]

Make sure that IP logging is disabled and that the IDS signatures are fine-tuned by making sure that unnecessary IDS signatures are disabled so as not to take cpu overhead.

[marcabal]

Are the SSL connections crossing both segments being monitored InLine?

If so then it may be causing confusion in the TCP State tracking being done by the sensor.

This is a known issue when trying to monitor 2 sides of a firewall inline and the firewall is doing tcp sequence number randomization.

For example the client is on the Internet connecting through the firewall to a Web Server on the DMZ.

The sensor is deployed inline on both the Internet side of the firewall and the DMZ.

The packets coming from the client will first be seen on the Internet side inline pair and so the sensor will begin tracking the connection using the sequence numbers from the client.

As the packet pass through the firewall the firewall will often randomize the sequence as it passes the packets on to the servers.

These modified packets pass through the second inline pair of the sensor and immediately the sensor detects that the packets have been modified.

The sensor's TCP Normalizer engine will immediately drop these packets assuming that the packets are being modified by an attacker. It does not know that the packets are intentionally being modified by the firewall.

This issue can sometimes be reduced by turning off the tcp sequence randomization being done by the firewall. The sensor will see the packets as duplicates without modification and will usually let the packets through.

Another way around it is to have the firewall use NAT so that the addresses in the packets differ on each side of the firewall. In this case the sensors see the packets as 2 entirely different streams and will analyze them separately and track their state separately.

Also be aware that even without the TCP sequence randomization being done by the firewall, there are still other possible areas that can cause issues.

The sensor is seeing duplicates of every packet. And for most of the time it can handle this properly. But some clients like to jump ahead in the TCP connection. The duplication of these jumps ahead in the TCP sequence numbers has the possibility of causing some confusion on the sensor and the sensor thinking that packets are coming in out of order.

You might be exceeding the sensor's performance thesholds. The sensor may be dropping packets if it can't keep up with the bandwidth being generated by the 2 inline pairs.

Also the sensor's performance numbers in the data sheets are based on HTTP traffic primarily. Other traffic types could yield differing performance numbers for the sensors.

So you will want to look at the output of "show interfaces" and look for any errors on the interfaces. Errors on the interfaces often are result of the sensor not being able to keep up with the traffic on the networks it is monitoring.

If this is the case then the sensor you have may not be able to handle the rates being produced on both network segments. You might need a second sensor for the second segment.

If using NAT then the sensor is treating the one connection as 2 connections. The sensor has limits on the number of connections it is able to monitor Generally it can monitor more connections than the bandwidth it can monitor so usually bandwidth is the limiting factor. But if you have a network with a large number of connections compared to bandwidth then you may hitting the upper limit of the sensor's connection tracking. A higher performing sensor would be needed, or a second sensor for the second segment.

Continued in next post.

[marcabal]

Are your connections long lived with not much data being passed? If so then you may be running into the sensor's handling of idle connections. If a TCP connection stays idle too long it will remove it from it's connection table to make room for new connections. You can increase the timer by modifying the "TCP Idle Timeout" parameter in signature 1301. You can also increase the "TCP Embryonic Timeout" in signature 1302. And the "TCP Closed Timeout" in signature 1303.

But first I would suggest adding the produceAlert action to these signatures and monitoring to see if they are firing. This will give you a better understanding if your connections are hitting these timeouts.

As for why you are not seeing alerts. The majority of the Normalizer signatures will not generate alerts by default.

You can go into IDM on the Signature configuration page. Select the signaturs by engine for engine normalizer. Then select the actions button and add produceAlert action to all the signatures.

Almost all of the situations I described above will cause one or more of the Normalizer signatures to be triggered.

Most of these have the produceAlert action removed by default because they can fire often in a normal network just cleaning up (i.e. Normalizing) the traffic on the network even when no hacker specific traffic is being seen. It is only in situations like yours where you are seeing a network issue do I recommend turning them on to help in debugging the issue.

[belenburg]

As part of my troubleshooting, I check the CPU utilization and it never seems to be over 2 - 4 percent. I do have IP logging enabled but would that be a cause if the CPU utilization is very low?

[marcabal]

I don't think the IP Logging would be causing you any problems with such low cpu utilization.

But you could create a simple filter to remove the log..Packet actions. (Leave the other fields as default so they match every alert).

And then see if makes any difference. If you don't see a difference then delete the filter.

This is easier than going to each signature and removing the action then trying to remember which signature to add it back in for.