FirePower 8140 performance

[gaboughanem]

hello,

I have a 8140 firepower sensor that is supposed to handle IPS 6G of throughput as per the datasheet.

This sensor is positioned at the edge of a small ISP serving internet traffic from dsl, 2g, 3g, etc...

This sensor is licensed for IPS license only (control_protection).

The ISP traffic is around 3.6G to 3.8G total inbound and outbound at peak time.

The access control is a simple with one rule (any any) with an action of inspecting traffic using a load balance security and connectivity intrusion policy (with no drop when inline....till now )

Firesight version is 5.4.1.5 , 8140 is 5.4.0.7

Since the sensor was deployed:

1- The cpu of its 14 cores (14 snort engines) are very high with an average over 85%.

2- many complaint received during peak hours that the connection is slow.

After monitoring, The traffic throughout is being bottlenecked by the sensor during peak hour and the interface speed on the sensors is not exceeding 3G Total.

3- facing issue on the firesight (Virtual Machine) logs, were the logs are being drained for some reason.

Simple Troubleshooting:

1- disable the access policy rule and keep it to its default action "network discovery":

a- lessens the cpu to around 30%

b-the traffic graph is back to normal of around 3.7G Total at peak hours....no end user complaint here.

2- applying an access policy rule with an intrusion policy of Connectivity over security (around only 500 signature) during peak or non-peak hours, did not change anything from high cpu or throughput choking.

I opened a tac case for a while now, and the latest info are :

1- that the the box's limit is being reached, with no further analysis, (limit: meaning it is restricting traffic maximum to 3G).

2- for the log draining, tac said that the firesight is not keeping up with the sensors.

Tac requests:

1- Restrict traffic to the sensor.

2- avoid some kind of traffic

3- limit some kind of traffic

4- try to trust small packets.

Actually, the above suggestions are not possible (except for suggestion number 4).

In summary,

any ideas?

has anybody faced this issue before?

the datasheet is 6G but the sensor is not allowing more than 3G,,,,why?

Regards,

George

[Ed Padilla Jr]

run Rule Profiling and find out what signatures are taxing the most your load.

[Jetsy Mathew]

Hello Team,

You have to confirm the rules which is taking more resource.

Its better to collect the following data and contact TAC.

http://www.cisco.com/c/en/us/support/docs/security/sourcefire-firepower-8000-series-appliances/118436-technote-firepower-00.html

Rate if this answer helps you.

Regards

Jetsy

[gaboughanem]

Thank you for your response.

I have read about the rule profiling and it states that:

1-  it shall increase the cpu utilization on the firepower and I am already reaching very high CPU peak.

2- it will restart the snort engines and it would cause some traffic disruption.

I will see what to do here.

any other ideas?

one more question, I am not able to find the answer about it, which is that the sensor has 16 cores, where Core0 and Core8 has very low, almost zero utilization, while other Cores are screaming.... :)???

Regards,

George

[Jetsy Mathew]

Hello George,

SInce you collected all data required, submit a TAC request and let them analyze it.

You can verify the core files from /var/common/ folder under the sensor.

We require the core file to verify for which process the core has generated.

FYI, whenever the snort resatrts , there will be a traffic interruption and its expected. We have to verify if there is any specific reason why snort restarts and core generates.

Without looking at the core files I cannot say what is the reason.

Rate if this answer helps you

Regards

Jetsy 

[Ed Padilla Jr]

the cpu cores do load balancing base on the type of traffic they need to analyse.   Usually the first core 0 is the supervisor, and instructs other what to execute.  That's were rule profiling is helpful.   At the end of rule profiling, the output file gets file in the /var/common folder.   The optimal time to run rule profiling is during peak time traffic.  Output file uploaded in a excel spreadsheet.  It is columnar type.   Your will see the SIG with the highest OverHead or taxing, in terms of time is taking to expect the packet , etc, etc. 

Like some one said, send the results to Cisco TAC and have a webex session for a good explanation.