topic High CPU utilization on Amp on Linux in Endpoint Security

High CPU utilization on Amp on Linux

avidavidowitz — Sat, 09 Mar 2019 01:46:10 GMT

Hi. We see high CPU utilization on all of our Linux systems (Mix of RH 6 and CentOS 6 ). Have tried upgrading to the latest connector (1.6.0.536) - but that did not solve the problem. Have tried installing on a few systems - same result. We are afraid to deploy on heavily utilized system if its going to eat all of the CPU... Does it always take all non used CPU at a given time? We see it using up to 96% sometimes. Any information would be appreciated. Thanks

Re: High CPU utilization on Amp on Linux

nspasov — Thu, 18 Jan 2018 19:32:19 GMT

A couple of questions:

1. Is your policy utilizing "exclusions?"If yes, are you using a custom one or the Cisco Recommended?

2. Do you have another A/V, EPP running on those hosts?

Thank you for rating helpful posts!

Re: High CPU utilization on Amp on Linux

avidavidowitz — Thu, 18 Jan 2018 21:06:41 GMT

@nspasov wrote:

A couple of questions:

1. Is your policy utilizing "exclusions?"If yes, are you using a custom one or the Cisco Recommended?

2. Do you have another A/V, EPP running on those hosts?

Thank you for rating helpful posts!

No exclusions - also, no other A/V or EPP running

Re: High CPU utilization on Amp on Linux

David Janulik — Fri, 19 Jan 2018 11:08:44 GMT

Hi,

for a detailed analysis, please open a ticket with following logs inc.

/var/log/cisco/

I can give few advice's at this moment:

get the agent PID ps aux | grep -i amp or stop the daemon with "initctl stop cisco-amp"
kill the agent process and see if it has any effect on high cpu. If the CPU gets healthier, we can help you to tune exclusions for most checked file cloud query lookups.
Policies - File Mode make sure you do not have "On Execute Mode" set in your policy.

Best Regards

David

Re: High CPU utilization on Amp on Linux

nspasov — Fri, 19 Jan 2018 18:58:39 GMT

I have the feeling the high CPU is due to the fact that you are not using exclusions. You can try this:

1. Copy your existing Linux policy

2. Attach the Default Exclusion set for Linux workstations

3. Create a test group

4. Attache the newly created policy

5. Attach one of the workstations that is experiencing the high CPU utilization

6. Test and see if this fixed the problem

Thank you for rating helpful posts!

Re: High CPU utilization on Amp on Linux

avidavidowitz — Sat, 20 Jan 2018 21:20:17 GMT

@nspasov wrote:

I have the feeling the high CPU is due to the fact that you are not using exclusions. You can try this:

1. Copy your existing Linux policy

2. Attach the Default Exclusion set for Linux workstations

3. Create a test group

4. Attache the newly created policy

5. Attach one of the workstations that is experiencing the high CPU utilization

6. Test and see if this fixed the problem

Thank you for rating helpful posts!

There are no default recommended exclusions for Linux: https://www.cisco.com/c/en/us/support/docs/security/sourcefire-fireamp-endpoints/118341-configure-fireamp-00.html#anc12

Can you point me somewhere else?

Re: High CPU utilization on Amp on Linux

avidavidowitz — Sat, 20 Jan 2018 21:28:48 GMT

@David Janulik wrote:

Hi,

for a detailed analysis, please open a ticket with following logs inc.

/var/log/cisco/

I can give few advice's at this moment:

get the agent PID ps aux | grep -i amp or stop the daemon with "initctl stop cisco-amp"

kill the agent process and see if it has any effect on high cpu. If the CPU gets healthier, we can help you to tune exclusions for most checked file cloud query lookups.

Policies - File Mode make sure you do not have "On Execute Mode" set in your policy.

Best Regards

David

Killing the agent drops CPU - it is for sure AMP related. I have no idea about default a default exclusion list - there doesn't seem to be anything listed for default exclusions on line: https://www.cisco.com/c/en/us/support/docs/security/sourcefire-fireamp-endpoints/118341-configure-fireamp-00.html#anc12

Regarding File Mode - it's set to Passive

Re: High CPU utilization on Amp on Linux

nspasov — Sat, 20 Jan 2018 21:33:21 GMT

Shoot you are right! I just checked my console and there is a default exclusion set for Linux workstations but it is blank 😞 At this point I would suggest reaching out to TAC and have them troubleshoot the issue and perhaps suggest some recommendations around exclusions for Linux based deployments.

Thank you for rating helpful posts!

Re: High CPU utilization on Amp on Linux

avidavidowitz — Sun, 28 Jan 2018 07:44:32 GMT

So going through the logs we saw hundreds of hits to the following directories in a 15 second period and excluded them causing CPU to drop to 2-3%. I would love to know exactly what the default exclusions should be and if I am excluding anything that really shouldn't be from a a security perspective.....

Re: High CPU utilization on Amp on Linux

hrithiktej — Tue, 08 May 2018 10:52:16 GMT

Hi David,

We have the exclusions set total 46 added i can see them in policy.xml file plus attached herewith is the file and process scan policy can you check and suggest any changes that can bring down the CPU usage on our oracle linux boxes.

On execute mode is set to default Passive

Re: High CPU utilization on Amp on Linux

brmcmaho — Tue, 08 May 2018 17:48:07 GMT

Just looking at the exclusion policy without seeing what's actually happening on the systems is unlikely to reveal very much. The best way to resolve high CPU issues is generally via a TAC case. The support engineer can help you collect and analyze usage information to determine the source of the problem.

Re: High CPU utilization on Amp on Linux

hrithiktej — Wed, 09 May 2018 11:33:36 GMT

Thanks i have raised a TAC case will post my findings here.