High CPU utilization on Amp on Linux

[avidavidowitz]

Hi. We see high CPU utilization on all of our Linux systems (Mix of RH 6 and CentOS 6 ). Have tried upgrading to the latest connector (1.6.0.536)  - but that did not solve the problem. Have tried installing on a few systems - same result. We are afraid to deploy on heavily utilized system if its going to eat all of the CPU... Does it always take all non used CPU  at a given time?  We see it using up to 96% sometimes. Any information would be appreciated. Thanks

[nspasov]

A couple of questions:

1. Is your policy utilizing "exclusions?"If yes, are you using a custom one or the Cisco Recommended?

2. Do you have another A/V, EPP running on those hosts?

Thank you for rating helpful posts!

[avidavidowitz]

---

@nspasov wrote:

A couple of questions:

1. Is your policy utilizing "exclusions?"If yes, are you using a custom one or the Cisco Recommended?

2. Do you have another A/V, EPP running on those hosts?

Thank you for rating helpful posts!

---

No exclusions - also, no other A/V or EPP running

[nspasov]

I have the feeling the high CPU is due to the fact that you are not using exclusions. You can try this:

1. Copy your existing Linux policy

2. Attach the Default Exclusion set for Linux workstations

3. Create a test group

4. Attache the newly created policy

5. Attach one of the workstations that is experiencing the high CPU utilization

6. Test and see if this fixed the problem

Thank you for rating helpful posts!

[avidavidowitz]

---

@nspasov wrote:

I have the feeling the high CPU is due to the fact that you are not using exclusions. You can try this:

1. Copy your existing Linux policy

2. Attach the Default Exclusion set for Linux workstations

3. Create a test group

4. Attache the newly created policy

5. Attach one of the workstations that is experiencing the high CPU utilization

6. Test and see if this fixed the problem

Thank you for rating helpful posts!

---

There are no default recommended exclusions for Linux: https://www.cisco.com/c/en/us/support/docs/security/sourcefire-fireamp-endpoints/118341-configure-fireamp-00.html#anc12

Can you point me somewhere else?

[nspasov]

Shoot you are right! I just checked my console and there is a default exclusion set for Linux workstations but it is blank :( At this point I would suggest reaching out to TAC and have them troubleshoot the issue and perhaps suggest some recommendations around exclusions for Linux based deployments.

Thank you for rating helpful posts!

[David Janulik]

Hi,

for a detailed analysis, please open a ticket with following logs inc.

/var/log/cisco/

I can give few advice's at this moment:

1. get the agent PID ps aux | grep -i amp or stop the daemon with "initctl stop cisco-amp"
2. kill the agent process and see if it has any effect on high cpu. If the CPU gets healthier, we can help you to tune exclusions for most checked file cloud query lookups.
3. Policies - File Mode make sure you do not have "On Execute Mode" set in your policy.

Best Regards

David

[avidavidowitz]

---

@David Janulik wrote:

Hi,

for a detailed analysis, please open a ticket with following logs inc.

/var/log/cisco/

I can give few advice's at this moment:

1. get the agent PID ps aux | grep -i amp or stop the daemon with "initctl stop cisco-amp"
2. kill the agent process and see if it has any effect on high cpu. If the CPU gets healthier, we can help you to tune exclusions for most checked file cloud query lookups.
3. Policies - File Mode make sure you do not have "On Execute Mode" set in your policy.

Best Regards

David

---

Killing the agent drops CPU - it is for sure AMP related. I have no idea about default a default exclusion list - there doesn't seem to be anything listed for default exclusions on line: https://www.cisco.com/c/en/us/support/docs/security/sourcefire-fireamp-endpoints/118341-configure-fireamp-00.html#anc12

Regarding File Mode - it's set to Passive

[avidavidowitz]

So going through the logs we saw hundreds of hits to the following directories in a 15 second period and excluded them causing CPU to drop to 2-3%. I would love to know exactly what the default exclusions should be and if I am excluding anything that really shouldn't be from a a security perspective.....

Path: /bin/bash

Path: /bin/date

Path: /bin/df

Path: /bin/ps

Path: /home/observium/rrd

Path: /opt/cisco/amp

Path: /usr/bin/perl

Path: /usr/bin/php

Wildcard: /proc/*

Wildcard: /usr/bin/snmp*

Wildcard: /usr/local/nagios/*

[hrithiktej]

Hi David,

We have the exclusions set total 46 added i can see them in policy.xml file plus attached herewith is the file and process scan policy can you check and suggest any changes that can bring down the CPU usage on our oracle linux boxes.

On execute mode is set to default Passive

[brmcmaho]

Just looking at the exclusion policy without seeing what's actually happening on the systems is unlikely to reveal very much.  The best way to resolve high CPU issues is generally via a TAC case.  The support engineer can help you collect and analyze usage information to determine the source of the problem.

[hrithiktej]

Thanks i have raised a TAC case will post my findings here.