topic Re: Firepower CPU History in Network Security

Firepower Cpu High

hafiez_abn — Sun, 10 Mar 2019 13:44:51 GMT

Hi all,

How to troubleshoot this error ?

One of the CPU goes high percentage and triggered critical alarm.

Screenshot taken from firesight dashboard.

A coupe of questions here:

nspasov — Fri, 06 Jan 2017 18:30:54 GMT

A coupe of questions here:

- What device is having the high CPU? (ASA, Sourcefire appliance, etc)

- What version and patch level are you running

Also, a couple of notes:

- You can login to the device and enter "expert" mode and issue "top" command which will show you what processes are using most of your CPU.

- You should note that only one of the CPU cores is being highly utilized. The current implementation of Snort in Sourcefire/FirePOWER is single threaded. Thus, it is possible that this happens during a CPU intensive process (updates, policy push, elephant flow, etc)

Thank you for rating helpful posts!

hi,

hafiez_abn — Fri, 06 Jan 2017 23:26:59 GMT

hi,

- device is ASA firepower module and running v5.4.0.2-33

- you can show the result as attached,

- is it normal for one of the CPU being high?

Your firepower module is

Oliver Kaiser — Sat, 07 Jan 2017 19:36:40 GMT

Your firepower module is running multiple processes of snort (ips engine). Depending on your traffic load one possible cause could be an elephant flow that is processed by a certain snort process that uses that specific core. (traffic is being load balanced based on 5-tuple (ip,src-port,dst-port,protocol).

Normally high cpu load on a single core is not an issue but it might be related to a bug. In any case you could connect to your firepower module and restart the ips engine during a maintenance windows (possible traffic loss for a few seconds) to see if that fixes your issue.

If the problem persists you might wanna open up a tac case to verify what is causing the high load.

To verify and possibly restart snort do the following on your firepower module via ssh

# change to bash shell
> expert 

# change user to root
admin@firepower:/# sudo su -

# execute top to verify which process is causing high cpu load (snort=ips, exit via ^C)
root@firepower:/# top 

# restart snort engine (might cause temporary traffic loss for a few seconds)
root@firepower:/# pmtool RestartById snort

Done restarts snort service,

hafiez_abn — Mon, 09 Jan 2017 08:39:54 GMT

Done restarts snort service, unfortunately the problem still persist. All snort shown normal percentage, refer to the attached picture.

Would open a TAC case for further assistance.

Just curiosity, where can I get materials to learn about firepower troubleshooting? It is hard to find Linux root's command for firepower from the Cisco website. Besides that, any recommendation book for analyst traffic from Firesight dashboard.

Thanks for sharing..

Unfortunately there isnt much

Oliver Kaiser — Mon, 09 Jan 2017 23:08:28 GMT

Unfortunately there isnt much documentation on the tools available to troubleshoot various firepower issues on root shell. Those tools shouldn't actually be touched according to Cisco since many issues should not occur.

As a starting point you might wanna look at the TAC documentation for firepower and FMC. The configuration guide also lists some commands but if you want to know what the various executables are doing you will need to research them yourself or check various blogs.

Just make sure you test on a lab system before you use your knowledge on a production system.

Hi,

hafiez_abn — Thu, 26 Jan 2017 07:00:26 GMT

Hi,

I'm already open TAC case for this issue.. unfortunately still pending for developer..actually CPU high doesn't comes from snort.

Do you have any idea?

If the issue had to be

Oliver Kaiser — Thu, 26 Jan 2017 09:22:31 GMT

If the issue had to be escalated to engineering, I think we wont find a solution to this ourselves. According to your screenshot it seems to be related to some stats collection process.

I have found this rna related bug CSCuv99982, but I am not sure that is really the issue here since your problem is about high cpu usage. TBH I would try to upgrade to 6.1.0.1, but lets just wait for development, they should know why exactly the process is causing high load.

regards

Oliver

Hi, this issue has been

hafiez_abn — Tue, 31 Jan 2017 10:44:24 GMT

Hi, this issue has been solved by following below workaround.

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCuv11738/?reffering_site=dumpcr

Glad that your issue was

nspasov — Tue, 31 Jan 2017 23:05:03 GMT

Glad that your issue was resolved! Also, thank you for taking the time to come back and post the solution!

Neno

Re: Your firepower module is

jaime.pedraza — Mon, 18 Sep 2017 16:23:23 GMT

Hi, do you know if there is a log where the history of the cpu consumption is stored, like the top output but in a time window (days, weeks)?

Firepower CPU History

Marvin Rhoads — Tue, 19 Sep 2017 03:23:32 GMT

There's no such log as far as I know. You could query the value via SNMP and save it off on your management system.

Re: Firepower CPU History

MSJ1 — Wed, 09 Jun 2021 13:46:18 GMT

While I see below for a FTD 2130 , I had 95% CPU. Is there any relation you see 95% CPU with all below 12 DataPath Parentages ?

> show processes cpu-usage sorted non-zero
Hardware: FPR-2130
Cisco Adaptive Security Appliance Software Version 9.12(2)115
ASLR enabled, text region aab3ca9000-aab84a6cc4
PC Thread 5Sec 1Min 5Min Process
- - 7.3% 7.3% 7.6% DATAPATH-3-1737
- - 7.3% 7.3% 7.6% DATAPATH-6-1740
- - 7.3% 7.3% 7.6% DATAPATH-7-1741
- - 7.2% 7.3% 7.6% DATAPATH-10-1744
- - 7.2% 7.3% 7.6% DATAPATH-2-1736
- - 7.2% 7.3% 7.6% DATAPATH-9-1743
- - 7.2% 7.3% 7.6% DATAPATH-1-1735
- - 7.2% 7.3% 7.6% DATAPATH-5-1739
- - 7.1% 7.3% 7.6% DATAPATH-11-1745
- - 7.1% 7.3% 7.6% DATAPATH-0-1734
- - 7.1% 7.3% 7.6% DATAPATH-4-1738
- - 7.1% 7.3% 7.6% DATAPATH-8-1742
0x000000aab5603148 0x000000fff42012a0 0.1% 0.1% 0.1% ARP Thread
0x000000aab65dc7bc 0x000000fff420d840 0.1% 0.0% 0.0% CERT API

Re: Firepower CPU History

Marvin Rhoads — Fri, 11 Jun 2021 16:59:48 GMT

CPU utilization of Firepower is reported out separately for FXOS vs. FTD vs. LINA. So you need to always distinguish which context you are looking at.

Can you give more info on where you see the 95%?

Re: Firepower CPU History

Uwe Siegrist — Wed, 12 Jan 2022 09:16:53 GMT

@Marvin Rhoads wrote:
...
Can you give more info on where you see the 95%?

I think he saw the high usage (95%) in the "show cpu usage detailed" output because I have exactly the same.