Firepower Cpu High

[hafiez_abn]

Hi all, 

How to troubleshoot this error ? 

One of the CPU goes high percentage and triggered critical alarm. 

Screenshot taken from firesight dashboard.

[nspasov]

A coupe of questions here:

- What device is having the high CPU? (ASA, Sourcefire appliance, etc)

- What version and patch level are you running

Also, a couple of notes:

- You can login to the device and enter "expert" mode and issue "top" command which will show you what processes are using most of your CPU. 

- You should note that only one of the CPU cores is being highly utilized. The current implementation of Snort in Sourcefire/FirePOWER is single threaded. Thus, it is possible that this happens during a CPU intensive process (updates, policy push, elephant flow, etc)

Thank you for rating helpful posts!

[hafiez_abn]

hi,

- device is ASA firepower module and running v5.4.0.2-33

- you can show the result as attached, 

- is it normal for one of the CPU being high? 

[Oliver Kaiser]

Your firepower module is running multiple processes of snort (ips engine). Depending on your traffic load one possible cause could be an elephant flow that is processed by a certain snort process that uses that specific core. (traffic is being load balanced based on 5-tuple (ip,src-port,dst-port,protocol).

Normally high cpu load on a single core is not an issue but it might be related to a bug. In any case you could connect to your firepower module and restart the ips engine during a maintenance windows (possible traffic loss for a few seconds) to see if that fixes your issue.

If the problem persists you might wanna open up a tac case to verify what is causing the high load.

To verify and possibly restart snort do the following on your firepower module via ssh

# change to bash shell
> expert 

# change user to root
admin@firepower:/# sudo su -

# execute top to verify which process is causing high cpu load (snort=ips, exit via ^C)
root@firepower:/# top 

# restart snort engine (might cause temporary traffic loss for a few seconds)
root@firepower:/# pmtool RestartById snort

[hafiez_abn]

Done restarts snort service, unfortunately the problem still persist. All snort shown normal percentage, refer to the attached picture.

Would open a TAC case for further assistance.

Just curiosity, where can I get materials to learn about firepower troubleshooting? It is hard to find Linux root's command for firepower from the Cisco website. Besides that, any recommendation book for analyst traffic from Firesight dashboard.

Thanks for sharing..

[Oliver Kaiser]

Unfortunately there isnt much documentation on the tools available to troubleshoot various firepower issues on root shell. Those tools shouldn't actually be touched according to Cisco since many issues should not occur.

As a starting point you might wanna look at the TAC documentation for firepower and FMC. The configuration guide also lists some commands but if you want to know what the various executables are doing you will need to research them yourself or check various blogs.

Just make sure you test on a lab system before you use your knowledge on a production system.

[hafiez_abn]

Hi, 

I'm already open TAC case for this issue.. unfortunately still pending for developer..actually CPU high doesn't comes from snort.

Do you have any idea?  

[Oliver Kaiser]

If the issue had to be escalated to engineering, I think we wont find a solution to this ourselves. According to your screenshot it seems to be related to some stats collection process.

I have found this rna related bug CSCuv99982, but I am not sure that is really the issue here since your problem is about high cpu usage. TBH I would try to upgrade to 6.1.0.1, but lets just wait for development, they should know why exactly the process is causing high load.

regards

Oliver

[hafiez_abn · Accepted solution]

Hi, this issue has been solved by following below workaround.

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCuv11738/?reffering_site=dumpcr

[nspasov]

Glad that your issue was resolved! Also, thank you for taking the time to come back and post the solution! 

Neno

[jaime.pedraza]

Hi, do you know if there is a log where the history of the cpu consumption is stored, like the top output but in a time window (days, weeks)?

[Marvin Rhoads]

There's no such log as far as I know. You could query the value via SNMP and save it off on your management system.

[MSJ1]

While I see below for a FTD 2130 , I had 95% CPU. Is there any relation you see 95% CPU with all below 12 DataPath Parentages  ?

> show processes cpu-usage sorted non-zero
Hardware: FPR-2130
Cisco Adaptive Security Appliance Software Version 9.12(2)115
ASLR enabled, text region aab3ca9000-aab84a6cc4
PC Thread 5Sec 1Min 5Min Process
- - 7.3% 7.3% 7.6% DATAPATH-3-1737
- - 7.3% 7.3% 7.6% DATAPATH-6-1740
- - 7.3% 7.3% 7.6% DATAPATH-7-1741
- - 7.2% 7.3% 7.6% DATAPATH-10-1744
- - 7.2% 7.3% 7.6% DATAPATH-2-1736
- - 7.2% 7.3% 7.6% DATAPATH-9-1743
- - 7.2% 7.3% 7.6% DATAPATH-1-1735
- - 7.2% 7.3% 7.6% DATAPATH-5-1739
- - 7.1% 7.3% 7.6% DATAPATH-11-1745
- - 7.1% 7.3% 7.6% DATAPATH-0-1734
- - 7.1% 7.3% 7.6% DATAPATH-4-1738
- - 7.1% 7.3% 7.6% DATAPATH-8-1742
0x000000aab5603148 0x000000fff42012a0 0.1% 0.1% 0.1% ARP Thread
0x000000aab65dc7bc 0x000000fff420d840 0.1% 0.0% 0.0% CERT API

[Marvin Rhoads]

CPU utilization of Firepower is reported out separately for FXOS vs. FTD vs. LINA. So you need to always distinguish which context you are looking at.

Can you give more info on where you see the 95%?

[Uwe Siegrist]

---

@Marvin Rhoads wrote:

...

Can you give more info on where you see the 95%?

---

I think he saw the high usage (95%) in the "show cpu usage detailed" output because I have exactly the same.