Firewall CPU Usage

[loc.nguyen]

Hi,

I have an issue with Cisco Firepower Threat Defense for Azure. It often alert severity critical for CPU Usage.

Randomly check shows CPU around 50%. Could you advise where I should check further?

Below is detail: 

FP-East# sh cpu detail

Break down of per-core data path versus control point cpu usage:
Core 5 sec 1 min 5 min
Core 0 45.6 (45.6 + 0.0) 49.0 (48.9 + 0.0) 45.0 (45.0 + 0.0)

Current control point elapsed versus the data and control point elapsed for:
5 seconds = 3.0%; 1 minute: 3.0%; 5 minutes: 2.9%

CPU utilization of external processes for:
5 seconds = 0.0%; 1 minute: 0.1%; 5 minutes: 0.0%

Total CPU utilization for:
5 seconds = 45.8%; 1 minute: 49.2%; 5 minutes: 45.3%

FP-East#

East# show version
--------------------[ FP-East ]---------------------
Model : Cisco Firepower Threat Defense for Azure (75) Version 6.6.5.1 (Build 15)
UUID : xxxxx
Rules update version : 2022-10-10-001-vrt
VDB version : 359
----------------------------------------------------

Cisco Adaptive Security Appliance Software Version 9.14(3)15
SSP Operating System Version 2.8(1.165)

Compiled on Tue 09-Nov-21 17:50 GMT by builders
System image file is "boot:/asa9143-6-smp-k8.bin"
Config file at boot was "startup-config"

FP-East up 2 days 11 hours

Hardware: NGFWv, 14336 MB RAM, CPU Xeon E5 series 2400 MHz, 1 CPU (4 cores)
Internal ATA Compact Flash, 65536MB
Slot 1: ATA Compact Flash, 65536MB
BIOS Flash Firmware Hub @ 0x0, 0KB

0: Int: Internal-Data0/0 : address is 000d.3a11.49f8, irq 0
1: Ext: GigabitEthernet0/0 : address is 000d.3a11.4146, irq 0
2: Ext: GigabitEthernet0/1 : address is 000d.3a11.4e77, irq 0
3: Int: Internal-Control0/0 : address is 0000.0001.0001, irq 0
4: Int: Internal-Data0/0 : address is 0000.0000.0000, irq 0
5: Ext: Management0/0 : address is 000d.3a11.49f8, irq 0
6: Int: Internal-Data0/1 : address is 0000.0100.0001, irq 0
7: Int: Internal-Data0/2 : address is 0000.0000.0000, irq 0
8: Int: Internal-Control0/1 : address is 0000.0001.0001, irq 0

Serial Number: xxxxx

Image type : Release
Key version : A

Configuration last modified by enable_1 at 05:10:28.683 UTC Wed Oct 12 2022
East#

Thanks

Loc

[balaji.bandi]

Total CPU utilization for:
5 seconds = 45.8%; 1 minute: 49.2%; 5 minutes: 45.3%

every 5min you may be getting the alerts. Maybe you can increase this level to 70% to see if that suppresses alarms?

[loc.nguyen]

Do we have a command to check which ones are using most of the CPU?

[loc.nguyen]

East# show processes cpu-usage sorted non-zero
Hardware: NGFWv
Cisco Adaptive Security Appliance Software Version 9.14(3)15
ASLR enabled, text region 561a4a4ff000-56xxx
PC Thread 5Sec 1Min 5Min Process
- - 75.7% 78.1% 75.2% DATAPATH-0-3700
East#

[balaji.bandi]

post below information ;

show cpu usage

show processes cpu-usage sorted non-zero 

[loc.nguyen]

Yeah, I did. Pls see the above.

[balaji.bandi]

show cpu usage

[loc.nguyen]

FP-East# show cpu usage
CPU utilization for 5 seconds = 34%; 1 minute: 35%; 5 minutes: 42%

FP-East#

[MHM Cisco World]

asa# show asp drop

please share the output 

[loc.nguyen]

FP-East# show asp drop

Frame drop:
NAT-T keepalive message (natt-keepalive) 203335
IPSEC tunnel is down (ipsec-tun-down) 130
SVC Module does not have a channel for reinjection (mp-svc-no-channel) 530
SVC Module does not have a session (mp-svc-no-session) 381
SVC Module is in flow control (mp-svc-flow-control) 126555
SVC Module unable to fragment packet (mp-svc-no-fragment) 151
Flow is being freed (flow-being-freed) 5646
No route to host (no-route) 12734
Flow is denied by configured rule (acl-drop) 3983837
Invalid SPI (np-sp-invalid-spi) 99
First TCP packet not SYN (tcp-not-syn) 834711
TCP failed 3 way handshake (tcp-3whs-failed) 4446
TCP RST/FIN out of order (tcp-rstfin-ooo) 5748
TCP SEQ in SYN/SYNACK invalid (tcp-seq-syn-diff) 36
TCP packet SEQ past window (tcp-seq-past-win) 27
TCP invalid ACK (tcp-invalid-ack) 36
TCP RST/SYN in window (tcp-rst-syn-in-win) 606
TCP packet failed PAWS test (tcp-paws-fail) 10
CTM returned error (ctm-error) 661
ICMP Inspect seq num not matched (inspect-icmp-seq-num-not-matched) 3
DNS Inspect id not matched (inspect-dns-id-not-matched) 481
Snort requested to drop the frame (snort-drop) 415547
Snort instance is down (snort-down) 2932
Snort instance is busy (snort-busy) 19142191
FP L2 rule drop (l2_acl) 26
Dropped pending packets in a closed socket (np-socket-closed) 148749
Connection to PAT address without pre-existing xlate (nat-no-xlate-to-pat-pool) 2781
TCP Proxy retransmited packet drop (tcp-proxy-retransmit-drop) 52
Blocked or blacklisted by the firewall preprocessor (firewall) 574198
Blocked or blacklisted by the SI preprocessor (si) 2
Blocked or blacklisted by the session preprocessor (session-preproc) 10
Blocked or blacklisted by the reputation preprocessor (reputation) 426
Blocked or blacklisted by the file process preprocessor (file-process) 2711
Blocked or blacklisted by the IPS preprocessor (ips-preproc) 28
Fragment reassembly failed (fragment-reassembly-failed) 652690
Packet is blacklisted by snort (snort-blacklist) 2967995
Packet is blocked as requested by snort (snort-block) 29223311

Last clearing: Never

Flow drop:
Tunnel being brought up or torn down (tunnel-pending) 6
Need to start IKE negotiation (need-ike) 2
VPN overlap conflict (vpn-overlap-conflict) 57292
VPN decryption missing (vpn-missing-decrypt) 23876
NAT reverse path failed (nat-rpf-failed) 180
Inspection failure (inspect-fail) 11968
SSL bad record detected (ssl-bad-record-detect) 122
SSL handshake failed (ssl-handshake-failed) 2123

Last clearing: Never
FP-East#

[loc.nguyen]

Do you have any ideas why it happens?

if you need more information, pls let me know.

[loc.nguyen]

Health Monitor Alert from fp-east.internal.cloudapp.net

Time: Wed Oct  5 06:04:24 2022 UTC

Severity: critical

Module: CPU Usage

Description: Using CPU03 150.00%

[MHM Cisco World]

You remember the sysopt we add before to preserve the TCP through VPN, 

https://community.cisco.com/t5/switching/asa-drops-sftp-connections/td-p/4698759

sysopt connection preserve-vpn-flows

 ""Enabling this feature does not create any additional overload on the internal CPU processing of the ASA because it is going to keep the same TCP connections that the device has when the tunnel is up.""

https://www.cisco.com/c/en/us/support/docs/security/asa-5500-x-series-next-generation-firewalls/113014-asa-userapp-vpntunnel.html

so I will ask you are you face this issue after add this command ?
if yes then remove it and check CPU level. 

the high CPU utilize of DataPath usually because VPN traffic.