topic Re: ASA High CPU Utilization in Network Security

ASA High CPU Utilization

HHeydarov — Fri, 21 Feb 2020 15:45:59 GMT

I have trouble with ASA 5520. Dispatch unit consumes 99% of CPU.

When I shut down all interface other than inside, CPU turns normal. When I turn on outside and others, same thing occur. What is the reason? I need expert advice.

Re: ASA High CPU Utilization

Bogdan Nita — Mon, 14 May 2018 14:25:27 GMT

Having only one interface up on the ASA, basically means no traffic will be forwarded so that does not help with finding out the cause.
Does the CPU rise when bringing up one of the other interfaces?

Further steps for troubleshooting:
show processes cpu-usage sorted non-zero - identify the process taking up the most of the CPU
show interface - check for input or output errors
show traffic - check interfaces with unusual high traffic

HTH

Bogdan

Re: ASA High CPU Utilization

Dennis Mink — Tue, 15 May 2018 03:32:55 GMT

I have seen cases where SYN attacks / flooding causes this. check you logs to see if you pickup any flooding/syn attack events, just a thought

Re: ASA High CPU Utilization

HHeydarov — Tue, 15 May 2018 05:15:21 GMT

show processes cpu-usage sorted non-zero

0x08283a1a   0x6d5d2e4c    97.1%    96.8%    94.0%   Dispatch Unit
0x090f9f8c   0x6d5c9aec     0.2%     0.2%     0.2%   Logger
0x08f567cd   0x6d5bce3c     0.2%     0.2%     0.2%   IP SLA Mon Event Processor
0x08c751b0   0x6d5b2458     0.1%     0.1%     0.1%   Unicorn Admin Handler

show interfaces

Interface GigabitEthernet0/0.30 "INSIDE", is up, line protocol is up
Hardware is i82546GB rev03, BW 1000 Mbps, DLY 10 usec
        VLAN identifier 30
        MAC address 70ca.9b85.06ce, MTU 1500
        IP address X.X.X.X, subnet mask 255.255.255.0
Traffic Statistics for "INSIDE":
        519451485 packets input, 31134611586 bytes
        553307749 packets output, 92540681141 bytes
        496212188 packets dropped
Interface GigabitEthernet0/0.80 "OUTSIDE", is up, line protocol is up
Hardware is i82546GB rev03, BW 1000 Mbps, DLY 10 usec
        VLAN identifier 80
        MAC address 70ca.9b85.06ce, MTU 1500
        IP address X.X.X.X, subnet mask 255.255.255.240
Traffic Statistics for "OUTSIDE":
        10447437 packets input, 7170429891 bytes
        9174785 packets output, 4363070281 bytes
        173728 packets dropped

This is a result of commands.

Re: ASA High CPU Utilization

HHeydarov — Tue, 15 May 2018 05:16:48 GMT

I use Manageengine Firewall Analyzer and I can`t see any attack log.

Re: ASA High CPU Utilization

Bogdan Nita — Tue, 15 May 2018 08:56:14 GMT

There are a lot of packets being dropped by the ASA, you could use the show asp drop to further investigate why are the packets being dropped.

It would also be a good idea to enable unicast RPF on all interface.

Re: ASA High CPU Utilization

HHeydarov — Tue, 15 May 2018 09:17:43 GMT

I enabled it, but did not give a result.

What is your recommendation after enabling it?

Re: ASA High CPU Utilization

Bogdan Nita — Tue, 15 May 2018 09:41:12 GMT

Do a clear asp drop, wait a couple of minutes and do show asp drop a couple of times, post the output.

Re: ASA High CPU Utilization

HHeydarov — Tue, 15 May 2018 10:35:58 GMT

Frame drop:
NAT-T keepalive message (natt-keepalive) 2
No valid adjacency (no-adjacency) 939
Flow is denied by configured rule (acl-drop) 2802544
Flow denied due to resource limitation (unable-to-create-flow) 12
First TCP packet not SYN (tcp-not-syn) 84
TCP failed 3 way handshake (tcp-3whs-failed) 40
TCP RST/FIN out of order (tcp-rstfin-ooo) 85
TCP RST/SYN in window (tcp-rst-syn-in-win) 2
TCP packet failed PAWS test (tcp-paws-fail) 1
Slowpath security checks failed (sp-security-failed) 3467
DNS Inspect id not matched (inspect-dns-id-not-matched) 2
FP L2 rule drop (l2_acl) 6
Interface is down (interface-down) 1164
Connection to PAT address without pre-existing xlate (nat-no-xlate-to-pat-pool) 139

Last clearing: 14:34:24 GMT May 15 2018 by enable_15

Re: ASA High CPU Utilization

HHeydarov — Tue, 15 May 2018 10:46:41 GMT

Frame drop:
NAT-T keepalive message (natt-keepalive) 23
No valid adjacency (no-adjacency) 3987
Flow is denied by configured rule (acl-drop) 12419988
Flow denied due to resource limitation (unable-to-create-flow) 14
First TCP packet not SYN (tcp-not-syn) 538
Bad TCP flags (bad-tcp-flags) 1
TCP failed 3 way handshake (tcp-3whs-failed) 557
TCP RST/FIN out of order (tcp-rstfin-ooo) 719
TCP RST/SYN in window (tcp-rst-syn-in-win) 4
TCP packet failed PAWS test (tcp-paws-fail) 6
Slowpath security checks failed (sp-security-failed) 17943
DNS Inspect id not matched (inspect-dns-id-not-matched) 6
FP L2 rule drop (l2_acl) 28
Interface is down (interface-down) 5291
Dropped pending packets in a closed socket (np-socket-closed) 20
Connection to PAT address without pre-existing xlate (nat-no-xlate-to-pat-pool)

Re: ASA High CPU Utilization

Bogdan Nita — Tue, 15 May 2018 10:52:37 GMT

It looks like a lot of packets are dropped by the acls, especially for a inside interface.
Do you have a deny any log statement at the end of the acl ?
If not configure it and monitor the logs.

Re: ASA High CPU Utilization

HHeydarov — Tue, 15 May 2018 10:58:22 GMT

ACLs Logging is set by default when you create it.

Re: ASA High CPU Utilization

HHeydarov — Tue, 15 May 2018 12:35:31 GMT

Still 90% CPU loaded.

Re: ASA High CPU Utilization

Florin Barhala — Tue, 15 May 2018 13:46:45 GMT

What if the last rule doesn't use log so logging toll goes away from the firewall?