Dispatch Unit - High Cpu Usage

Gurjit Singh Sra · ‎08-09-2014

Hi,

ASA5510 8.2.5(50)

The Dispatch unit process is contantly having high cpu usage for last 10 hours.

Things checked:

1. show proc cpu-usage

2. show perf

It seems fine . Output attached

3. Show interfaces for error

No error, overruns, underrruns on interfaces

4. show traffic

Total cumulative through put on approx 4 Mbps.

drop rate max is 3 pkts /sec randomly and rare occurance on some interfaces

5. Connections and Xlate seem normal.

approx 1100.

Counters were reset 1 hour before the data was collected.

-----------------------------------------------------------------

# sh cpu usage
CPU utilization for 5 seconds = 39%; 1 minute: 38%; 5 minutes: 44%

----------------------------------------------------------------

# sh processes cpu-usage sorted
PC Thread 5Sec 1Min 5Min Process
081aadc4 a79aff7c 35.7% 37.5% 42.5% Dispatch Unit
0853f89e a79a0b68 0.4% 0.2% 0.2% ARP Thread

----------------------------------------------------------------

# show perfmon

PERFMON STATS: Current Average
Xlates 0/s 0/s
Connections 21/s 32/s
TCP Conns 17/s 28/s
UDP Conns 1/s 1/s
URL Access 0/s 0/s
URL Server Req 0/s 0/s
TCP Fixup 0/s 0/s
TCP Intercept Established Conns 0/s 0/s
TCP Intercept Attempts 0/s 0/s

---------------------------------------------------------

sh interface e0/0 | inc overrun

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort

fw01/act# sh interface e0/1 | inc overrun

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort

-----------------------------------------------------------------------

# sh conn all
1135 in use, 8777 most used

--------------------------------------------------------------------------

# sh xlate count
112 in use, 265 most used

------------------------------------------------------------------------

# show asp drop frame
No route to host (no-route) 870
Flow is denied by configured rule (acl-drop) 103915
First TCP packet not SYN (tcp-not-syn) 1317
Bad TCP checksum (bad-tcp-cksum) 2
TCP failed 3 way handshake (tcp-3whs-failed) 6695
TCP RST/FIN out of order (tcp-rstfin-ooo) 4025
TCP packet SEQ past window (tcp-seq-past-win) 13
TCP Out-of-Order packet buffer full (tcp-buffer-full) 1949
TCP Out-of-Order packet buffer timeout (tcp-buffer-timeout) 600
TCP RST/SYN in window (tcp-rst-syn-in-win) 5
TCP dup of packet in Out-of-Order queue (tcp-dup-in-queue) 617
TCP packet failed PAWS test (tcp-paws-fail) 1248
IPSEC tunnel is down (ipsec-tun-down) 2
Slowpath security checks failed (sp-security-failed) 1699
DNS Inspect id not matched (inspect-dns-id-not-matched) 4
FP L2 rule drop (l2_acl) 15436
Dropped pending packets in a closed socket (np-socket-closed) 2

---------------------------------------------------------------------------------

Please let us know what reason can be there for high cpu usage by Dispatch unit under current statistics?

What else should be checked to ensure cpu usage comes down?

Regards,

Gurjit Singh

Network Engineer

Spooster IT Services.

nkarthikeyan · ‎08-13-2014

Hi Gurjar,

r u getting the below mentioned syslog messages?

Flow is denied by configured rule (acl-drop) 103915

106023, 106100, 106004

TCP Out-of-Order packet buffer full (tcp-buffer-full) 1949

TCP Out-of-Order packet buffer full:

    This counter is incremented and the packet is dropped when appliance receives an 
out-of-order TCP packet on a connection and there is no buffer space to store this packet. 
Typically TCP packets are put into order on connections that are inspected by the 
appliance or when packets are sent to SSM for inspection. There is a default queue size 
and when packets in excess of this default queue size are received they will be dropped.

Recommendations:

    On ASA platforms the queue size could be increased using queue-limit configuration 
under tcp-map.

Similarly you need to check many reasons for the asp drop logs that you have captured and you need to monitor how much it is increasing and the difference.......

but 40 % CPU utilization is a okay kind of thing and you do not need to worry if that happens only during peak hours ans it is not increasing drastically more and more.

http://www.cisco.com/c/en/us/td/docs/security/asa/asa82/command/reference/cmd_ref/s2.html#wp1435096

Regards

Karthik