cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
765
Views
1
Helpful
3
Replies

High acl-drop rate (Firepower 2140)

NetworkNinjaAC
Level 1
Level 1

We have a firepower 2140 as our perimeter firewall and is running ASA software. We are noticing the ASP-DROP counters increase at a very rapid rate. I did a "clear traffic" wait 1 minute "show traffic" and we saw we are dropping 12600 packets/s on the outside interface for 1 minute average. Our CPU utilization currently is staying at over 95% and I believe the high packet drop rate is the reason for that. I did a small asp-drop packet capture on the outside and saw a lot of acl-drops for traffic originating from other European countries inbound to our internal servers. Not sure what steps to take in this situation since this traffic is already getting blocked by the firewall. What solutions could we implement to prevent this traffic from hitting our perimeter firewall in the first place?

3 Replies 3

tvotna
Spotlight
Spotlight

You can't do anything on the firewall itself to decrease CPU utilization if it is caused by acl-drop'ed packets and service-policies won't help too (they will make things worse). The only feature that can help a little is "shun <ip>" (to block certain senders), because it lives earlier in the packet processing path than inbound ACL. Also, don't enable "service resetinbound" if you don't really need to (although it contributes a little). The only real solution is a powerful anti-DDoS product in front of Cisco firewall. And yes, it is well-known that dropped packets can cause higher CPU utilization than passed packets. The reason of this behavior is unknown.

On the other hand, "show traffic" and "show interface" count all dropped packets in accelerated security path. I'd analyze each drop counter separately by running "show asp drop" few times and calculating acl-drop rate over time manually. Also, it's hard to say if 12600 cps drop rate is critical for 2140. I would say no, it is not. It is possible that there is something else, e.g. high cps rate or high failover replication rate or combination of factors with high traffic rate. Check:

show cpu detailed
show resource usage
show asp event dp-cp
show perfmon detail
clear traffic + show traffic

 

Below are outputs from one of our production firewalls. This is FP2130, which has less CPU cores and has huge 4MB ACL with object-group-search access-control enabled, which can increase CPU load. Drop rate is 500 cps or so, but CPU is still high for this traffic rate.

# show cpu det

Break down of per-core data path versus control point cpu usage:
Core 5 sec 1 min 5 min
Core 0 39.3 (38.9 + 0.4) 44.8 (44.7 + 0.1) 45.8 (45.6 + 0.0)
Core 1 39.5 (39.3 + 0.2) 44.9 (44.8 + 0.2) 45.7 (45.6 + 0.1)
Core 2 40.3 (40.3 + 0.0) 44.5 (44.4 + 0.2) 45.6 (45.4 + 0.1)
Core 3 39.9 (39.9 + 0.0) 45.3 (45.1 + 0.1) 45.8 (45.7 + 0.0)
Core 4 41.3 (41.1 + 0.2) 44.7 (44.6 + 0.1) 45.6 (45.5 + 0.0)
Core 5 40.1 (39.9 + 0.2) 44.7 (44.6 + 0.1) 45.6 (45.5 + 0.1)
Core 6 39.9 (39.9 + 0.0) 44.4 (44.3 + 0.1) 45.6 (45.5 + 0.1)
Core 7 39.5 (39.5 + 0.0) 44.9 (44.8 + 0.1) 45.7 (45.6 + 0.0)
Core 8 41.3 (41.3 + 0.0) 45.1 (45.0 + 0.1) 45.9 (45.7 + 0.2)
Core 9 40.5 (40.5 + 0.0) 45.6 (45.5 + 0.1) 46.1 (45.9 + 0.0)
Core 10 40.9 (40.9 + 0.0) 44.7 (44.6 + 0.1) 45.5 (45.4 + 0.2)
Core 11 39.7 (39.7 + 0.0) 44.7 (44.6 + 0.1) 45.7 (45.6 + 0.1)

Current control point elapsed versus the maximum control point elapsed for:
5 seconds = 37.5%; 1 minute: 28.9%; 5 minutes: 28.2%


CPU utilization of external processes for:
5 seconds = 0.3%; 1 minute: 0.2%; 5 minutes: 0.1%


Total CPU utilization for:
5 seconds = 40.8%; 1 minute: 45.3%; 5 minutes: 46.1%

Internal-Data0/1:
received (in 608.930 secs):
72686777 packets 27201716712 bytes
119368 pkts/sec 44671336 bytes/sec
transmitted (in 608.930 secs):
74119399 packets 29017596869 bytes
121720 pkts/sec 47653419 bytes/sec
1 minute input rate 4 pkts/sec, 1133 bytes/sec
1 minute output rate 4 pkts/sec, 1082 bytes/sec
1 minute drop rate, 0 pkts/sec
5 minute input rate 0 pkts/sec, 201 bytes/sec
5 minute output rate 0 pkts/sec, 211 bytes/sec
5 minute drop rate, 0 pkts/sec

Resource Current Peak Limit Denied Context
SSH Server 2 6 5 0 admin
Syslogs [rate] 0 279 unlimited 0 admin
Conns 26 314 unlimited 0 admin
Hosts 5 11 unlimited 0 admin
Inspects [rate] 0 81 unlimited 0 admin
Routes 9 9 unlimited 0 admin
Syslogs [rate] 0 1594 unlimited 0 CONTEXT
Conns 4545 84797 unlimited 0 CONTEXT
Xlates 38 42 unlimited 0 CONTEXT
Hosts 693 1681 unlimited 0 CONTEXT
Conns [rate] 2269 13506 unlimited 0 CONTEXT
Inspects [rate] 311 9781 unlimited 0 CONTEXT
Routes 66 66 unlimited 0 CONTEXT

# show perfmon det

Context: CONTEXT
PERFMON STATS: Current Average
Xlates 0/s 0/s
Connections 2127/s 31255/s
TCP Conns 1805/s 28005/s
UDP Conns 41/s 519/s
URL Access 0/s 0/s
URL Server Req 0/s 0/s
TCP Fixup 0/s 0/s
TCP Intercept Established Conns 0/s 0/s
TCP Intercept Attempts 0/s 0/s
TCP Embryonic Conns Timeout 6/s 303/s
FTP Fixup 0/s 0/s
AAA Authen 0/s 0/s
AAA Author 0/s 0/s
AAA Account 0/s 0/s
HTTP Fixup 0/s 0/s

VALID CONNS RATE in TCP INTERCEPT: Current Average
N/A N/A

SETUP RATES:
Connections for 1 minute = 2060/s; 5 minutes = 2158/s
TCP Conns for 1 minute = 1728/s; 5 minutes = 1836/s
UDP Conns for 1 minute = 47/s; 5 minutes = 41/s

 

Review Cisco Networking for a $25 gift card