Packet is blocked as requested by snort with Prefilter anyany fastpath

[Tritontek]

Is there any way that Snort can still block or drop a packet/traffic even if i already added a prefilter policy that sets as any any network and with fastpath? Also i have disabled all my access control policy except for default ACP that is set to "Trust All Traffic"

These are the diagnostic data gathered below, random users experienced dropped traffic specifically accesses to cloud servers.

326: 12:06:43.000030 192.168.7.122.54741 > 20.189.173.14.443: P 1630889135:1630889364(229) ack 3677944706 win 1024 Drop-reason: (snort-block) Packet is blocked as requested by snort, Drop-location: frame 0x000055e9e3316112 flow (NA)/NA

327: 12:06:43.257463 192.168.5.100.56591 > 142.251.221.3.443: P 91537800:91538757(957) ack 1105471243 win 512 Drop-reason: (snort-block) Packet is blocked as requested by snort, Drop-location: frame 0x000055e9e3316112 flow (NA)/NA

328: 12:06:43.305953 192.168.9.102.55730 > 13.107.136.10.443: P 1070331683:1070331874(191) ack 3766062448 win 1024 Drop-reason: (snort-block) Packet is blocked as requested by snort, Drop-location: frame 0x000055e9e3316112 flow (NA)/NA

ASP Drop data below:

Are these drop counts normal or expected even after creating a fastpath inf pre filter policy and disabled all ACP rules and just retained the default action which is "TRUST ALL TRAFFIC" 

We are using FTD7.0.6 and FMC7.3.1

[MHM Cisco World]

the fastpath prefilter must allow all traffic pass without inspect via Snort. 

do you clear conn after add prefilter ?

Thanks A Lot
MHM

[Marvin Rhoads]

It is like @MHM Cisco World noted.

A properly formed and applied prefilter policy with "clear conn" having been done afterwards will take Snort out of the path altogether.

[Marius Gunnerud]

The counters have never been cleared, clear the counters and then review the output again to get an accurate picture of what is being dropped.

[Tritontek]

Hi Sir,

i have cleared the counters now and ran some packet capture and from the results i just gather all the data with DROP action. Here are the few of them below:

NOTE: These are the results when i added the pre-filter policy and used the ACP rule with allow all any any. I can see a lot of Phase 5, Access-List and Snort being dropped. Where can i check on these below more further?

Phase: 5
Type: ACCESS-LIST
Subtype:
Result: DROP
Config:
Implicit Rule
Additional Information:

Result:
input-interface: OUTSIDE1_INT(vrfid:0)
input-status: up
input-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule, Drop-location: frame 0x00005623b4b75ccf flow (NA)/NA

18: 07:37:37.427407 94.102.57.152.49370 > 61.245.4.9.443: S 2333418438:2333418438(0) win 29200 <mss 1460,sackOK,timestamp 1622119284 0,nop,wscale 7>

Phase: 5
Type: ACCESS-LIST
Subtype:
Result: DROP
Config:
Implicit Rule
Additional Information:

Result:
input-interface: OUTSIDE1_INT(vrfid:0)
input-status: up
input-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule, Drop-location: frame 0x00005623b4b75ccf flow (NA)/NA

19: 07:37:37.463385 94.102.57.152.49416 > 61.245.4.9.443: S 386166707:386166707(0) win 29200 <mss 1460,sackOK,timestamp 1622119320 0,nop,wscale 7>

Phase: 5
Type: ACCESS-LIST
Subtype:
Result: DROP
Config:
Implicit Rule
Additional Information:

Result:
input-interface: OUTSIDE1_INT(vrfid:0)
input-status: up
input-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule, Drop-location: frame 0x00005623b4b75ccf flow (NA)/NA

20: 07:37:37.567567 94.102.57.152.49562 > 61.245.4.9.443: S 1952941204:1952941204(0) win 29200 <mss 1460,sackOK,timestamp 1622119424 0,nop,wscale 7>

Phase: 5
Type: ACCESS-LIST
Subtype:
Result: DROP
Config:
Implicit Rule
Additional Information:

Result:
input-interface: OUTSIDE1_INT(vrfid:0)
input-status: up
input-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule, Drop-location: frame 0x00005623b4b75ccf flow (NA)/NA

21: 07:37:38.064526 61.245.4.9.63298 > 188.172.208.139.5938: P 3659605041:3659605097(56) ack 419956790 win 1019
22: 07:37:38.138603 188.172.208.139.5938 > 61.245.4.9.63298: P 419956790:419956846(56) ack 3659605097 win 10310

Phase: 5
Type: ACCESS-LIST
Subtype:
Result: DROP
Config:
Implicit Rule
Additional Information:

Result:
input-interface: OUTSIDE1_INT(vrfid:0)
input-status: up
input-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule, Drop-location: frame 0x00005623b4b75ccf flow (NA)/NA

171: 07:37:41.378734 54.192.150.44.443 > 61.245.4.9.64536: S 1880898966:1880898966(0) ack 379764042 win 65535 <mss 1440,nop,nop,sackOK,nop,wscale 9>

Phase: 5
Type: ACCESS-LIST
Subtype:
Result: DROP
Config:
Implicit Rule
Additional Information:

Result:
input-interface: OUTSIDE1_INT(vrfid:0)
input-status: up
input-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule, Drop-location: frame 0x00005623b4b75ccf flow (NA)/NA

178: 07:37:41.464773 54.192.150.44.443 > 61.245.4.9.12066: . ack 1165596856 win 65535

Phase: 5
Type: ACCESS-LIST
Subtype:
Result: DROP
Config:
Implicit Rule
Additional Information:

Result:
input-interface: OUTSIDE1_INT(vrfid:0)
input-status: up
input-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule, Drop-location: frame 0x00005623b4b75ccf flow (NA)/NA

188: 07:37:41.482991 20.107.224.24.443 > 61.245.4.9.61721: . ack 2573826798 win 16387

Phase: 5
Type: ACCESS-LIST
Subtype:
Result: DROP
Config:
Implicit Rule
Additional Information:

Result:
input-interface: OUTSIDE1_INT(vrfid:0)
input-status: up
input-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule, Drop-location: frame 0x00005623b4b75ccf flow (NA)/NA

204: 07:37:41.581360 61.245.4.9.64536 > 54.192.150.44.443: P 379764042:379764359(317) ack 1880898967 win 128
205: 07:37:41.581512 52.27.216.242.443 > 61.245.4.9.64534: . ack 3247401357 win 110

Here is the ASP DROP result: No more Snort drop but there are a lot of other drops.

[MHM Cisco World]

can you share the packet tracer (after add detail keyword in end) with it result 

Thanks A Lot
MHM

[Tritontek]

Here are some counters a ran using Packet Tracer detailed and Packet Capture in FMC GUI. I have noticed that if i run some captures in FMC GUI snort drops some packets to an IP but when i run packet tracer detailed in CLI it gives me a pass or allow results which is so confusing and most of these IP adresses are from amazon, cloudfront and i am not sure if these are safe or not.

Here are 2 of the few results below:

nslookup 52.222.174.37 - server-52-222-174-37.cdg50.r.cloudfront.net

FROM FMC GUI PACKCAPTURE: 52.222.174.37

CAPTURE:
Phase: 5
Type: SNORT
Subtype:
Result: DROP
Config:
Additional Information:
Snort Trace:
50:9A:4C:50:3B:89 -> CC:7F:76:49:EE:66 0800
172.21.1.10:50538 -> 52.222.174.37:443 proto 6 AS=0 ID=2 GR=1-1
Packet 1096542: TCP ***AP***, 11/07-18:26:15.709954, seq 1604190099, ack 3647251291, dsize 723
AppID: service: HTTPS(1122), client: SSL client(1296), payload: Cisco(2655), misc: (0)
Firewall: allow rule, 'INSIDE-TO-OUTSIDE_RULE' , allow
Policies: Network 0, Inspection 0, Detection 0
Verdict: pass
Snort Verdict: (block-packet) drop this packet

Result:
input-interface: INSIDE_INT(vrfid:0)
input-status: up
input-line-status: up
Action: drop
Drop-reason: (none) Not a blocking packet, Drop-location: frame 0x00005623b5a38112 flow (NA)/NA

108: 18:26:15.712426 52.222.174.37.443 > 172.21.1.10.50538: . ack 1604190822 win 64
109: 18:26:15.905913 52.222.174.37.443 > 172.21.1.10.50538: P 3647251291:3647251525(234) ack 1604190822 win 131
110: 18:26:15.906676 172.21.1.10.50538 > 52.222.174.37.443: P 1604190822:1604190886(64) ack 3647251525 win 1023

FROM FTD CLI PACKET TRACE: 52.222.174.37

> packet-tracer input INSIDE_INT tcp 172.21.1.10 443 52.222.174.37 443 detailed

Phase: 12
Type: EXTERNAL-INSPECT
Subtype:
Result: ALLOW
Config:
Additional Information:
Application: 'SNORT Inspect'

Phase: 13
Type: SNORT
Subtype:
Result: ALLOW
Config:
Additional Information:
Snort Trace:
00:00:00:00:00:00 -> CC:7F:76:49:EE:66 0800
172.21.1.10:443 -> 52.222.174.37:443 proto 6 AS=0 ID=0 GR=1-1
Packet 518990: TCP ******S*, 11/07-18:27:48.479970, seq 1617196482, dsize 0
Session: new snort session
AppID: service: (0), client: (0), payload: (0), misc: (0)
Firewall: allow rule, id 268453889, allow
Policies: Network 0, Inspection 0, Detection 0
Verdict: pass
Snort Verdict: (pass-packet) allow this packet

nslookup 44.225.146.46 - ec2-44-225-146-46.us-west-2.compute.amazonaws.com

FROM FMC GUI PACKCAPTURE: 44.225.146.46 

Phase: 4
Type: EXTERNAL-INSPECT
Subtype:
Result: ALLOW
Config:
Additional Information:
Application: 'SNORT Inspect'

Phase: 5
Type: SNORT
Subtype:
Result: DROP
Config:
Additional Information:
Snort Trace:
50:9A:4C:50:3B:89 -> CC:7F:76:49:EE:66 0800
172.21.1.10:50540 -> 44.225.146.46:443 proto 6 AS=0 ID=2 GR=1-1
Packet 1096558: TCP ***AP***, 11/07-18:26:21.619963, seq 3630937333, ack 3142007512, dsize 616
AppID: service: HTTPS(1122), client: SSL client(1296), payload: AppDynamics(7166), misc: (0)
Firewall: allow rule, 'INSIDE-TO-OUTSIDE_RULE' , allow
Policies: Network 0, Inspection 0, Detection 0
Verdict: pass
Snort Verdict: (block-packet) drop this packet

Result:
input-interface: INSIDE_INT(vrfid:0)
input-status: up
input-line-status: up
Action: drop
Drop-reason: (none) Not a blocking packet, Drop-location: frame 0x00005623b5a38112 flow (NA)/NA

150: 18:26:21.630399 44.225.146.46.443 > 172.21.1.10.50540: . ack 3630937949 win 128
151: 18:26:21.834002 44.225.146.46.443 > 172.21.1.10.50540: . 3142007512:3142008892(1380) ack 3630937949 win 110
152: 18:26:21.834032 44.225.146.46.443 > 172.21.1.10.50540: . 3142008892:3142010272(1380) ack 3630937949 win 110
153: 18:26:21.834490 172.21.1.10.50540 > 44.225.146.46.443: . ack 3142010272 win 1024

FROM FTD CLI PACKET TRACE: 44.225.146.46

Phase: 12
Type: EXTERNAL-INSPECT
Subtype:
Result: ALLOW
Config:
Additional Information:
Application: 'SNORT Inspect'

Phase: 13
Type: SNORT
Subtype:
Result: ALLOW
Config:
Additional Information:
Snort Trace:
00:00:00:00:00:00 -> CC:7F:76:49:EE:66 0800
172.21.1.10:443 -> 44.225.146.46:443 proto 6 AS=0 ID=1 GR=1-1
Packet 850434: TCP ******S*, 11/07-18:29:30.859940, seq 1410578557, dsize 0
Session: new snort session
AppID: service: (0), client: (0), payload: (0), misc: (0)
Firewall: allow rule, id 268453889, allow
Policies: Network 0, Inspection 0, Detection 0
Verdict: pass
Snort Verdict: (pass-packet) allow this packet

this is so confusing, everytime i run trace or capture there will be a new IP that will be blocked or dropped by snort and mostly are from amazon AWS. I dont know where and why snort is blocking this since this is a legit and known IP with proper DNS record.

[MHM Cisco World]

do you deploy the ACP after add it?

Thanks A Lot
MHM

[Tritontek]

Yes sir! Definitely! Every changes we make we always deploy it to the device and we only have 1 device.

i am confused already, right now our FTD is connected to an isolated network and am not confident of this is ready for redeployment to the production network. Is this some kind of a bug?

[Marvin Rhoads]

You may find it useful to use the command "system support firewall-engine-debug" to troubleshoot a specific source or destination address.

This article explains its use in detail:

https://www.cisco.com/c/en/us/support/docs/security/firepower-ngfw/214577-firepower-data-path-troubleshooting-phas.html

[Tritontek]

Hi Sir,

may i ask what is the best way to bypass all inspection and just have a straight forward packet in and packet out. i think that is the best way for me to determine where the problem is. i want to disable all inspections first then enable them 1 by 1 to know which one is blocking and which one is not.

[Marvin Rhoads]

You have not yet shared any screen shot of your prefilter policy or Access Control Policy. Either of those could be affecting your trace results. Otherwise, the title of this thread would generally work and have no issues.

You also did not try the "system support firewall-engine-debug" and share the output that I suggested earlier.

[Tritontek]

Hi Sir,

Apologies, i forgot to attach the files. Here are the details below sir. I have also attached the debug txt file in this reply.

Here is my Pre-Filter Policy.

Here is my ACP, all the blocking rules are currently disabled.

and the ACP advanced settings i made

The problem right now is that our FTD is now running in an isolated network where 1 PC is connected with necessary production apps and websites being used and accessed and whenever we re-integrate the FTD back to the production network that is the time the problem is rampant. 

Here are some errors we had before we isolated the FTD.

Here is Tekla Trimble Model Sharing queuing to cloud servers:

This one is for VMWare Horizon connection to cloud servers:

right now production network without the FTD is running smoothly but everytime we join the FTD back to the network everyone gets crazy with this cloud server connections being dropped/blocked. This problem has been running for months on/off and randomly.

[Marvin Rhoads]

The debug.txt doesn't show any drops or blocks so that is inconclusive.

I notice in your ACP that you appear to have multiple outside interfaces. Are both active and, if so, how do you handle routing to ensure that nothing is ever asymmetric (i.e. outbound traffic goes out one interface while return traffic arrives on a different one)?

[Tritontek]

Hi Sir, i have ran a new packet trace without pre-filter policy and i can see a lot of Snort drop and acl-drop including our Amazon and Azure cloud servers. that is the main concern we are facing that Snort is blocking our production servers from the cloud. i have attached the CLI log file and packet capture.

Regarding the 2 outside interface, as of now only 1 is active since it is in isolated network but when we integrate the FTD to the production 2 Outside interfaces will be utilized, I use metric 1 for my outside 1 and metric 2 for my outside2 and they are assigned into 2 different zones and i also have a PBR flexconfig so that i can distribute the internet usage into 2 depending on what vlans the users belong. see images below.

Hi Sir @Marius Gunnerud i have posted it here everything including the system support logs

see log files attached. this log files were captured while pre-filter policy, service policy was removed, ACP allow rule is active and ACP default action was set to Balanced Connectivity and Security. I am confused on how to deal with this one because i cannot afford to run a network with Pre-filter any any and fastpath since it is very vurnerable and it is also hard to add all our amazon cloud server IPs because some of our client's cloud server IP are in multi region and it changes from time to time. i dont know what really happend to this FTD but the issue just suddenly came out without doing anything. Maybe this is a bug? snort should only block/drop unregistered/broken/malicious packets but why block/drop amazon/azure IP packets?