Firepower and snort issues

[CraigTompkins99918]

Hello all.  My company has been using Cisco ASA in the datacenter for years and I'm pretty comfortable with them.  We recently replaced them with Firepower 2100's as our ASAs went end of life and we were sold on the added benefit of FTD.  Since installing them about a month ago we've had 3 separate issues where applications don't work and it's come back to the Snort engine.  The first time, the issue went away and we don't know why (no changes between it not working and the next day it was working).  The 2nd is communication with Ring Central (our VoIP provider) where neither TAC nor RC support can help, they end up pointing the finger at each other.  To fix this we testing routing only RC traffic out the ASA and that worked.  Replacing the EOL ASA with current Meraki MX tonight.  The 3rd happened today.

We have setup a new relationship with another company we need to share files with.  They only allow SFTP over port 990.  So I created firewall rules for the 3 offices that have users where this software will be used.  In small branch offices with Meraki firewalls, my firewall rule works as expected.  From the Datacenter, behind the Firewpower the same rule does not work.  The Access rule is allow any source IP any source port to connect from the inside interface to the other company's public IP over port 990.

As you can guess from the above, it does not work.  A FMC packet trace shows the packet was dropped.  Type is SNORT and the drop reason is the same as always "(firewall) Blocked or blacklisted by the firewall preprocessor".

What does that even mean?  Why would my allow rule not be honored? I've attached the relevant SNORT block info with IP addresses blacked out.

TIA for any help

[Marius Gunnerud]

Are you hairpinning the traffic on the outside interface?  If not, then the image you have provided does not depict the traffic you are referring to.

Are you using application matching?  I have seen several times where traffic does not match the expected rule because there is a Security Intelligence update which causes the FTD to misinterpret the application header being used and therefore the traffic is no longer matched.  On the other side, I have also seen an update from the application developers where, also, the headers are slightly changed therefore not matching the application setting in the rule.

Because of these inconsistencies I have opted to only use application when it is not possible to match purely on the source / destination and port, for example chat applications such as facebook messenger.

[CraigTompkins99918]

We are not hairpinning traffic.  It really should be as simple as SFTP app runs in VDI desktop in Datacenter -> default gateway is Nexus 9K doing all the routing in the DC -> firepower edge device -> Internet.  Then of course the external server replies and it comes in in reverse.

I'm not using any application matching.

[Marius Gunnerud]

Well, the screenshot you provided shows that the ingress interface and egress interface is the "outside" interface.  So something is not matching up on what you are seeing.

As for the traffic drop, are you using intrusion policy on the access rule?  If yes, try to disable it and see if traffic is now passed.  If traffic passes we have identified where the drop is occurring.

another possible reason for the drop could be security intelligence, that the destination IP has been flagged as not safe.

[MHM Cisco World]

the TCP is add as Conn in Lina and then the packet is gone to Snort, 
in Snort is talk 3 min. for traffic to inspect, if the inspection more than 3 min the Snort send recovery to LINA, 
here the packet is drop and you see the log error message. 
solution is config timeout as fig. below 
MHM 

[CraigTompkins99918]

Thank you for the input.  I've aske our CCIE consultant for some assistance as well and pointed him to this post.  Hopefully he'll find something I've missed.

[CraigTompkins99918]

Well, I was missing something.  I had a deny rule above the allow rule.  I don't know why I missed that, but I did.  The packet tracer still shows "(firewall) Blocked or blacklisted by the firewall preprocessor". but the application does actually work.  I'm glad it's working, but the packet tracer on the ASA was always accurate.  Apparently this one is not.

[Marvin Rhoads]

Packet-tracer will always fail for certain traffic types since it's a synthetic packet without the expected application payload. It's better to troubleshoot using packet capture with trace and actual application traffic via the FMC advanced troubleshooting tools for the device (or from the cli using system support firewall-engine-debug).

That said, you may find it useful to move the flow into your prefilter policy with action = fastpath if you trust the source and destination. That will avoid Snort altogether.

[CraigTompkins99918]

Cisco TAC did try moving the flow into the prefilter policy as you suggested with our RingCentral issue and that didn't help.

[Herald Sison]

is this resolved? how? i have a somewhat similar issue with you just this month. we are using FTD7.0.6 with FMC7.3.1 ASA5508X box, my issue was that a particular application using a model sharing feature via cloud servers and it suddenly drop its connection. At first the problem will just affect a few random cloud users then later on it will affect to all cloud users.

i have disabled all ACP rule in my ACPolicy and just used the default action and select "Trust All Traffic" and even created a Pre-Filter Policy that sets any any network with FastPath action but still the same issue occurs and TAC ran some TS in to the CLI and we saw some these and the list goes on.

326: 12:06:43.000030 192.168.7.122.54741 > 20.189.173.14.443: P 1630889135:1630889364(229) ack 3677944706 win 1024 Drop-reason: (snort-block) Packet is blocked as requested by snort, Drop-location: frame 0x000055e9e3316112 flow (NA)/NA

327: 12:06:43.257463 192.168.5.100.56591 > 142.251.221.3.443: P 91537800:91538757(957) ack 1105471243 win 512 Drop-reason: (snort-block) Packet is blocked as requested by snort, Drop-location: frame 0x000055e9e3316112 flow (NA)/NA

328: 12:06:43.305953 192.168.9.102.55730 > 13.107.136.10.443: P 1070331683:1070331874(191) ack 3766062448 win 1024 Drop-reason: (snort-block) Packet is blocked as requested by snort, Drop-location: frame 0x000055e9e3316112 flow (NA)/NA

and these

for more than months already TAC still cant find any workaround and cant even say what was the issue or problem.

[RAL]

Had something similar with CDO-FMC & FPR1120-v7.3.0

Mandatory section:
ACL-Rule1 permits servers 192.168.x.1+2 to server 10.x.x.1 on multiple ports, including http
ACL-Rule2 drops 192.168.x.0/24 to RFC1918 addresses on http+https
192.168.x.1 to 10.x.x.1:80 traffic dropped by SNORT.

Moved permit rule to pre-filter section with fastpath.
Traffic not matching the ACL, and still dropped by SNORT.

Various other tweaks, tried with no effect.
Changed the the pre-filter rule to contain http only, instead of port-group or multiple ports.
Traffic now matching pre-filter ACL and logs show being permitted in fastpath.