FMC Deployment Failed with Sensitive Data Detection RegEx

CRadoumis — Fri, 21 Feb 2020 16:09:02 GMT

I am trying to push a Sensitive Data Detection Policy to detect email and possible passwords being entered in a URL, but whenever I deploy the policy I am given a Snort validation error.

FMC 6.2.3.2

ASA with FP Module 6.2.2

Here is the RegEx - Nothing special. Some word boundaries, and forward lookups.

(?=.*?\b(u|usernames?|users?|uname)\b)(?=.*\b([a-zA-Z0-9.!#$%&'*+\/=?^_`\{|\}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)))(?=.*\b(p|passwd|password|pswd|pass)\b).*$

Here is the log message that is shown during a 'pigtail deploy' from the module itself.

Validating snort configuration at /var/tmp/Apply_38654709268/code//SF/NGFW/PolicyApply.pm line 1945.
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: SYSTEM COMMAND: /usr/local/sf/bin/testSnortConfiguration.sh /var/sf/detection_engines/a9fbe130-830f-11e6-b01b-65ed47c105ae/libs /var/sf/detection_engines/a9fbe130-830f-11e6-b01b-65ed47c105ae/snort /var/sf/detection_engines/a9fbe130-830f-11e6-b01b-65ed47c105ae/snort.conf /var/cisco/deploy/sandbox/snortTest/output.txt /var/cisco/deploy/sandbox/snortTest/now --treat-drop-as-alert -G 2 -T -A none -Q --daq pcap --daq-dir /usr/local/sf/lib/daq -l /var/cisco/deploy/sandbox/snortTest/ --dirty-pig --suppress-config-log -d
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: SYSTEM RESULT: $VAR1 = {
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: 'stderr' => undef,
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: 'stdout' => '>> COMMAND TO RUN AT PROMPT
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: >> export LD_LIBRARY_PATH=/var/sf/detection_engines/a9fbe130-830f-11e6-b01b-65ed47c105ae/libs;
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: >> /var/sf/detection_engines/a9fbe130-830f-11e6-b01b-65ed47c105ae/snort -c /var/sf/detection_engines/a9fbe130-830f-11e6-b01b-65ed47c105ae/snort.conf -Z /var/cisco/deploy/sandbox/snortTest/now --treat-drop-as-alert -G 2 -T -A none -Q --daq pcap --daq-dir /usr/local/sf/lib/daq -l /var/cisco/deploy/sandbox/snortTest/ --dirty-pig --suppress-config-log -N > /var/cisco/deploy/sandbox/snortTest/output.txt 2>&1
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: >>
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: ',
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: 'rcode' => 256
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: };
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: SYSTEM OUTPUT
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: Enabling inline operation
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: Running in Test mode
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: ERROR: SDF Pattern "(?=.*?\b(u|usernames?|users?|uname)\b)(?=.*\b([a-zA-Z0-9.!#$%&'*+\/=?^_`\{|\}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)))(?=.*\b(p|passwd|password|pswd|pass)\b).*$" contains curly brackets with non-digits inside.
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: Fatal Error, Quitting..
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: Snort configuration invalid. at /var/tmp/Apply_38654709268/code//SF/NGFW/PolicyApply.pm line 1991, <OUTPUT> line 4.
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: Error validating snort configuration at /var/tmp/Apply_38654709268/code//SF/NGFW/PolicyApply.pm line 2041.
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: ERRORS:
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: $VAR1 = {
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: 'errorStruct' => [
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: {
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: 'line_number' => undef,
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: 'section' => 'UNKNOWN',
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: 'uuid' => undef,
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: 'message' => 'ERROR: SDF Pattern "(?=.*?\\b(u|usernames?|users?|uname)\\b)(?=.*\\b([a-zA-Z0-9.!#$%&\'*+\\/=?^_`\\{|\\}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)))(?=.*\\b(p|passwd|password|pswd|pass)\\b).*$" contains curly brackets with non-digits inside.
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: '
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: }
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: ],
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: 'errorLines' => [
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: 'ERROR: SDF Pattern "(?=.*?\\b(u|usernames?|users?|uname)\\b)(?=.*\\b([a-zA-Z0-9.!#$%&\'*+\\/=?^_`\\{|\\}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)))(?=.*\\b(p|passwd|password|pswd|pass)\\b).*$" contains curly brackets with non-digits inside.
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: '
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: ]
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: };
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: $VAR1 = bless( {
ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: '-stacktrace' => 'Snort configuration validation failed due to ERROR: SDF Pattern "(?=.*?\\b(u|usernames?|users?|uname)\\b)(?=.*\\b([a-zA-Z0-9.!#$%&\'*+\\/=?^_`\\{|\\}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)))(?=.*\\b(p|passwd|password|pswd|pass)\\b).*$" contains curly brackets with non-digits inside.

As you can see, the '\' characters are being appended and negated by another '\' making the RegEx malformed. I have opened a TAC case on this but they are saying they can't look at RegEx, but are not explaining why the RegEx is being rewritten in the policy deploy process. When going back to the policy itself, I am seeing the correct RegEx. It is only in the deploy logs that shows the invalid one.

ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: ERROR: SDF Pattern "(?=.*?\b(u|usernames?|users?|uname)\b)(?=.*\b([a-zA-Z0-9.!#$%&'*+\/=?^_`\{|\}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)))(?=.*\b(p|passwd|password|pswd|pass)\b).*$" contains curly brackets with non-digits inside.

is being rewritten with

ACTQ: 08-24 18:13:04 FIREPOWER ActionQueueScrape.pl[378]: 'ERROR: SDF Pattern "(?=.*?\\b(u|usernames?|users?|uname)\\b)(?=.*\\b([a-zA-Z0-9.!#$%&\'*+\\/=?^_`\\{|\\}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)))(?=.*\\b(p|passwd|password|pswd|pass)\\b).*$" contains curly brackets with non-digits inside.

Does anyone have any idea how to fix this, or to adjust the RegEx to make it so it is not rewritten in policy deploy?

Thank you much.

Re: FMC Deployment Failed with Sensitive Data Detection RegEx

Mohammed al Baqari — Fri, 24 Aug 2018 19:41:05 GMT

Based on the error message, there is a curly bracket without digits in the
regex which is this one {|\}

Try to replace it or for testing remove it.

topic FMC Deployment Failed with Sensitive Data Detection RegEx in Network Security

FMC Deployment Failed with Sensitive Data Detection RegEx

Re: FMC Deployment Failed with Sensitive Data Detection RegEx