FMC Deleted Rule by itself !?

ida71 — Mon, 10 May 2021 08:35:18 GMT

We recently had a major issue where the FMC deleted a rule apparently by itself !

FMC1600 physical device running 6.6.1

Senario, I deleted some out of date office IP's & associated rules. Applied policy to FTD's & a major incident became evident. After some diagnosis it transpired that our primary customer web access rules was missing !

Looking at the policy diff compare, shows all the actions I took, then at the bottom (last item) it shows the web access rules as deleted from old policy & added in new saved policy. But when looking at the live installed policy the rule was missing !?

I recreated the rule manually (no roll back function in FMC) which restored the service. Cisco TAC agree the diff compare is odd, but can not provide an explanation for why a rule that was NOT modified appeared in the diff compare nor why it was missing from the policy where it clearly indicates it should exist ???

We have tried various tests with TAC, but troubleshoot files don't indicate any issues & TAC have NOT been able to replicate it in a lab, nor restore my backups, as they have discovered that physical FMC backup will NOT restore to Virtual LAB vFMC.

Has anyone else experienced any weird rule issues ?

Points to note,

FMC diff compare does NOT record actual rule numbers, it numbers the changes as Rule1-RuleX as they are made, no relation to the rulebase rule number, only the Rulename is consistent with the rulebase.

The audit log does NOT record changes to policy, only the policy save diff compare shows changes made between policy opened & new policy saved.

Re: FMC Deleted Rule by itself !?

Marvin Rhoads — Mon, 10 May 2021 09:02:16 GMT

I have encountered a similar issue with Firepower 6.7.0 managed with FDM. In my base, site-site VPN configs were lost and had to be recreated manually. TAC was likewise unable to replicate - even though they observed it happening in real time on the production Firepower 2140 HA pair.

Re: FMC Deleted Rule by itself !?

ida71 — Mon, 10 May 2021 09:10:48 GMT

Thanks Marvin, at least I'm not going crazy. I have spent a while looking at previous diff compares & I don't see this replicated previously. As an indication there have been over 5000 successful policy changes in the last 14 months.

As a precaution we are now viewing policy diff compares before applying policy & only applying policy out of core hours, which is a pain.

topic Re: FMC Deleted Rule by itself !? in Network Security

FMC Deleted Rule by itself !?

Re: FMC Deleted Rule by itself !?

Re: FMC Deleted Rule by itself !?