FMC FTD HA Management - Cloud Delivered FMC does what on prem FMC cant

[cmcclinton]

Cisco have recently launched Cloud Delivered FMC, and the question that comes to mind is surely when using FMC from the cloud, every FTD will be remote, and therefore HA management of remote office FTDs will need to be supported!

Support for remote branch HA FTDs is not supported by FMC and this has been a major pain point for many customers, and quite frankly is an embarrassment for Cisco. 

So Cloud Delivered FMC launches and reading the manual for Cloud-Delivered Firewall Management Center immediately shows that straight out of the gate this pain point has been fixed it does indeed support remote HA management and specifically describes it.

"High Availability Support on Threat Defense Devices in a Remote Branch Office Deployment" https://www.cisco.com/c/en/us/td/docs/security/cdo/cloud-delivered-firewall-management-center-in-cdo/managing-firewall-threat-defense-services-with-cisco-defense-orchestrator/m_device-ops-ha.html

The equivalent section of the 7.2 on premise manual does not have that sub section. https://www.cisco.com/c/en/us/td/docs/security/secure-firewall/management-center/device-config/720/management-center-device-config-72/high-availability.html

So on premise FMC customers who have been asking for this feature get... well get <insert rude word>!

Why?
When is this being adding to on premise FMC?
If it can be done from Cloud FMC there is no reason it can't work from on prem FMC.

This feature has been sorely missing for a long time and now it pops up as fully supported in Cloud FMC!
I hope i am very wrong but, is this a case of features being held back from FMC to push customers to subscription based Cloud FMC?!

I hope I am wrong and this is coming to FMC very very soon. I am not too pleased seeing this feature disparity, as a Cisco partner I have been an apologist for some the short comings of Firepower for quite a few years now, but have given Cisco the benefit of the doubt that these feature would come with time.

[Eric R. Jones]

I'm sure it's been asked but, Is it easier to modify and/or faster to roll
out modifications to cloud based apps then it is to modify on prem FMC? I
too have issues with Cisco move to the MS model of nickel and diming you on
licenses subscriptions as well as other providers.

[Marius Gunnerud]

I am not entirely sure what you are missing here?

The first document you are mentioning is for managing FTD devices at remote offices via FMC via CDO which is allowing you to use any data interface for this purpose.

As for on-prem FMC you have two options to manage remote offices.  The first option and my preferred option for this, is to configure that "outside" data interface, or internet facing interface, as the management interface.  High availability will still be available here, but you would need a secondary public IP for the standby device.  Just remember to configure access lists for SSH access.

The second option is to manage the devices over a site to site VPN.  Not optimal if the VPN goes down for whatever reason, but still, you will be able to configure the FTD devices in HA.

[cmcclinton]

Hi Marius

I hope I am wrong about this as it would make FTD HA deployment much easier. I am aware of remote management via the outside data interface. But are you sure about deploying HA pairs using this management method?

The manual specifically calls it out as not supported "About Using the FTD Data interface for Management" , "High Availability is not supported. You must use the Management interface in this case."

https://www.cisco.com/c/en/us/td/docs/security/secure-firewall/management-center/device-config/720/management-center-device-config-72/get-started-device-management.html

I raised a case with Cisco GVE and they confirmed that HA pairs are not supported in branch office deployments unless FMC has direct access to the management ports, by setting up a secondary management network using a management VPN as you describe.

In my case the customer does have multiple public IPs that could be used for each firewall in the HA pair as you also describe, but the manual and Cisco GVE are saying HA is not supported. Then out comes cloud FMC and that manual specifically discusses and describes remote HA support, which is great news if the on premise version also clearly supported the same functionality.

Have you successfully deployed HA pairs managed by a remote FMC?
Do they test and function as expected?

Even if they do the manual still reads "High Availability is not supported."

I guess that is what I am missing and I have ben reluctant to deploy FTD in HA pairs at remote sites because of that statement.

[Massimo Baschieri]

It seems to me you forgot a last option, which is placing management interfaces directly on the wild internet, you need 4 public ip addresses in case of an HA pair and you need to apply ssh access list on the management interfaces, but it works.

[giovanni.augusto]

you can also do this way: (code 6.7+) to mange FTDs in HA via a remote Internet facing FMC on Internet only sites (no internal connectivity like SDWAN or MPLS)

prerequisites :

• FMCs reachable via Internet (either 1:1 NAT or via an internet interface)
• Register FTDs always with NAT-ID and without IP address entry on the FMCs

Steps:

1. configure network on the primary FTD
2. change management interface on FTD from management to data interface, choose your outside Internet 
3. configure Internet facing IP addresses on the data interface and verify internet reachability
4. start registering your FMC using its public IP address on the FTD use NAT-ID while doing this
5. add your FTD in the FMC by the same means: use NAT-ID and do not use a fixed IP
6. Once FTD is registered, deploy an ACP and NAT policies to allow FTD management (internal)  interface to access the public IP of the FMC and consider the necessary 1:n DNAT (and consider the rest like routing, etc), deploy the policies (In case you plan to have 2 FTDs in HA include both management IPs in both policies)
7. in FMC change management interface for the FTD from data to internal management: using NAT-ID and the previous policies will allow the FTD to automatically restore FTD->FMC connectivity for the sftunnel
8. in case you have a secondary FTD you can register it to FMC directly from the management interface pointing to the FMC public IP (is possible because of #6) and use NAT-ID also in this case (equal to step #4)
9. once registered the secondary FTD you can manage the HA regularly from the FMC.

Notes:

1. if you 1:1 NAT your FMC internal IP and you have 2 FMCs in HA, you will have to modify the FTD registered manager IP of the secondary at least (configure manger edit should be the command). To avoid this you could consider using an interface on the FMC with a public IP attached to it (not suitable for cloud deployed FMCv though)
2. I suggest you template the NAT and ACP policies to achieve this in a policy hierarchy so you can restrict access to it and operate only on a child policy for each firewall

[Massimo Baschieri]

Nice trick, but what if something goes wrong during a deployment or you make mistakes in the configuration and you lose management interface from FMC?

[Marius Gunnerud]

Deployment is separate from the management interface, so unless you are routing the management traffic through an FTD data interface you will not be affected by a wrong deployment.  The only time you will have issues is if the ISP has done something that will affect access to the internet.

[giovanni.augusto]

If you have a mistake at the initial deployment you can rollback via CLI.

An obvious assumption is that you have CLI access (no ZTP for physical boxes I think, I have read some preconfig option only for virtual FWs but didn't try that yet)

you can find the rollback command here : https://www.cisco.com/c/en/us/td/docs/security/firepower/command_ref/b_Command_Reference_for_Firepower_Threat_Defense/c_3.html#wp2778684510

As I mentioned you should minimize mistakes in your configuration for this critical part and this is why I suggest you template it by using at least policy inheritance (a 2 level inheritance should be sufficient for you to have control) and use override objects so you don't have critical configuration portion in your regular policy.

NOTE1: This whole procedure is valid if you need an HA deployment with FTDs->FMC and you have no internal connectivity option (SDWAN/MPLS/etc)

NOTE2: In case you have an option for internal connectivity but somehow need the FTD operational to bootstrap your connectivity, you can follow up to step #6 then configure your polices for connectivity and then continue on to step 7 but then instead of routing traffic through the FTD itself you can just use your FMC internal IP (if any). This is also the point of using NAT-ID only, to have such flexibility

NOTE3: I have not worked with the most recent FTD/FMC version to time (7.2) so I don't know if an option was finally introduced to have HA deployments over data-interface management with FTD->FMC

[Massimo Baschieri]

Nice to know, it can be useful in case you do not have enough public ip addresses available in order to place management interfaces on the internet.

Please note that the document staes that rollback procedure doesn't work in case of HA, I think in that case you need to do a much annoying task like breaking HA before trying rollback procedure or even repeating your onboarding procedure from the very beginning.

[Marius Gunnerud]

You need to remember that the management interface and the Data interfaces are separate.  So if you are not routing the management interface traffic through the FTD data interfaces, i.e. the management interface as a public IP with the default gateway being the ISP, any deployment mistakes you make will not affect the management traffic.

[cmcclinton]

Hi all

Thanks for all of the suggestions for remote HA management, I had come across most of them or variations of them in other forums or posts. My original comment was sharing and expressing some frustration that cloud deployed FMC now supports HA natively over data management interfaces. 

I thing the very fact that customers have had to come up with as many creative solutions as they have to simply manage a remote pair of HA firewalls illustrates that there is a problem here that hasn't been addressed by Cisco as quickly as it should be.

Some of these creative workarounds can be difficult to convince customers to approve when Cisco themselves don't support remote branch HA deployments. Indeed in many cases even convincing customers to stick with Cisco firewalls at all can be challenging when competitors can point to Cisco documentation that clearly expresses the limits of remote FMC management.

Remote HA is bad enough, lets not even bring up PPPoE outside interfaces and remote FMC management.

[giovanni.augusto]

---

My original comment was sharing and expressing some frustration that cloud deployed FMC now supports HA natively over data management interfaces. 

Do you mean CDO or is it a newer version of FMC?

[cmcclinton]

It is FMC in CDO that supports it, but reading the manual and looking at the deployment examples FMC is doing the work.
I just fail to to see why this cant be done by on premise FMC alone, its a software limitation that Cisco need to fix, and have needed to fix for sometime now. 

Hey, maybe I am wrong on this and its fine the way it is. 

https://www.cisco.com/c/en/us/td/docs/security/cdo/cloud-delivered-firewall-management-center-in-cdo/managing-firewall-threat-defense-services-with-cisco-defense-orchestrator/m_device-ops-ha.html