Firepower deployments really slow

[ncowger]

I have new pair of NGFW 2110's.  I have a virtual FPMC.  This is a new build with relatively few rules (10) and NAT statements (14).  If I make a simple change to the policy and deploy it, it seems to take a really long time.  I'm regularly seeing 7+ minutes.  Is this normal?  Why?     

[mikael.lahtela]

Hi everyone,

I'm working with many different deployments and I would say 8 minutes with FMCv and HA pair 2110 is normal.

There is a big difference on a empty box, stand alone or ha pair. ranging from 2 minutes to 10 minutes.

I believe Cisco will be doing something about this in coming releases.

br, Micke

[adammckay1]

It's the same for me on a physical FPMC 1000 with around 15 rules and some very basic NAT & HA configuration, for a single FPR2110 pair - somewhere between 5-7 minutes per deploy even with a single change. I wouldn't say this is a FMCv-specific issue at all and from the horses mouth I was told this was "normal".

It's frustrating because under some circumstances traffic may be dropped during a deploy (the circumstances where this can happen are vague and the documentation has conflicting information with the on-box help, which has information that conflicts with other on-box help I just double-checked and it looks like the documentation has been updated to be clearer). We're scheduling any policy change for after-hours as a result, even if it's a single access policy item addition or removal.

[workforcesoftware]

Yeah, I've also heard this is normal from several resources within Cisco. The issue of traffic dropping on deployment is the biggest issue I have with the new system. Gone are the days of making changes during production hours, with little to no impact on the end-user. That was the one thing I loved the most about the ASAs, especially at our headquarters.

[c.flavell]

I have a ASA5506 converted to FTD (6.2.3.4-42) and using FDM (the local manager) and even that is slow. A simple change to the BVI address on a empty firewall takes minutes. Either the deployment manager is trying to connect to some external server or the deployment is on a clock cycle so only checks for work every x seconds but it is unacceptable. Interestingly even show network from the console CLI takes a few seconds to respond. It also takes quite a while after boot for the https server to become available.

[Marvin Rhoads]

6.3 has improved deployment times significantly (~2x better). Unfortunately the ASA 5506-X and 5512-X are not eligible for 6.3 upgrades.

[elcommunication]

I'm new into the ASA firepower stuff and I think the deployment times are really slow up to 5 minutes. I'm getting gray hair before they're done. And if I deploy a change on a live environment and figure out the rule breaks connectivity for my users it takes at least 5 minutes to revert the changes

[Nikolaj Pabst]

Hi,

Are you running 6.2.3.X and is it a cluster?

In general 6.2.3 are MUCH faster than previous releases, and will give you a much better experience.

[elcommunication]

I'm running  6.2.3.1 but it's not a cluster.

[Nikolaj Pabst]

How are the hardware on the VM?

[elcommunication]

It's the default on the VM. 4 core and 8GB ram. And the actual host has dual six core amd opteron 2435 with very low load

[Nikolaj Pabst]

Try boosting it to 32GB Memory - it should be treated as a database server :-)

I guess you will get a huge performance boost.

[elcommunication]

Doubled the ram to 16GB. Still a 7 minute deploy-time on a simple ACL line change.

But before I rebooted it used about 7.2 of 8GB RAM and now with 16GB about the same

[Marvin Rhoads]

The slow deployments are primarily due to architectural limitations of the underlying database design - not the resources on either the FMC or managed device.

Cisco has been working on improving this but it's not there just yet.

[shaun.stull1]

I've got a pair of 2110's running in HA and rarely see a deployment that finishes in less than 7 minutes.  I am told by Cisco that this is the way it is and improvements are coming in the next release.  I heard the same thing prior to upgrading to 6.2.3 as well and didn't see much if any improvement... 

[robinson]

Any updates here? I'm inheriting a 2110 with an FMCv, simple changes take 7 minutes. It's 3/2019, there has to be fix by now? One ACL take 7 minutes? That's just crazy.