topic Re: NGFWv-AWS - EBS volume encryption issue in Network Security

NGFWv-AWS - EBS volume encryption issue

varunes10 — Mon, 27 Jan 2020 17:29:38 GMT

I am running multiple NGFWv in our AWS environment from the marketplace. When trying to setup the EC2 instance for the firewall image and if I selected to encrypt the ebs volume with default aws/ebs key then the appliance does not respond on boot up. I also do not see any system logs for the EC2 during boot up. I do see system logs while terminating the instance. We have a policy to have all ebs volumes encrypted else will be flagged non compliant. Does the NGFWv support encryption of the ebs volume in AWS? Any insight into this will be helpful. Thank you.

System logs on shutting the instance down.

IO memory blocks requested from bigphys 32bit: 87680

INIT: version 2.88 booting

Starting udev

Configuring network interfaces... done.

Populating dev cache

TODO: Remove /tmp/disable_dpdk to enable dpdk on ngfwv

Found virtual boot drive /dev/xvda1

Found virtual disk0 drive /dev/xvda2

fsck.fat 3.0.28 (2015-05-16)

Starting check/repair pass.

FATs differ but appear to be intact. Using first FAT.

Cluster 258046 out of range (207673272 > 2092549). Setting to EOF.

Cluster 258047 out of range (223535692 > 2092549). Setting to EOF.

Cluster 258048 out of range (10059955 > 2092549). Setting to EOF.

Cluster 258049 out of range (133079543 > 2092549). Setting to EOF.

Cluster 258050 out of range (65038938 > 2092549). Setting to EOF.

Cluster 258051 out of range (219439810 > 2092549). Setting to EOF.

Cluster 258052 out of range (46504770 > 2092549). Setting to EOF.

Cluster 258053 out of range (10103903 > 2092549). Setting to EOF.

Cluster 258054 out of range (250869187 > 2092549). Setting to EOF.

Cluster 258055 out of range (131507214 > 2092549). Setting to EOF.

Cluster 258056 out of range (79941071 > 2092549). Setting to EOF.

Cluster 258057 out of range (6384958 > 2092549). Setting to EOF.

Cluster 258058 out of range (259674023 > 2092549). Setting to EOF.

Cluster 258059 out of range (51362568 > 2092549). Setting to EOF.

Cluster 258060 out of range (262079956 > 2092549). Setting to EOF.

Cluster 258061 out of range (256313087 > 2092549). Setting to EOF.

Cluster 258062 out of range (34401673 > 2092549). Setting to EOF.

Cluster 258063 out of range (257800812 > 2092549). Setting to EOF.`

Re: NGFWv-AWS - EBS volume encryption issue

jimholla — Mon, 27 Jan 2020 18:54:09 GMT

We are working with AWS on this issue. They have identified that their decryption algorithms are returning random data for uninitialized storage blocks. More to come on this.

Re: NGFWv-AWS - EBS volume encryption issue

varunes10 — Mon, 27 Jan 2020 19:21:34 GMT

Thank you for the response @jimholla. Please let me know when AWS finds a resolution for this issue so I can retry deploying the image with ebs encryption.

Re: NGFWv-AWS - EBS volume encryption issue

varunes10 — Mon, 24 Feb 2020 21:35:44 GMT

@jimholla - Hi, is there any update on this issue with AWS? Has this been resolved on their end?

Re: NGFWv-AWS - EBS volume encryption issue

jimholla — Tue, 25 Feb 2020 01:03:03 GMT

Sorry to say, but there has been no progress. AWS has identified the issue and they were working on a fix. I'll check on the status to see if we can push this forward.

Re: NGFWv-AWS - EBS volume encryption issue

varunes10 — Thu, 16 Apr 2020 15:54:17 GMT

Hi @jimholla,

Has there been any update from AWS on this issue? Please let me know. Thank you.

Re: NGFWv-AWS - EBS volume encryption issue

jimholla — Thu, 16 Apr 2020 16:59:07 GMT

Yes. We have tested a fix for this issue and it works! Now we need to republish our AMIs in AWS. Unfortunately that will take a couple of weeks.

Re: NGFWv-AWS - EBS volume encryption issue

varunes10 — Thu, 16 Apr 2020 17:01:25 GMT

Awesome. Thank you for the update @jimholla

Re: NGFWv-AWS - EBS volume encryption issue

pangrazi — Tue, 19 May 2020 22:40:21 GMT

I am still having the same issue even with the latest 6.6.0-90 ami published today. Is there any update on this?

Re: NGFWv-AWS - EBS volume encryption issue

jimholla — Wed, 20 May 2020 12:17:58 GMT

We have a fix for this issue but we need tom complete testing and then republish the AMIs. It will probably take a few more weeks before you can use encrypted storage.

Re: NGFWv-AWS - EBS volume encryption issue

pangrazi — Wed, 20 May 2020 12:35:16 GMT

How do you build a system without encrypted storage?

Re: NGFWv-AWS - EBS volume encryption issue

michaelkane2 — Sun, 28 Jun 2020 16:16:27 GMT

Hey there @jimholla, been a few weeks since your last message, but I still see 6.6.0-90 as the latest version for Cisco Firepower NGFW Virtual (NGFWv) - BYOL and Cisco Firepower NGFW Virtual (NGFWv) in the AWS Marketplace. Can you please give us an update as to when the fix might be released, approximately?

Thanks!

Re: NGFWv-AWS - EBS volume encryption issue

jimholla — Tue, 30 Jun 2020 19:00:12 GMT

Howdy,
We've started the process of republishing AMIs to the marketplace that resolve the encrypted storage issue. The first one available is 6.4.0-113. More will be coming soon.
Jim

Re: NGFWv-AWS - EBS volume encryption issue

StephenFyffe5557 — Wed, 26 Aug 2020 16:37:28 GMT

Has this problem been corrected? It seem I am running into the same issue:

Re: NGFWv-AWS - EBS volume encryption issue

pangrazi — Wed, 26 Aug 2020 17:14:02 GMT

Not that I have seen.

I had to disable encryption to get it to load.

Re: NGFWv-AWS - EBS volume encryption issue

varunes10 — Wed, 26 Aug 2020 17:19:33 GMT

Same here. Still waiting on an update from @jimholla on the fix.

Re: NGFWv-AWS - EBS volume encryption issue

jimholla — Wed, 26 Aug 2020 17:58:01 GMT

At this point, it has only been fixed with the 6.4.0-113 image that is in the marketplace. Other versions will be fixed but we are gated by another issue. I do not have an ETA.

Re: NGFWv-AWS - EBS volume encryption issue

ivan.auger — Thu, 07 Jul 2022 20:34:12 GMT

Has there been any progress on this? Issue also occurs with 7.0.2-88

Re: NGFWv-AWS - EBS volume encryption issue

iamjeroenris — Thu, 13 Oct 2022 11:47:02 GMT

I do not see very helpfull feedback on this topic! Is there no solution for this issue?

I am expiriencing the same problem at the moment for the Cisco Secure Firewall Threat Defense Virtual - BYOL AMI in AWS.