Module Disk Usage: High unmanaged disk usage on /Volume.

[Enid Vallja]

Hi,

I have a critical notification on my Sourcefire - Health Monitor. 

It says "Module Disk Usage: High unmanaged disk usage on /Volume" even though the Sourcefire has a lot of free memory.

Please tell me what can I do with the "/Volume" directory to free some memory.

Best regards,

Enid.

[Dinesh Verma]

Hi Enid,

Mostly /var/tmp/ contains a lot of temporary files. So you might want to try this (This is general):

--> SSH to the module

--> Go to expert mode

--> Become root: sudo su

--> Go to: cd /var/tmp/

--> Remove temp stuff using this command: rm -rf Apply_*

Apply_<random> files are created when you deploy things from FMC to the module. Other place you might want to check is

/var/sf/updates/  (here you can delete all older upgrade files).

/var/common/ (delete old TS and core files)

Command: du -sh * (is useful to list  out which directories with size). Find out the directories with huge size and if there is unwanted backup or TS files, delete them.

This article is quite helpful too: http://www.cisco.com/c/en/us/support/docs/security/firesight-management-center/118719-technote-firesight-00.html 

Regards,

Dv

[Enid Vallja]

Hi Dv,

I don't think this has to do with the /Volume directory because I deleted yesterday some filesto free some memory but it still points the /Volume.

Regards,

Enid.

[Dinesh Verma]

Run the command: df -TH and provide me the output. We will see if it's false health alarm.

[Enid Vallja]

This is the output (memory.jpg), and please take a look to the alerts (alerts.jpg).

[Dinesh Verma]

/var (/Volume) looks fine to me. Just to double check that you gave me the output from device for which you got the health alarm for disk space. Please check.

If it's the same device then I would say it's a false alarm.You can go to health monitor and click on the device which has alarm and then click on Run All Module and wait for a while and see if alert goes away.

Being root on the device, you can try restarting diskmanager service too:

pmtool restartbyid diskmanager (User should be root).

On totally a different note, it's disk space we're talking here, not memory.

[Enid Vallja]

I clicked on Run All Modules and still the problem is like in the photo.

And I already did a restart yesterday but the notification won't go away.

[Jetsy Mathew]

Hello Enid,

We need to verify if there is any logs or old backups saved in the device. First of all we need to verify if this is a right health alert or if its a false alarm.

It would be great if you can open a Cisco TAC case so that we can investigate this further.

Regards

Jetsy 

[Dinesh Verma]

That's tricky. As jetsy said, open up a TAC case and we will help you fix this health alarm.

[Lee Win Neng]

Hi Dv,

Good day to you.

I had carried out the same procedure and commands as suggested by you and was working fine for me. Is there any bugs that causing this issue of exceed on temporary files on the module that causing "high unmanaged disk usage on /Volume".

Kindly need your assistance and advice for this issue, thank you.

[heritierday]

Hi DV,

It is working for me as i have removed all the old updates in /var/sf/updates/.

Thanks a lot.

[Barrett Cowan]

This is to build off Dv post, and just to be clear here, this is for a FirePower module, not FMC (although, there wouldn't be much difference). Other places you can look that has helped me as well

1. /var/sf/SRU/
   • Delete all Sourcefire_Rule_Updates but last couple
2. /var/sf/sru/ (notice the case difference)
   • Delete all vrt folders but last couple
3. /var/sf/updates/
   • Delete all unwanted updates
4. /var/sf/vdb/
   • Delete all VDB Fingerprint Database files but last couple
5. /var/common/
   • Delete everything
6. Everything can be deleted under the following directories
   • /var/sf/detection_engines/<some GUID>/backup/
   • /var/sf/detection_engines/<some GUID>/instance-1/backup/
   • /var/sf/detection_engines/<some GUID>/instance-2/backup/
7. If you’re feeling frisky, and have some time … /var/sf/upgrades_backup/
   • Delete all old backups

After deleting everything above, run the following command if you’re not able to just reboot the module

1. pmtool restartbyid diskmanager

I’ve experienced this issue on 6.0 – 6.2.2.x, I don’t think I’ve dealt with it since 6.2.3. I currently have 85 deployments of mixed FirePower & FTD (FTD has a LOOONG way to go .. BTW). Been through a lot of TAC cases and issues with FirePower.

[ArchiTech89]

Barrett has it goin' on. I work at Cisco on Firepower/NGFW devices (FTD) and this is as good a summary of locations for mitigating disk usage problems I've seen. Kudos to him!

[Barrett Cowan]

Thank you @ArchiTech89, much appreciated!

Adding to my previous post, just came by an issue today on v6.1.0.7, classic "High unmanaged disk usage on /Volume". Poking around the directories, I found a file in the /var/log directory named messages.1 that was 26G. Deleted it and rebooted, everything good now.

Just for context, I can reference many other deployments we have out there to compare. This allowed me to figure out how the messages file was too large/out of place.

[mansoorsherazi]

Thank a lot, I was scratching my head for the last 3 days. I was unable to upgrade my devices from fmc. your solution made my day.

Again Thanks and best of luck