04-12-2018 01:26 AM - edited 03-01-2019 06:31 PM
It's a small installation with less than 100 devices
When i run following commands the result showing the following:
ade # df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/smosvg-rootvol
3.9G 324M 3.4G 9% /
tmpfs 32G 11G 21G 34% /dev/shm
/dev/mapper/smosvg-altrootvol
93M 5.6M 83M 7% /altroot
/dev/sda1 485M 48M 412M 11% /boot
/dev/mapper/smosvg-home
93M 6.2M 83M 7% /home
/dev/mapper/smosvg-localdiskvol
244G 188M 231G 1% /localdisk
/dev/mapper/smosvg-optvol
1.4T 1.3T 6.6G 100% /opt
/dev/mapper/smosvg-recvol
93M 5.6M 83M 7% /recovery
/dev/sda2 97M 5.6M 87M 7% /storedconfig
/dev/mapper/smosvg-storeddatavol
9.5G 151M 8.9G 2% /storeddata
/dev/mapper/smosvg-tmpvol
2.0G 36M 1.8G 2% /tmp
/dev/mapper/smosvg-usrvol
6.8G 967M 5.5G 15% /usr
/dev/mapper/smosvg-varvol
3.9G 194M 3.5G 6% /var
ade #
ade # df /opt
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/smosvg-optvol
1424876728 1345588408 6908724 100% /opt
ade #
ade # du -h --max-depth=6 /opt | grep [0-9]G | sort -k2
du: cannot read directory `/opt/lost+found': Permission denied
du: cannot read directory `/opt/CSCOlumos/rcmds': Permission denied
du: cannot read directory `/opt/oracle/base/fast_recovery_area/WCS/archivelog/2018_04_12': Permission denied
12G /opt/CSCOlumos
1.5G /opt/CSCOlumos/logs
64K /opt/CSCOlumos/matlab/toolbox/matlab/imagesci/@H5G
2.9G /opt/CSCOlumos/staging
1.6G /opt/CSCOlumos/staging/app
3.1G /opt/CSCOlumos/updates
1.4G /opt/CSCOlumos/updates/PI_3_1_3-1.0.16
1.6G /opt/oracle/base/diag/tnslsnr
1.6G /opt/oracle/base/diag/tnslsnr/FMCL-PRIME1
1.6G /opt/oracle/base/diag/tnslsnr/FMCL-PRIME1/wcstns
3.3G /opt/oracle/base/fast_recovery_area
3.1G /opt/oracle/base/fast_recovery_area/WCS
3.1G /opt/oracle/base/fast_recovery_area/WCS/onlinelog
29G /opt/oracle/base/oradata
1.4G /opt/oracle/base/oradata/STBYWCS
1.2G /opt/oracle/base/oradata/STBYWCS/datafile
28G /opt/oracle/base/oradata/WCS
28G /opt/oracle/base/oradata/WCS/datafile
3.0G /opt/oracle/base/product
3.0G /opt/oracle/base/product/12.1.0
3.0G /opt/oracle/base/product/12.1.0/dbhome_1
1.2G /opt/oracle/base/product/12.1.0/dbhome_1/lib
ade #
What files can i remove or minimize to free up more space?
I have tried ncs cleanup this solved the problem temporarily.
04-12-2018 03:43 AM
There is nothing you can do to bring the DB to a normal proportion.
Do ncs stop.
Now you can add a disk!! Do not make any disk bigger, it will not work.
reboot the system
The OS will see the new disk space on reboot and add it to the partitions.
I would suggest to do an export of everything you can export in PI, devices, maps, groups, etc.
Then document everything you cannot export.
Then deploy the OVA again and import.
Then check the data retention settings and reduce them to the minimum acceptable for you.
Then make sure you get an alarm from PI when disk space usage on PI goes over the limit.
04-12-2018 11:54 PM
04-12-2018 04:12 AM
04-13-2018 12:00 AM
04-13-2018 01:25 AM
04-13-2018 02:17 AM
04-14-2018 12:58 PM - edited 04-15-2018 02:14 AM
A backup also requires space in /opt even if it is 'remote'
You will have to add a disk, so you can make the system work for a short time, so you can export things.
Or you can try to deploy 3.4 or 3.3 and restore the last backup.
But also then I would do an export of everything possible, deploy the ova again and reimport.
04-14-2018 03:44 PM
@danlin wrote:
Thanks for reply!
Ok, when you say export maps and data.
Whats the best way to do that? I'll have backup running to external ftp
Should i'll reinstall to 3.4 and then apply the backup from 3.1?
That's correct.
04-15-2018 11:55 PM
This is the graph showing disk space over time :)
04-16-2018 12:02 AM
The setting of a threshold bigger or equal to 100% is asking for troubles.
This is manually set by someone.
I believe out-of-the-box it is 40%
04-25-2018 06:41 AM
After some deepdiving in the filesystem i found the source of problem.
In this directory below there was thousands of directories named incdir_XXXXX
and they contained very big .trc and .trm files.
/opt/oracle/base/diag/rdbms/wcs/wcs/incident
After contact with TAC the solution was to remove files, and now i got more than 1TB empty space
This may be some useful info to someone.
I don't now why the alarm threshold is 90% but i'm pretty sure that's nothing we have changed since installation. Checked an other installation and there was also 90%?!
But now it's changed to 65%
07-31-2018 09:06 AM
Thanks a lot for your information.
Do you delete directories or files inside directories?
08-13-2018 05:40 AM
I deleted the directories with files within.
@antonio.dinapoli wrote:
Thanks a lot for your information.
Do you delete directories or files inside directories?
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide