It's a small installation with less than 100 devices
When i run following commands the result showing the following:
ade # df -h
Filesystem Size Used Avail Use% Mounted on
3.9G 324M 3.4G 9% /
tmpfs 32G 11G 21G 34% /dev/shm
93M 5.6M 83M 7% /altroot
/dev/sda1 485M 48M 412M 11% /boot
93M 6.2M 83M 7% /home
244G 188M 231G 1% /localdisk
1.4T 1.3T 6.6G 100% /opt
93M 5.6M 83M 7% /recovery
/dev/sda2 97M 5.6M 87M 7% /storedconfig
9.5G 151M 8.9G 2% /storeddata
2.0G 36M 1.8G 2% /tmp
6.8G 967M 5.5G 15% /usr
3.9G 194M 3.5G 6% /var
ade # df /opt
Filesystem 1K-blocks Used Available Use% Mounted on
1424876728 1345588408 6908724 100% /opt
ade # du -h --max-depth=6 /opt | grep [0-9]G | sort -k2
du: cannot read directory `/opt/lost+found': Permission denied
du: cannot read directory `/opt/CSCOlumos/rcmds': Permission denied
du: cannot read directory `/opt/oracle/base/fast_recovery_area/WCS/archivelog/2018_04_12': Permission denied
What files can i remove or minimize to free up more space?
I have tried ncs cleanup this solved the problem temporarily.
There is nothing you can do to bring the DB to a normal proportion.
Do ncs stop.
Now you can add a disk!! Do not make any disk bigger, it will not work.
reboot the system
The OS will see the new disk space on reboot and add it to the partitions.
I would suggest to do an export of everything you can export in PI, devices, maps, groups, etc.
Then document everything you cannot export.
Then deploy the OVA again and import.
Then check the data retention settings and reduce them to the minimum acceptable for you.
Then make sure you get an alarm from PI when disk space usage on PI goes over the limit.
A backup also requires space in /opt even if it is 'remote'
You will have to add a disk, so you can make the system work for a short time, so you can export things.
Or you can try to deploy 3.4 or 3.3 and restore the last backup.
But also then I would do an export of everything possible, deploy the ova again and reimport.
Thanks for reply!
Ok, when you say export maps and data.
Whats the best way to do that? I'll have backup running to external ftp
Should i'll reinstall to 3.4 and then apply the backup from 3.1?
The setting of a threshold bigger or equal to 100% is asking for troubles.
This is manually set by someone.
I believe out-of-the-box it is 40%
After some deepdiving in the filesystem i found the source of problem.
In this directory below there was thousands of directories named incdir_XXXXX
and they contained very big .trc and .trm files.
After contact with TAC the solution was to remove files, and now i got more than 1TB empty space
This may be some useful info to someone.
I don't now why the alarm threshold is 90% but i'm pretty sure that's nothing we have changed since installation. Checked an other installation and there was also 90%?!
But now it's changed to 65%
I deleted the directories with files within.
Thanks a lot for your information.
Do you delete directories or files inside directories?