Background: During NCS' short life, a number of customers had problems (NCS failing to start, backups failing, etc.) where exhausting disk space was at the root, but they had overlooked the gentle warnings that a critical alarm gave. Prime Infrastructure 1.2 has disk space messages that are highly intrusive, and were meant to be intrusive so customers wouldn't run out of space and not know it.
The threshold for these alarms is a hard set 50% utilization. At the most basic level, in order to copy out temp files to create a backup copy of the database, the /opt partition shouldn't be more than 50% utilized. Say, for example, one's /opt partition is 100GB, and their database is 75 GB, where is Prime Infrastructure supposed to be able to put 75 GB of temp files in order to create a backup? The 50% threshold is very important, particularly if NFS mounts are not being used to hold these temp files or the final backup.tar.gpg files the backup process creates.
In all installations, if the utilization level of the /opt partition (or /dev/mapper/smosvg-optvol in the Prime Infrastructure GUI under Administration > Appliance) is less than 60%, it's possible that the temporary archive log files from the Oracle database have accumulated and can be cleaned with the following commands at the Prime Infrastructure command line interface.
Prime Infrastructure release 1.3.0.20 resolved a number of backup/restore, migration and database issues. The subsequent 1.3.1 patch (filename PI_1.3.0.20_Update_1-12.tar.gz ) has fixed even more issues with backup/restore and reduced the filesize of the resulting backup.tar.gpg file that the backup process creates. Be sure to keep up to date with the latest bug fixes by upgrading to 1.3.0.20 and any subsequent patches, as soon as practicably possible.
ncs stop
ncs cleanup
ncs start
NOTE: It's critical that Prime Infrastructure be stopped BEFORE starting an ncs cleanup. In some installations that are on the border of alarming, the "ncs cleanup" command may clear enough space to stop the alarms.
In installations with a virtual appliance, adding additional disk space is straightforward: Stop Prime Infrastructure with the "halt" command, add a second virtual hard disk to the VM, then power the VM back on. On boot up, the file system table will be updated with the new disk, and Prime Infrastructure will detect it and add it to both the /opt and /localdisk partitions in an 85%/15% mix. On logging into the GUI after adding the additional virtual hard disk, the disk space alarms will be gone. Unfortunately, increasing the size of the existing virtual hard disk in the virtual machine is not an option, as Prime Infrastructure's disk grow utility will not see that; it is looking for an additional virtual hard disk to be added to the VM.
In installations where High Availability (HA) is configured, remove HA
Administration > High Availability > HA Configuration > Remove button
before adding additional disk space. The primary and secondary can receive the additional disk space separately and brought back online, then HA restored.
