Cisco Prime Infrastructure 3.1 High Disk Usage

[NeverOutofTune]

/dev/mapper/smosvg-optvol is growing about 1 to 1.5 GB per day.  I have scaled down all logging settings I could find.  I have one 5520 controller pair with 75 Access Points across 20 sites with 300 clients.  What can I do to reduce/stop logging.  I've tried ncs cleanup.

From PI CLI: ade # df -h

Filesystem	Size	Used	Avail	Use%
/dev/mapper/smosvg-rootvol	3.9G	296M	3.4G	8%
tmpfs	5.9G	1.9G	4.1G	32%
/dev/mapper/smosvg-altrootvol	124M	5.6M	113M	5%
/dev/sda1	485M	41M	419M	9%
/dev/mapper/smosvg-home	124M	6.2M	112M	6%
/dev/mapper/smosvg-localdiskvol	88G	7.6G	76G	10%
/dev/mapper/smosvg-optvol	454G	244G	188G	57%
/dev/mapper/smosvg-recvol	124M	5.6M	113M	5%
/dev/sda2	97M	5.6M	87M	7%
/dev/mapper/smosvg-storeddatavol	9.7G	151M	9.0G	2%
/dev/mapper/smosvg-tmpvol	2.0G	36M	1.9G	2%
/dev/mapper/smosvg-usrvol	6.8G	1.2G	5.3G	18%
/dev/mapper/smosvg-varvol	3.9G	183M	3.5G	5%

[iportuga]

Hi,
I recommend you to review the application log level settings and check which files or subdirectories are the root cause for filesystem consumption growth.

[NeverOutofTune]

iportuga,  thanks for the quick reply.  I believe I already did as you suggested.  The offending file is smosvg-optvol which is growing about 1G per day and I have reduced all logging to the minimum permitted.

/dev/mapper/smosvg-optvol

454G

244G

188G

57%

I do not see a reduction in the growth pattern and looking other suggestions.

[NeverOutofTune]

Using Prime 3.1

I was able to work with TAC and found growth of "incdir_*"  directories located at:

/opt/oracle/base/diag/rdbms/wcs/wcs/incident

To see:

ssh to Prime

# shell
ade # sudo su -
-bash-4.1#  cd /opt/oracle/base/diag/rdbms/wcs/wcs/incident
-bash-4.1#  ls -lrth

drwxrwxr-x. 2 oracle xmpdba 4.0K Apr 17 09:22 incdir_144884
drwxrwxr-x. 2 oracle xmpdba 4.0K Apr 17 11:24 incdir_144870
drwxrwxr-x. 2 oracle xmpdba 4.0K Apr 17 13:27 incdir_144793

:

drwxrwxr-x. 2 oracle xmpdba 4.0K May 1 02:18 incdir_145020
drwxrwxr-x. 2 oracle xmpdba 4.0K May 1 04:21 incdir_144642
drwxrwxr-x. 2 oracle xmpdba 4.0K May 1 06:23 incdir_145053

Temporary resolution is to delete all these folders:

-bash-4.1# rm -rf incdir*

I am working on upgrading to a newer version of Prime to see if this is a bug in 3.1.

[Nickolus Looper]

How long has the whole ncs cleanup process taken for most to run?

On PI 3.3.1, the process has been running for about 1.5 hours.

It's currently 23:11 EDT.

KMCI4NMS2001/admin# ncs cleanup
***************************************************************************
!!!!!!! WARNING !!!!!!!
***************************************************************************
The clean up can remove all files located in the backup staging directory.
Older log files will be removed and other types of older debug information
will be removed
***************************************************************************
Do you wish to continue? ([NO]/yes) yes

***************************************************************************
!!!!!!! DATABASE CLEANUP WARNING !!!!!!!
***************************************************************************
Cleaning up database will stop the server while the cleanup is performed.
The operation can take several minutes to complete
***************************************************************************
Do you wish to cleanup database? ([NO]/yes) yes

***************************************************************************
!!!!!!! USER LOCAL DISK WARNING !!!!!!!
***************************************************************************
Cleaning user local disk will remove all locally saved reports, locally
backed up device configurations. All files in the local FTP and TFTP
directories will be removed.
***************************************************************************
Do you wish to cleanup user local disk? ([NO]/yes) yes
===================================================
Starting Cleanup: Wed May 30 21:45:42 EDT 2018
===================================================
{Wed May 30 21:49:44 EDT 2018} Removing all files in backup staging directory
{Wed May 30 21:49:44 EDT 2018} Removing all Matlab core related files
{Wed May 30 21:49:44 EDT 2018} Removing all older log files
{Wed May 30 21:50:41 EDT 2018} Cleaning older archive logs
{Wed May 30 21:50:51 EDT 2018} Cleaning database backup and all archive logs
{Wed May 30 21:50:51 EDT 2018} Cleaning older database trace files
{Wed May 30 21:50:51 EDT 2018} Removing all user local disk files
{Wed May 30 21:55:41 EDT 2018} Cleaning database
{Wed May 30 21:55:44 EDT 2018} Stopping database
{Wed May 30 21:55:47 EDT 2018} Starting database
{Wed May 30 21:56:26 EDT 2018} Starting database clean