CUCM - How to Lower Disk Usage on Partitions

[Matthew Martin]

Hello All,

CallManager: 10.5(2)

After upgrading our CUCM servers from 8.6 to 10.5 the active partition's disk usage has been hovering just shy of 90%. I thought I remember there being a way to lower the amount of usage on the partition, by possibly lowering the high/low watermark (*in RTMT?)..?

Is there anyway to lower this number so the Active Partition doesn't stay at about 90% full at all times?

Any thoughts or suggestions would be greatly appreciated!

Thanks in Advance,
Matt

[Brandon Buffin]

Matt,

Here are a couple of ways to free up space.

Retrieve Core Files Using RTMT

• Use RTMT to retrieve any core files that may have been in the system:

1. Log in to RTMP and double-click Collect Crash Dump.
2. From the pop-up window, select Cisco Call Manager. Then click Next.
3. Click Next twice.
4. From the Collection Time portion of the pop-up window, select the radio button for Relative Range, and specify the time in minutes, hours, days, weeks and months.
5. On the Download File Options, select whether you want to zip the file. The Download File Location points to the default location. You can change the location by clicking the Browse button. Regardless of whether you zip the file, a pop-up window displays a warning message.
6. Wait for RTMT to retrieve any core files.

You can use the CLI to determine if the system has any core files on the active or inactive partitions. The system provides a set of commands to display and delete any existing core files, as follows:

1. Log in to the Unified CM CLI as administrator.
2. From the CLI, you can display any existing core files. Select the active/inactive partition where you want to display the core files by using the utils core active list or file list activelog core commands.
3. Delete the specific core file by using the command: file delete activelog core <core file name>
4. Check for available disk space.

Purge Log Files by Changing the Log Partition Watermarks

• Another way to create additional disk space is by changing the high and low watermarks on the system. This informs Unified CM of the numbers of log files to purge once the watermark is reached. Use RTMT as follows:

1. Launch RTMT and log in to the desired cluster.
2. From the left pane, select Alert Central.
3. On the right pane, double-click LogPartitionHighWaterMarkExceeded. Change the threshold value to 60.
4. On the right pane, double-click LogPartitionLowWaterMarkExceeded. Change the threshold value to 50.
5. This data is polled every five minutes. Allow five to 10 minutes and then check the drive partitions for additional disk space by using one of the methods described above.

If you want to free up more disk space in the common partition, change the LogPartitionHighWaterMarkExceeded and LogPartitionLowWaterMarkExceeded threshold values to lower numbers (for example, 30/20).

http://docwiki.cisco.com/wiki/Unified_CM_L2_Upgrade_Disk_Space_issues#Purge_Log_Files_by_Changing_the_Log_Partition_Watermarks

You can also use the free common space COP file:

https://software.cisco.com/download/release.html?mdfid=285963825&flowid=77896&softwareid=282204704&release=COP-Files&relind=AVAILABLE&rellifecycle=&reltype=latest

Hope this helps.

Brandon

[Matthew Martin]

Hey Brandon, thanks for the reply! Excellent detail, much appreciated!

I'll give that a shot, thinking I'll do the High/Low Watermark steps since that sounds like it will keep up on the usage automatically once it's set.

Thanks Again,

Matt

[Manish Gogna]

Hi Matt,

In addition to the great tips from Brandon you may consider the following points:

• Look for the following to find whats contributing to the disk usage:
•  
  • Check if there are a lot of Core dump files generated
  • Check if the Troubleshooting Trace Settings are turned on from the Cisco Unified Serviceability interface; if yes, please turn it off.
  • Check all service trace settings not using Detailed level. If Detailed settings, please change service trace settings to the default Significant level
  • If the remote access to the machine is enabled, please run the cmd "du -H" under the /common and get the command output.
• Check if the LPM service is up and running. Collect the LPM log from the RTMT Trace and Log Central.

http://docwiki.cisco.com/wiki/Troubleshooting_DiskSpace_and_Partition_related_alerts

Manish

[Matthew Martin]

Hey Manish, thanks for the reply!

Where would the core dump files you mentioned be located?

Also, I found these cdr files in the dir below. Are these the source files for when you run a CDR Reports?

admin:file list activelog cm/cdr_repository
<dir> car
<dir> destination1
<dir> destination2
<dir> destination3
<dir> preserve
<dir> processed
<dir> tmp
<dir> trans
dir count = 8, file count = 0
admin:

admin:file list activelog cm/cdr_repository/preserve
<dir> 20160827
<dir> 20160828
<dir> 20160829
<dir> 20160830
<dir> 20160831
<dir> 20160901
<dir> 20160902
<dir> 20160903
<dir> 20160904
<dir> 20160905
<dir> 20160906
<dir> 20160907
<dir> 20160908
<dir> 20160909
<dir> 20160910
<dir> 20160911
<dir> 20160912
<dir> 20160913
<dir> 20160914
<dir> 20160915
<dir> 20160916
<dir> 20160917
<dir> 20160918
<dir> 20160919
<dir> 20160920
<dir> 20160921
<dir> 20160922
<dir> 20160923
<dir> 20160924
<dir> 20160925
<dir> 20160926
<dir> 20160927
dir count = 32, file count = 0

Each of the directories in "cm/cdr_repository/*", except tmp and trans all contain the same directories as listed above. So it looks like a month's worth of CDR Data is stored on CUCM... Does that sound correct?

Thanks Again,
Matt

[Manish Gogna]

Hi Matt,

To check the core files you need to use the following command

utils core active list

regarding your second query, once CDR Agent on the node running the CallManager service sends the files to the Publisher, they are stored in the 'preserve/yyyymmdd' directory on the Publisher. If CAR is activated, symbolic links will be created to these files in the 'car/yyyymmdd' location. If billing servers are configured, symbolic links to these files will also be created in the 'destinationX/yyyymmdd' location (where X can be 1,2,3 --- since at most 3 billing servers can be configured).

The files will remain in the 'preserve' location until CDR Loader processes these files and enters corresponding records into the CDR database. Once the CDR files are processed, they are moved to the 'processed' location.

In addition to CDR Loading being enabled, if Billing Servers are configured, the CDR files will continue to remain in the 'preserve' location until both operations are completed -- i.e. until both CDR Loading and transfering file to the billing server are completed.

Hence, in typical working scenarios, all the CDR flat files will in the 'processed' location as opposed to the 'preserve' location. This signifies that all operations for those files have been completed.

https://supportforums.cisco.com/document/53056/understanding-cdr-call-detail-records

Manish

[Yaniv Malachi]

Hi Brandon / Manish

regarding the partition usage

the logging partition we can reduce it usage by delete log/trace file

but how can we reduce the usage of the active partition only ?

the core file are stored on the active partition ? what else can we remove

for reduce the active partition ?

Regards

Yaniv

[Deepak Mehta]

+5 for sharing such good info.

[orlandoguy82]

I have 2 BE6K clusters with the 1000 user OVAs and both sit at 95% active partition being used. I did the processes described above; lowering the low and high watermark and finding and deleting all the core dumps I found (there were quite a few do to some work I was aware of) and neither of the operations budges the active partition usage down at all.

[Jan Slabber]

Hi there. Did you manage to get this sorted because I'm sitting with the same issue. Thanks

[goranpilat]

Hi!

Did you manage to reduce active partition? I am trying to install localization and it keeps refusing for what I suspect is the lack of space on active partition

tnx a lot

[Jan Slabber]

Hello,

No unfortunately not. I have tried everything possible so I will have to log a TAC. When they finish I will post the remedy here.

[Jan Slabber]

Hi Brandon,

I can't find the COP file referred to in the link, which one is it please?

Thanks!

[ramalsubramanian]

Hi Jan,

Cop file ciscocm.free_common_space_v1.9.k4.cop.sha512 found in below link

https://software.cisco.com/download/home/286306100/type/282204704/release/COP-Files 

[Jan Slabber]

Hi and thank you for the reply.

Is this file also for System version: 10.5.2.16901-1 ?