DRF Backup Fail - CAR issue

[Victor Circelli]

Hello,

We have a customer with CUCM 8.0.2.40000-1

Since 3 days ago backups started to fail.

It fails on the first backup item, which is the CAR/CDR.

It stucks on this and never go ahead.

To solve this we need to restart DRF Master and Local agents but even after this, backup does not complete.

If we deactive CAR/CDR backup than it works

We notice CAR Scheduler service stopped.

How can I troubleshoot this? I read some bugs but didn't find anything that could be related to our problem.

Regards

[Aman Soi]

Hi Victor,

It is possible that CAR /CDR  size is huge which is stopping to take backup Are u using freeftpd sftp to take backup?

Any error message which comes .

regds,

aman

[Victor Circelli]

Hi Aman,

Unfortunatelly I don't know which software they're using since it's maintened by the customer.

Are there any way that I can monitor the size of CAR files? Or wipe the old files?

edit: No error messages. DRF backup status keeps stuck  50% on CAR (1st item).

Backup history says "ONGOING"

Regards

[Aman Soi]

Hi Victor,

I won't be able to help u in finding the CDR Size but u can refer the link used for high and low water mark,preservation days.

http://www.cisco.com/en/US/docs/voice_ip_comm/cucm/service/8_6_1/admin/sacdrm.html

Can u cross-check the settings in your call manager system with the default ones configured. in above link?

regds,

aman

[Victor Circelli]

Hi Aman,

Settings are the default one... 3000mb / 80% / 40% / 30 days

Do you have any other clue?

Regards

[Aman Soi]

Hi Victor,

the settings are OK.

Can u ask the customer to try with another sftp software or else open TAC case with cisco?

regds,

aman

[Aman Soi]

Hi Victor,

I was just looking into the bugs toolkit.

I think your problem is somewhat similar to mentioend in the bug.

CSCsl16967 Bug Details

DRS stuck in CCMDB backup if many CDR files in Preserved folder

Symptom:Callmanager 5.1.3 is able to backup the components to SFTP server successfully earlier, but as with the system running and the CDR records starting to build up, the backup start to fail at some stage.Condition: huge set of CDR/CMR files may accumulate in Preserve folders (e.g 110,000) caused a big backup.log file exceeded 1 MB(e.g 20MB)Typical Scneario: Customer does not want to use CAR but using third party billing server instead and CAR loader is disabled, in callmanager 5.1.3, this can result in large number of CDR/CMR accumulated in Preserve folder.

Workaround

:

In 5.1.3 and later

Stopping CAR service will not stop accumulating flat CDR Files.

To clean up the CDR files temporarily to that DRS can proceed,

1. stop the CDR Agent service on all servers in the cluster, so no new CDR files will be pushed to the publisher.

2. check to make sure all the files have been pushed to the billing server(s), do the following command

ls -R /var/log/active/cm/cdr_repository/destination*

to make sure there is no symbolic link in any of the subfolders

3. Stop CDR Repository Manager, CAR Scheduler and CAR Web Service on the publisher

4. remove all the files under /var/log/active/cm/cdr_repository/preserve/< date> that have been accumulated. And remove all the symbolic links under /var/log/active/cm/cdr_repository/car/< date> with the following commands,

rm -rf /var/log/active/cm/cdr_repository/preserve/*

rm -rf /var/log/active/cm/cdr_repository/car/*

5. restart CDR Repository Manager, CAR Scheduler, CAR Web Services on the publisher.

To stop further accumulation of CDR files, start CAR Scheduler service, set the loader to schedule to continuously loading, load CDR only.

1. create a ccmadmin account if not yet in user group management on ccmadmin page

2. log in to CAR, go to System->Scheduler->CDR Load,

3. check "Continuous Loading 24/7" and "Load CDR only" boxes

4. click "update"

5. goto System->Database->Configure Automatic Database Purge,

6. set both "Min Age of Call Detail Records" and "Max Age of Call Detail Records' to 1

7. click "update"

8. go to Report Config->Automatic Generation/Alert

9. For each report, select "Disabled" status and click "Update"

Finally, restart CDR Agent service on all the servers.

workaround

for DRS :

1. Restart DRS Master Agent and Local Agent.

2. Delete the backup.log from location /common/drf

regds,

aman

[Victor Circelli]

Hi Aman,

I opened a PDI case and they told me that since CUCM 5.1 the CAR backup is bundled with the CCMDB component, so there's no need to select CDR/CAR on DRS backup.

Thank you and best regards!

[Aman Soi]

Hi Victor,

Did u resolve the issue?

regds,

aman

[Victor Circelli]

Aman,

Actually they suggested an workaround, since the CDR/CAR component (which failed) is bundled with CCMDB, so it's not necessary to backup CDR/CAR component