Missing CDR records in CAR Call Manager 7.1.3

[Richard McLoughlin]

Hi

I'm investigating missing CDR records from both CAR and the billing server for test calls that I initiated well within the preservation date. I think I've discovered and remedied the main issue which was that the CDR Agent service had been stopped on the subscriber that my phone was registered to when the test calls were made.

Having started the CDR agent on the subscriber, the CDRs are still missing. The outputs from the commands "file list activelog /cm/cdr_repository/preserve/20150519" and "file list activelog /cm/cdr_repository/processed/20150519" clearly show that CDR files are being processed.

I've studied these excellent documents which have been invaluable in understanding the CDR process but I've got a couple more questions.

https://supportforums.cisco.com/document/53056/understanding-cdr-call-detail-records#comment-10510616 and

https://supportforums.cisco.com/document/55821/troubleshooting-cdr

1. Am I correct to assume that if the CDR Agent service isn't running on a subscriber, this in itself doesn't prevent the generation of CDR flat files, it just means that they don't get transferred to the CDR repository on the publisher (the role of the CDR Agent)?  If this is the case, where are the files stored on the subscriber and if  I enable the CDR agent on the subscriber, should these files then transfer to the CDR repository assuming high water marks, preservation dates etc  have not been exceeded?

2. If the CAR Scheduler and  CDR Repository Manager services don't really perform any function other than on the Publisher, is it best practise to stop these services on the subscribers or is is best to leave them running?

As always - thanks in advance for your valued advice

Richard McLoughlin

[Ronak Agarwal]

Hi Richard,

1) The CDR Agent collects the CDR/CMR flat files that the Cisco Unified Communications Manager generates and sends these files to the publisher through SFTP.

Yes, you are right. CDR agent generates flat files for SUB but it does not transfer them to the PUB.
If you enable CDR agent, then yes it should push it to the PUB.

Also, try running in the command run sql select MANUAL_PURGE_STATUS from car:tbl_system_preferences in the sub, it should give error.

2) CAR Scheduler and  CDR Repository Manager services are not there in SUB. It is only present in pub.

Regards,

Ronak Agarwal

[Richard McLoughlin]

Hi Ronak

Thanks for your response. Can I just clarify something please?Document https://supportforums.cisco.com/document/53056/understanding-cdr-call-detail-records#comment-10510616 states under "Service Interaction"

• "CallManager service on the Subscriber generates CDR/CMR flat files locally.
• CDR Agent service on the Subscriber transfers these files to the cdr_repository directory structure on the Publisher. Once the transfer is complete, the local copy of these files on the Subscriber is deleted".

If I have understood you correctly, in your second paragraph you write "CDR agent generates flat files for SUB but it does not transfer them to the PUB". Did you mean "the CallManager service generates flat files for SUB but it does not transfer them to the PUB" or is the support document referenced above incorrect?

As per your suggestion, I've run the command "run sql select MANUAL_PURGE_STATUS from car:tbl_system_preferences" on the problematic subscriber and I get the following response back
"Database not found or no system permission". Incidentally, I get the same response from any ofthe other 3 subscribers as well (we have 4 in total).

Finally you write "CAR Scheduler and  CDR Repository Manager services are not there in SUB. It is only present in pub". So are the buttons displayed under Tools - Control Centre - Network Services just cosmetic because Cisco CDR Agent and Cisco CAR Scheduler are listed along with the ability to Start, Stop and Restart just as on the Publisher. Screen shot from the subscriber attached for your reference.

Thanks for your advice so far.

Best

Richard

[Ronak Agarwal]

Richard,

1) Until and unless you won't have CCM service started on the node, CDR agent won't have much role to play.
CCM service will help the node to be a part of call processing server.

If you have the CCM service deactivated, CDR agent service state won't really matter. Hope I make myself clear.

CCM service - helps the node to process calls.
CDR agent - will help in transferring the generated files to the PUB.

2) I beleive CDR scheduler and CDR repository manager "is there" in Subscriber for CUCM 7.x. However, It runs as a Network Service on all nodes in a cluster. But, in reality only the CDR Repository Manager on the Publisher performs any actions. On all other nodes, the service simply starts up, but then goes to sleep.

See more at: https://supportforums.cisco.com/document/53056/understanding-cdr-call-detail-records#CDR_Repository_Manager

3) CAR scheduler and CAR repository is not there in Subscribers post CUCM 9.x.

Post for any questions.

[Richard McLoughlin]

Hi Ronak

I think we are talking at crossed purposes here.

Re point 1 - agreed, but I don't understand why you've included it because at no point have I indicated that the CCM service has not started or is not running. 

The point I was trying to clarify is exactly which service generates the flat file CDR files on the subscriber. I believe it is the CCM Service which generates these files as part of its call processing and not as you originally stated in your first post  "CDR agent generates flat files for SUB but it does not transfer them to the PUB."

The document you reference is the same document I referenced in my opening question. The reason I raised the question about if services should be running was that DRF services are also "network services that run on all nodes. However if you leave the DRF Master running on anything other than the Publisher, there is the potential for conflict over who has overall control which can impact on the integrity of your backups - which we have experienced ourselves in the past. 

I had applied the same logic to the CDR services CAR Scheduler CDR Repository Manager - if they don't need to be running, should they be running on anything other than the publisher. Incidentally,

I've just found out  that even if I do stop the CAR scheduler on my subscribers and TFTP server, and then make a change to the CAR schedule on the publisher, it restarts the CAR scheduler across all nodes and not just the publisher - well it does in 7.1.3.22000-2

Thanks for the clarification over the versions of Call Manager and the services they do and don't have

[Ronak Agarwal · Accepted solution]

Richard,

Let me correct myself. The document is correct. Inside the Cisco Unified Communications Manager, the "Call Control process" generates CDR records.
So in SUB, if ccm service is active, the call control process of the ccm service will generate CDR.

Like I said, CDR repository and Scheduler is active only on PUB.

Also, in the latter releases (post 9.x) of the cucm, SUB only has DRF local. So DRF master is only on the PUB.

I think developers would have noticed issues in 7.x regarding these services on subscribers and hence moving forward they removed it from the sub's to avoid issues and confusion.

Regards,
Ronak Agarwal

[Richard McLoughlin]

Hello Ronak

Thanks very much for taking the time to clarify further. Can I trouble you just one more time for your thoughts on another curiosity that I've come across.

I was trying to see if I could determine where the flat files reside on the subscribers prior to their transfer to the publisher and was under the impression the the CAR related directory structure existed only on the publisher

If I run a "admin:file list activelog /cm" on any of the 4 subscribers, I see a directory structure similar to that found on the publisher.

admin:file list activelog /cm
<dir>   bin
<dir>   cdr
<dir>   cdr_repository
<dir>   log
<dir>   phoneguide_locales
<dir>   report
<dir>   tftpdata
<dir>   trace
dir count = 8, file count = 0

Also I note that sub directories "cdr_repository/processed" and "cdr_repository/preserved" have subdirectories dated from 24/09/2014 to 25/10/2014.

However, if I try to query the contents of a processed folder, e.g "file list activelog /cm/cdr_repository/processed/20140924" I get returned "dir count = 0, file count = 0 " and if I try to query the contents of a preserved folder,  e.g "file list activelog /cm/cdr_repository/preserved/20140924", I get returned "no such file or directory can be found".

The only event that I can attribute this to is that we performed a full cluster wide DRF restore on the 24/10/2014. Should I be worried by these directories and can I safely delete them?

Complete file list below.

admin:file list activelog /cm/cdr_repository/preserve/*
<dir>   20140924
<dir>   20140925
<dir>   20140926
<dir>   20140927
<dir>   20140928
<dir>   20140929
<dir>   20140930
<dir>   20141001
<dir>   20141002
<dir>   20141003
<dir>   20141004
<dir>   20141005
<dir>   20141006
<dir>   20141007
<dir>   20141008
<dir>   20141009
<dir>   20141010
<dir>   20141011
<dir>   20141012
<dir>   20141013
<dir>   20141014
<dir>   20141015
<dir>   20141016
<dir>   20141017
<dir>   20141018
<dir>   20141019
<dir>   20141020
<dir>   20141021
<dir>   20141022
<dir>   20141023
<dir>   20141024
<dir>   20141025
dir count = 32, file count = 0
admin:file list activelog /cm/cdr_repository/processed/*
<dir>   20140924
<dir>   20140925
<dir>   20140926
<dir>   20140927
<dir>   20140928
<dir>   20140929
<dir>   20140930
<dir>   20141001
<dir>   20141002
<dir>   20141003
<dir>   20141004
<dir>   20141005
<dir>   20141006
<dir>   20141007
<dir>   20141008
<dir>   20141009
<dir>   20141010
<dir>   20141011
<dir>   20141012
<dir>   20141013
<dir>   20141014
<dir>   20141015
<dir>   20141016
<dir>   20141017
<dir>   20141018
<dir>   20141019
<dir>   20141020
<dir>   20141021
<dir>   20141022
<dir>   20141023
<dir>   20141024
<dir>   20141025
dir count = 32, file count = 0

Thanks once again.

Richard

[Charles Hill]

Hey Richard,

I know this was referenced in the links you posted, but I'll throw this out just in case.

Verify the CDR flag is set to true on all servers in cluster.

Hope this helps.

[Richard McLoughlin]

Hi Charles

Thanks for you suggestion. That was among the first things that I checked, even to the point of toggling it off and then on again just to reinforce the setting on the problematic subscriber.

If you've got any other suggestions, I'd be pleased to hear/read them.

All the best

Richard

[Phoenix3]

Even on the TFTP we enable CDR Flag?