CDR Database Error - Error Code 10012 - CUCM 8.6.2

[mischm]

Hi,

we are running CUCM 8.6.2 and I get at CAR webpage the following error message and CDR data also are not written to SFTP server since 2 weeks.

Before everything was working fine. 2 weeks ago we changed the IP-address of CUCM publisher and subscriber.

I already tried these steps but it`s not working and I still get the same error:

>>>Please verify also the status for the following services:

"Cisco SOAP - CDRonDemand Service" = active on the publisher server

"Cisco CDR Repository Manager" = active on the publisher server

"Cisco CDR Agent" = active on all the callmanager servers in your cluster.

"CAR Schedular" = active on publisher server only.

If still the same then, please try the following:

1. Disable the "CDR Enabled Flag" parameter in all the servers in your cluster.

2. Restart the "Cisco CDR Agent" service in all the servers in your cluster.

3. Restart the "Cisco CDR Repository Manager" service in the publisher server.

4. Restart the "Cisco Database Layer Monitor" service in all the servers in your cluster.

5. Enable again the "CDR Enabled Flag" parameter in all the servers in your cluster.

Can anybody help me in this issue to make CDR working again? Thanks.

BR
Michael

[Carlo Poggiarelli]

Hi Micheal

Please give a look to the following page

http://www.cisco.com/en/US/products/sw/voicesw/ps556/products_tech_note09186a0080933fcf.shtml#error2

HTH

Regards

Carlo

Please rate all helpful posts

"The more you help the more you learn"

[mischm]

Hi Carlo,

I had already this morning before this thread a look to the link you wrote.

But everytime I run actually a SQL query against CAR database I get the following error:

Failed to get Connection.Connector(NOT CONNECTED: Driver=com.informix.jdbc.IfxDriver;)

BR

Michael

[mischm]

Hi,

at CDR webpage I also get the error Cisco CAR DB serice is not running.

Try to start the Cisco CAR DB service from CLI:

Service Manager is running

Cisco CAR DB[STARTING]

Cisco CAR DB[STARTING]

Cisco CAR DB[STARTING]

Cisco CAR DB[STARTING]

Component is not running

Cisco CAR DB[NOTRUNNING]

Would a restart of the publisher maybe help to solve the issue?

BR

Michael

[asandborgh]

We have the same issue after IP address changes - did you ever find a fix?

Many thanks!

Art

[Bryan Geoghan]

Art, did you find a fix?

[jecker]

Did anyone find a fix for this?

I am having a similar issue after upgrade from 8.6.2 to 10.5.2 and configuring a domain name.

Thanks. 

[rossporubski]

Does it happen all the time?  I do get this error, but its only if I try to run a report against a different DID or extension immediately following a successful report. Essentially, I can log into CDR, run a report then I have to log out and log back in to do another, and so on ....and so on.... Otherwise I will get the error page every time I try to do multiple reports on unique numbers.

[jecker]

I get it everytime when i'm searching for a particular extension.

A CDR dump does seem to work just fine and has all the correct data. 

[jecker]

I was able to resolve the issue by doing the following:

1-Disable "CDR Enabled flag" Service Parameter on each server individually:                        

2-Reset these services (Individually on each server as well)        

        a. Cisco CAR Scheduler.

        b. Cisco CAR Web Agent.

       c. Cisco CDR Agent.

       d. Cisco CDR Repository Manager 

3 -Re-enable "CDR Enabled flag" Service Parameter on each server individually again.

Thanks.

[igor.lunavictoria]

It works! Many thanks. I have the same problem in CUCM 10.5.

Note: This procedure does not affect the service.

[christoph.hable]

Same Error after upgrade from 11.5SU7 to 12.5SU4 - worked  after restarting the services. I didn't dis- and re-enabled the CDR Enabled flag service parameter. 

Found following error in the CAR Scheduler logs. Error is not visible in newer logs after services restart.

admin:file search activelog /cm/trace/carsch/log4j/carsch00001.log ERROR

Searching path: /var/log/active//cm/trace/carsch/log4j/carsch00001.log
Searching file: /var/log/active//cm/trace/carsch/log4j/carsch00001.log
2021-05-26 19:45:27,361 ERROR [main] dbl.ConnectionManager$ConnectionManagerEntry - com.cisco.ccm.dbl.ConnectionManager$ConnectionManagerEntry@13ed2f0
2021-05-26 19:45:27,367 ERROR [main] dbl.Connector - connect() SQLException
2021-05-26 19:45:27,369 ERROR [main] dbl.Connector - handleConnectionException: ErrorCode is -27002
2021-05-26 19:45:27,369 ERROR [main] dbl.Connector - handleConnectionException: SQLstate is IX000
2021-05-26 19:45:27,369 ERROR [main] dbl.Connector - General failure:
2021-05-26 19:45:27,389 ERROR [main] beans.EBSystemPreferences - loadCurrentData(): Error while loading Sys Pref data
2021-05-26 19:45:27,390 ERROR [main] database.SchemaCreator - error while setting the manual purge status to 0
2021-05-26 20:02:46,409 INFO [main] services.Scheduler - Param Name - MAX_ERROR_RECORD_ID -> Param Value - 1
2021-05-26 20:20:01,538 INFO [Migration] migration.TblSystemPreferences - Update_MaxRecordId_In_SystemPreference(): MAX_ERROR_RECORD_ID found in inactive partition as 9589470, update to 1
2021-05-26 20:20:01,539 INFO [Migration] migration.TblSystemPreferences - Update_MaxRecordId_In_SystemPreference(): New MAX_ERROR_RECORD_ID = 1 (0 + 0 + 0 + 1)
2021-05-26 20:20:01,804 ERROR [main] services.Scheduler - runTasks(): Sleep InterruptedException caught while running the task [PopulateSchedules]. Will continue with next task
2021-05-26 20:21:58,127 ERROR [DailyCdrLoad] controller.CtrLoader - CheckPartitionData(): Partition part_invalid Contains 415 records
2021-05-26 20:23:48,139 ERROR [main] services.Scheduler - runTasks(): Sleep InterruptedException caught while running the task [PopulateSchedules]. Will continue with next task
2021-05-26 20:55:26,275 ERROR [ShutdownHandlerThread] services.Scheduler - marker file not found /common/cm/db/car/migration/marker (No such file or directory)

Search completed

[276578]

we restart the publisher and it works fine

admin:utils service list

Requesting service status, please wait...
System SSH [STARTED]
Cluster Manager [STARTED]
Service Manager is running
Getting list of all services
>> Return code = 0
A Cisco DB[STARTED]
A Cisco DB Replicator[STARTED]
Cisco AMC Service[STARTED]
Cisco AXL Web Service[STARTED]
Cisco Audit Event Service[STARTED]
Cisco Bulk Provisioning Service[STARTED]
Cisco CAR DB[STARTED]
Cisco CAR Scheduler[STARTED]
Cisco CAR Web Service[STARTED]
Cisco CDP[STARTED]
Cisco CDP Agent[STARTED]
Cisco CDR Agent[STARTED]
Cisco CDR Repository Manager[STARTED]
Cisco CTIManager[STARTED]
Cisco CTL Provider[STARTED]
Cisco CallManager Admin[STARTED]
Cisco CallManager Cisco IP Phone Services[STARTED]
Cisco CallManager Personal Directory[STARTED]
Cisco CallManager SNMP Service[STARTED]
Cisco CallManager Serviceability[STARTED]
Cisco CallManager Serviceability RTMT[STARTED]
Cisco Certificate Authority Proxy Function[STARTED]
Cisco Certificate Change Notification[STARTED]
Cisco Certificate Expiry Monitor[STARTED]
Cisco Change Credential Application[STARTED]
Cisco DRF Local[STARTED]
Cisco DRF Master[STARTED]
Cisco Database Layer Monitor[STARTED]
Cisco Dialed Number Analyzer[STARTED]
Cisco Dialed Number Analyzer Server[STARTED]
Cisco DirSync[STARTED]
Cisco E911[STARTED]
Cisco ELM Admin[STARTED]
Cisco ELM Client Service[STARTED]
Cisco ELM DB[STARTED]
Cisco ELM Server[STARTED]
Cisco Extension Mobility[STARTED]
Cisco Extension Mobility Application[STARTED]
Cisco License Manager[STARTED]
Cisco Location Bandwidth Manager[STARTED]
Cisco Log Partition Monitoring Tool[STARTED]
Cisco RIS Data Collector[STARTED]
Cisco RTMT Reporter Servlet[STARTED]
Cisco SOAP - CDRonDemand Service[STARTED]
Cisco SOAP - CallRecord Service[STARTED]
Cisco Serviceability Reporter[STARTED]
Cisco Syslog Agent[STARTED]
Cisco Tomcat[STARTED]
Cisco Tomcat Stats Servlet[STARTED]
Cisco Trace Collection Service[STARTED]
Cisco Trace Collection Servlet[STARTED]
Cisco Trust Verification Service[STARTED]
Cisco Unified Mobile Voice Access Service[STARTED]
Cisco User Data Services[STARTED]
Cisco WebDialer Web Service[STARTED]
Host Resources Agent[STARTED]
MIB2 Agent[STARTED]
Platform Administrative Web Service[STARTED]
SNMP Master Agent[STARTED]
SOAP - Diagnostic Portal Database Service[STARTED]
SOAP -Log Collection APIs[STARTED]
SOAP -Performance Monitoring APIs[STARTED]
SOAP -Real-Time Service APIs[STARTED]
System Application Agent[STARTED]
Cisco CallManager[STOPPED] Service Not Activated
Cisco DHCP Monitor Service[STOPPED] Service Not Activated
Cisco ELM Resource API[STOPPED] Service Not Activated
Cisco Extended Functions[STOPPED] Service Not Activated
Cisco IP Manager Assistant[STOPPED] Service Not Activated
Cisco IP Voice Media Streaming App[STOPPED] Service Not Activated
Cisco Intercluster Lookup Service[STOPPED] Service Not Activated
Cisco Messaging Interface[STOPPED] Service Not Activated
Cisco TAPS Service[STOPPED] Service Not Activated
Cisco Tftp[STOPPED] Service Not Activated
Cisco UXL Web Service[STOPPED] Service Not Activated
Primary Node =true

[putney_m]

Have you been able to resolve this issue? I am running into this on a 9.1.1.20000-5 cluster after the IP of the publisher was changed.

Thanks,

Michael

[mischm]

Hi,

yes we solve the problem but not with any suggestions made before.

We reinstalled the CUCM Pub after we faced these problems and used restore option using the last vaild backup.

After new installation of CUCM Pub everything is working fine.

BR

Michael