CUIC on a UCCX - Dataset status is Failed. (Database error)

[Kim Nielsen]

Hi,

Suddenly this Saturday, some of our rapports stopped working. Detailed call by call CCDR report is working, when you choose today, but when you choose yesterday, we are getting a "Dataset status is Failed (Database error).

Details on the error is:

Error information:

com.cisco.ccbu.cuic.businesslogic.datasource.CuicDbException: DbException: CuicDataSourceServiceManagerImpl.getDataSet() { Nested SQLException; SQLState: IX000 Vendor code: -1265 Message: Overflow occurred on a datetime or interval operation. } at com.cisco.ccbu.cuic.businesslogic.datasource.CuicDataSourceServiceManagerImpl.getDataSetBuilder(CuicDataSourceServiceManagerImpl.java:860) at com.cisco.ccbu.cuic.businesslogic.engine.CuicReportEngineWorker.executeQueryUsingDatasetBuilder(CuicReportEngineWorker.java:659) at com.cisco.ccbu.cuic.businesslogic.engine.CuicReportEngineWorker.runReport(CuicReportEngineWorker.java:503) at com.cisco.ccbu.cuic.businesslogic.engine.CuicReportEngineWorker.run(CuicReportEngineWorker.java:458) at com.cisco.ccbu.infra.threads.InstrumentedRunnable.run(InstrumentedRunnable.java:88) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) at java.lang.Thread.run(Thread.java:662) at com.cisco.ccbu.infra.threads.ThreadPoolThread.run(ThreadPoolThread.java:164) Caused by: java.sql.SQLException: Overflow occurred on a datetime or interval operation. at com.informix.util.IfxErrMsg.getSQLException(IfxErrMsg.java:407) at com.informix.jdbc.IfxSqli.a(IfxSqli.java:3461) at com.informix.jdbc.IfxSqli.E(IfxSqli.java:3774) at com.informix.jdbc.IfxSqli.dispatchMsg(IfxSqli.java:2580) at com.informix.jdbc.IfxSqli.receiveMessage(IfxSqli.java:2496) at com.informix.jdbc.IfxSqli.a(IfxSqli.java:1766) at com.informix.jdbc.IfxSqli.executeStatementQuery(IfxSqli.java:1704) at com.informix.jdbc.IfxSqli.executeStatementQuery(IfxSqli.java:1635) at com.informix.jdbc.IfxResultSet.a(IfxResultSet.java:206) at com.informix.jdbc.IfxStatement.executeQueryImpl(IfxStatement.java:1229) at com.informix.jdbc.IfxPreparedStatement.executeQuery(IfxPreparedStatement.java:395) at com.informix.jdbc.IfxCallableStatement.executeQuery(IfxCallableStatement.java:238) at com.mchange.v2.c3p0.impl.NewProxyCallableStatement.executeQuery(NewProxyCallableStatement.java:2131) at com.cisco.ccbu.cuic.businesslogic.datasource.CuicDataSourceServiceManagerImpl.executeAndLoadDataset(CuicDataSourceServiceManagerImpl.java:978) at com.cisco.ccbu.cuic.businesslogic.datasource.CuicDataSourceServiceManagerImpl.getDataSetBuilder(CuicDataSourceServiceManagerImpl.java:846) ... 8 more Caused by: java.sql.SQLException at com.informix.util.IfxErrMsg.getSQLException(IfxErrMsg.java:407) at com.informix.jdbc.IfxSqli.E(IfxSqli.java:3779) ... 20 more

I have tried to restart the CUIC service, but no change. It's still giving me this error.

I'm thinking that it's having something to do with the date/time, because of the "1265 Message: Overflow occurred on a datetime or interval operation" that is seen in the error.

But I can't seem to find any info on this error, other than it's some kind of date/time error in the Informix database.

Anyone else seen this error? (It's an 9.0(2) version of the uccx.)

Br

Kim

[arnayyar]

Hi Guys,

This issue has been reported to TAC already.

Latest update:

Soon a field notice will be released regarding this issue, follow that for further updates or I will post the confirmed resolution here.

[Daniel Gagne]

Hi everyone,

We are hitting the same issue with all 3 of our UCCX clusters.

We are running version 10.6.1.11001-31.

We can't run reports for last Friday which was 9/9/2016 and relative date set to 'today' on many reports.

I can't access the bug détails either, anyone had any luck on accessing the bug details?

I will open a TAC case for all of ou 3 UCCX clusters

[Grace.Maxim]

Same issue here, we are on UCCX 10.6.1.11001-31, can't run these reports below since 9/9/2016. I have opened a TAC case, but planning on re-start tonight, has the re-start worked for anyone else?

Agent Not Ready Reason Code Summary

Agent State Detail Report

Agent State Summary by Agent

Agent Summary Report

[drdillio2]

Restarting did not help in our environment.

[Richard Simmons]

Hi,

I'm seeing the same error when i look at the logs on our Wallboard Server - Overflow occurred on a datetime or interval operation - The Wallboard is from VSR2 and has been working fine since January.

I'm unable to see the BugID posted due to insufficient permissions - is there a work around for this or do i need to raise a TAC case?

Thanks,

Richard

[Richard Simmons]

Thanks - the  patch also fixed my issue with the Wallboard

Regards,

Richard

[arnayyar · Accepted solution]

Hi,

This is an outage. All customers are experiencing this issue. As a solution, download and apply the attached cop file. This is applicable from any version from 8.x to 11.x.

Please download the cop file for the bug fix and save it an sFTP server. Applying this cop will NOT cause any service impact and no reboot or switch-version is required.

Let me know if you need assistance applying this cop which is the solution to this issue.

Instructions: Please refer readme for more:

Save the cop.sgn to the root of the sFTP directory.
Run utils system upgrade initiate on both Nodes one by one incase of HA and just one Node incase of standalone.
Choose the cop file and let it install.
Close web browser and run CUIC and the reports should be working after this solution.

[Mike Iversen]

I applied the patch to our UCCX 9.0(2) cluster and issues are resolved. Thanks!

[svigano]

COP worked for us.  We're on Cisco Unified Intelligence Center
Version:10.6(1) build 1 (10_6_1_10000_39)

[kenobiwan]

Worked for us. Thanks, arnayyar.

[Sascha Monteiro]

it fails to install, the md5sum is correct of the file after unzip

Accessing the file.  Please wait...

Validating the file...

The selected file is not valid.

I am running: 

11.0.1.99000-102

edit:

09/13/2016 20:08:26 upgrade_get_file.sh|Create md5 "/common/download/ciscouccx.ReportFix.cop.sgn.md5"|<LVL::Info>

09/13/2016 20:08:26 upgrade_get_file.sh|MD5(/common/download/ciscouccx.ReportFix.cop.sgn)= 75:ef:b2:cb:08:08:12:c3:f9:61:5d:3c:77:03:f2:d1|<LVL::Debug>

09/13/2016 20:08:26 upgrade_get_file.sh|Create md5 complete|<LVL::Info>

09/13/2016 20:08:26 upgrade_get_file.sh|Authenticate file "/common/download/ciscouccx.ReportFix.cop.sgn"|<LVL::Info>

09/13/2016 20:08:26 upgrade_get_file.sh|File:/usr/local/bin/base_scripts/upgrade_get_file.sh:664, Function: process_download_file(), File authentication failed (rc=1)|<LVL::Error>

[Sascha Monteiro]

I need ciscocm.version3-keys.cop.sgn? I could not find this in the downloads section

[Sascha Monteiro]

so I found and installed ciscocm.version3-keys.cop.sgn, rebooted, but still getting "File authentication failed"

[arnayyar]

If you are facing the same issue discussed above and getting the same error message then you are applying the wrong cop in there.

Try the attached cop.

[Kim Nielsen]

Super Arnayyar!

Installed the cop file today, and rapports is running fine again. Will go through all of the uccx systems that we support and, see if they are affected.

Thanks a lot.

\Kim