Buy or Renew
Buy or Renew
Cisco + Splunk: It’s a new day for your data. Learn more.
 
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
cancel
Start a conversation
6552
Views
5
Helpful
15
Replies

UCCX %MIVR-HR_MGR-3-HISTORICAL_DB_WRITE_FAILED

Kapil Atrish
Level 3
Level 3

‎01-06-2012 09:08 AM - edited ‎03-14-2019 09:08 AM

Hi,

I've a client with UCCX HA setup (System version: 8.0.2.10000-41).

I saw this alert generated yesterday in the RTMT under Alert Central. It came from Secondary UCCX server (It is running as Master). The client has some weird issues since yesterday, the agents started getting Licensing Error, CSD is not updating Agents statistics, Real time data in RT tables (for wallboard) is not getting updated etc. I bounced the LRM service and agents were able to login fine.

I looked at Bug: CSCtq91894 but not very helpful. Does anyone know what could have caused this alert and any possible fixes? I am attaching the MIVR logs covering this event.

Module Name : HISTORICAL_REPORTING_MANAGER

Module Failure Name : HISTORICAL_DATABASE

UNKNOWN_PARAMTYPE:Module Run-time Failure Cause : 3

Module Failure Message : Historical database queue is full. Historical data saved in files. Queue size= 0 Total number of lost records= 1 Please check dat

AppID : Cisco Unified CCX Engine

ClusterID :

NodeID : CELCC6802

TimeStamp : Thu Jan 05 09:50:11 AST 2012.

The alarm is generated on Thu Jan 05 08:50:11 EST 2012.Module Name : HISTORICAL_REPORTING_MANAGER
Module Failure Name : HISTORICAL_DATABASE
UNKNOWN_PARAMTYPE:Module Run-time Failure Cause : 3
Module Failure Message : Historical database queue is full. Historical data saved in files. Queue size= 0 Total number of lost records= 1 Please check dat
AppID : Cisco Unified CCX Engine
ClusterID :
NodeID : CELCC6802
TimeStamp : Thu Jan 05 09:50:11 AST 2012.
The alarm is generated on Thu Jan 05 08:50:11 EST 2012.

Thanks,

Kapil

1 person had this problem
Labels:
120094-Cisco002MIVR090.zip
0 Helpful
15 Replies 15

Gajanan Pande
Cisco Employee
Cisco Employee

‎01-08-2012 10:40 AM

On UCCX version 8.0.2 if you does not have SU4, its most recommended to have it. It has lots of bug fixes including the one you are likely to hit. http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&bugId=CSCtl93897

SU4 can be downloaded from here http://www.cisco.com/cisco/software/release.html?mdfid=270569179&flowid=5217&softwareid=280840578&release=8.0(2)_SU4&rellifecycle=&relind=AVAILABLE&reltype=latest

If your setup is already in production, I'd recommend to have Cisco TAC confirm it for you.

Pls rate the post if it helps.


GP.

0 Helpful

yavuzsab
Level 1
Level 1
In response to Gajanan Pande

‎04-10-2012 06:25 PM

We seem to be receiving these same alerts on UCCX 8.5(1)SU2 with COP file on HA.

0 Helpful

anchoudh
Level 9
Level 9
In response to yavuzsab

‎04-10-2012 07:20 PM

Hi,

After searching for the error logs cam across a TAC case reference:617010385

Which is pointing to below 2 defects which have a fix for this issue.

---------------------------------------------------------

CSCtj88620: run time error while generating agent related historical reports

Description: HRC throws run time error while generating some agent related reports. HRC log shows the error "A subquery has returned not exactly one row".

Following is seen in the RTMT for the same time frame.

Module Name : HISTORICAL_REPORTING_MANAGER

Module Failure Name : HISTORICAL_DATABASE

UNKNOWN_PARAMTYPE:Module Run-time Failure Cause : 3

Module Failure Message : Historical database queue is full. Historical data saved in files. Queue size= 0 Total number of lost records= 300 Please check d  AppID : Cisco Unified CCX Engine

ClusterID :   NodeID : uccx1a

TimeStamp : Wed Nov 03 08:02:16 EDT 2010.

The alarm is generated on Wed Nov 03 08:02:16 EDT 2010.

To Be fixed: 8.0(2)SU2 8.5(1)SU3

-------------------------------------------------------------

CSCtl93897: UCCX Engine tries to update missing table, which causes RTMT alerts

Description:

When running a historical report, the customer's system generates RTMT alerts such as:

Subject: [RTMT-ALERT-StandAloneCluster] HistoricalDataWrittenToFiles

Module Name : HISTORICAL_REPORTING_MANAGER Module Failure Name : HISTORICAL_DATABASE UNKNOWN_PARAMTYPE:Module Run-time Failure Cause : 3

Module Failure Message : Historical database queue is full. Historical data saved in files. Queue size= 0 Total number of lost records= 1 Please check dat AppID : Cisco Unified CCX Engine ClusterID : 

After further investigations, the cause of this issue was found to be the UCCX engine, which tries to write data to a table from a previous version, called the WorkflowTask table. Given that there is no longer a table within the database by this name, this causes the system to generate RTMT alerts.

While there is no direct impact on the system in a critical way, such as core functionality or critical data loss, this still needs to be addressed, as it causes concerns for the customer who may see these errors.

To Be fixed: 8.5(1)SU3 8.0(2)SU4

--------------------------------------------------------------

As I see you are currently running on UCCX 8.5(1)SU2 with COP file on HA, need to come to 8.5(1)SU3.

As of now I can only see 8.5 SU2 posted in CCO.

http://www.cisco.com/cisco/software/release.html?mdfid=270569179&flowid=5217&softwareid=280840578&release=8.0%282%29_SU4&relind=AVAILABLE&rellifecycle=&reltype=latest

Please contact Cisco TAC for the ES or any other available COP file for this issue.

Hope it helps.

Anand

Please rate helpful posts by clicking on the stars below the right answers !!

yavuzsab
Level 1
Level 1
In response to anchoudh

‎04-10-2012 07:40 PM

You saved me from raising a TAC case

Thank you very much Anand.

-Yavuz

0 Helpful

anchoudh
Level 9
Level 9
In response to yavuzsab

‎04-10-2012 07:51 PM

Hi Yavuz,

Glad to know that :-).

Please dont forget to rate the helpful posts by clicking on the stars below the right answer.

Thanks,

Anand

0 Helpful

yavuzsab
Level 1
Level 1
In response to anchoudh

‎04-10-2012 08:26 PM

Anand, Do you have an exact date on when UCCX 8.5(1)SU3 be release on CCO downloads page? Thanks, Yavuz

0 Helpful

anchoudh
Level 9
Level 9
In response to yavuzsab

‎04-10-2012 08:39 PM

Hi Yavuz,

The expected release date for UCCX 8.5(1)SU3 is April 18th, unless there are no major issues find during regression testing which might be in progress.

I hope there are no delays in this.

Hope it helps.

Anand

Please rate helpful posts !!

0 Helpful

yavuzsab
Level 1
Level 1
In response to anchoudh

‎04-10-2012 09:03 PM

Fantastic news!

Would you be able to obtain the SU3 release notes from CCBU?

It will be nice to find out how many caveats SU3

0 Helpful

anchoudh
Level 9
Level 9
In response to yavuzsab

‎04-11-2012 12:28 AM

Hi Yavuz,

Releaae notes will come along with the SU, as there will be changes till the last moment in it, and we will not get a copy of it till it is released.
Wait for few more days and you will get it in CCO.

Hope it helps.

Anand

Please rate helpful posts !!

0 Helpful

Saima Harris
Level 1
Level 1
In response to Gajanan Pande

‎10-21-2015 08:10 AM

Hi Gajanan,

I'm getting the same alert for UCCX 10.5.1.

Do you know if same bug (CSCtj88620) is affecting 10.5.1 as well or if there is another bug related to that ?

Also getting "CCXToCUICCVDSyncFailed" alert, do you know if this is related to a bug as well ? 

=

HistoricalDataWrittenToFiles
Module Name : HISTORICAL_REPORTING_MANAGER
Module Failure Name : HISTORICAL_DATABASE
UNKNOWN_PARAMTYPE:Module Run-time Failure Cause : 3
Module Failure Message : Historical database queue is full. Historical data saved in files. Queue size= 0 Total number of lost records= 1 Please check dat
AppID : Cisco Unified CCX Engine
ClusterID :

==

 

CCXToCUICCVDSyncFailed
RTR : User with userID:mpoulter,sv025,sv021,sv013,sv024,sv029,sv022,ta031,sv032,sv020,sv011,sv018,sv030,sv003,sv031,ct14,sv036,rn064,sv
AppID : Cisco Unified CCX Cluster View Daemon

 

sharri

0 Helpful

Deepak Rawat
Cisco Employee
Cisco Employee
In response to Saima Harris

‎10-21-2015 08:58 AM

Hi Saima,

 

In your other below post, wherein I replied few minutes back you did not mention about the CCXToCUICCVDSyncFailed alert hence that response is categorically for "Historical database queue is full" issue. :

https://supportforums.cisco.com/discussion/11454016/uchrmgr-2-uccxhistoricaldatawrittentofiles

Now coming to this CCXToCUICCVDSyncFailed alert in conjunction with Historical database queue is full, you are definitely hitting the below defect wherein the alerts will immediately generate after the schedule purge is done. You can verify the same by going to Cisco Unified CCX Administration >> Tools >> Historical Reporting >> Purge Schedule Configuration and check the time for "Daily Purge At". Then match this time with the time of alerts and you will find it the same confirming the defect:

https://tools.cisco.com/bugsearch/bug/CSCur00452/?reffering_site=dumpcr

 

This defect is fixed in 10.6(1) train and since 10.6(1) SU1 is also out, it will be best to upgrade to that option in order to resolve the issue.

 

Little background on this defect CSCur00452:


In a HA environment, the sync request should only come from publisher UCCX but because of the issue in code, the sync request comes from the secondary node as well and since there is no mechanism in the code to entertain that, false alerts are generated.Starting version 10.6(1)  onwards, there had been a fix implemented in the code itself wherein the sync request does not come from secondary node avoiding these false alerts to generate.

 

Regards

Deepak

0 Helpful

Saima Harris
Level 1
Level 1
In response to Deepak Rawat

‎10-21-2015 10:46 AM

Thanks Deepak for the detailed information.

I will try those and let you know the outcome

 

Sharri 

0 Helpful

Saima Harris
Level 1
Level 1
In response to Deepak Rawat

‎10-21-2015 11:06 AM

Hi Deepak,

So that means if we are getting both alerts 'CCXToCUICCVDSyncFailed' & 'Historical database queue is full'

at the same time as 'Purging Schedule Daily' then we are hitting the defect below:

CSCur00452

 

Sharri

 

 

0 Helpful

Deepak Rawat
Cisco Employee
Cisco Employee
In response to Saima Harris

‎10-21-2015 12:46 PM

Hi Saima,

Yes that is correct.

 

Regards

Deepak

0 Helpful
Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Customers Also Viewed These Support Documents