UCCX CUIC Reorts

[Ali Alqwasmi]

Dear Team

Good day...

We are having a problem on our UCCX CUIC reports.  the system shows only today's reports. Any reports from previous days will have no Data in them.

I have restarted the DB services from UCCX Serviceability and from CLI, but still the same.

Can you please advise?

Many thanks

Ali

[D_Lebedev]

Hi,

I've faced with same issue in the past.

In my case there was internal CCX component replication issue.

See attach.

Hope it helps.

[Ali Alqwasmi]

Hi D_Lebedev

Thanks for your reply...

I tried to reset the replication and reset the servers, however the problem still exists. 

I even got a new problem now. The CCX Engine service keeps shutting down. When I start it manually the status turns into Partial Service.

I don't know what could be causing all of this?

Ali

[D_Lebedev]

Hi,

1) Expand CCX engine and List here any service that is not "IN
SERVICE".

2) Collect MIVR logs with RTMT.

When you connect with RTMT,pay attention on any yellow or red error notification and messages.

3) Which ver exactly of CCX do you have? How many servers are in the cluster?

4) Connect to CLI and paste here output of "utils dbreplication status" and "utils dbreplication runtimestate" and "utils service list"

[Ali Alqwasmi]

Hi D_Lebedev and thank you for your support

Her are the answers:

1) in serviceability the service "CCX Engine" is Shutdown. when i start it it goes in partial service then shuts down again.

2) in RTMT I collected the MIVR logs. What do I need to see in them?

also in RTMT one of the errors under SyslogSeverityMatchFound events generated is:  %[ServiceName=SOAP -Log Collection APIs][Reason=Service stopped

and the other is Critical Service Down: the CCX Engine

3) the CCX version is 9.0.1

4) outcome of the CLI

utils dbreplication status:

Tue May 23 17:53:15 2017 main() DEBUG: -->
Tue May 23 17:53:17 2017 main() DEBUG: Replication cluster summary:
SERVER ID STATE STATUS QUEUE CONNECTION CHANGED
-----------------------------------------------------------------------
g_mv3luccx01prvn1_ccm9_0_2_11001_24 2 Active Local 0
g_mv3luccx01prvn2_ccm9_0_2_11001_24 3 Active Connected 0 May 23 17:09:02

Replication setup is currently in progress. Please try later or stop replication setup

Tue May 23 17:53:18 2017 main() DEBUG: <--

end of the file reached

output from utils dbreplication runtimestate:

DB and Replication Services: ALL RUNNING

DB CLI Status: No other dbreplication CLI is running...

Cluster Replication State: Replication status command started at: 2017-05-23-17-52
Replication status command COMPLETED 656 tables checked out of 704
Processing Table: devicenumplanmapremdestmap
No Errors or Mismatches found.

Use 'file view activelog cm/trace/dbl/sdi/ReplicationStatus.2017_05_23_17_52_22.out' to see the details

DB Version: ccm9_0_2_11001_24
Number of replicated tables: 704
Repltimeout set to: 300s

Cluster Detailed View from MV3LUCCX01PRVN1 (2 Servers):

PING CDR Server REPL. DBver& REPL. REPLICATION SETUP
SERVER-NAME IP ADDRESS (msec) RPC? (ID) & STATUS QUEUE TABLES LOOP? (RTMT) & details
----------- ------------ ------ ---- -------------- ----- ------- ----- -----------------
MV3LUCCX01PRVN1 10.173.50.44 0.027 Yes (2) Connected 0 match Yes (2) PUB Setup Completed
MV3LUCCX01PRVN2 10.173.50.43 0.270 Yes (3) Connected 0 match Yes (2) Setup Completed

output from utils service list:

Requesting service status, please wait...
System SSH [STARTED]
Cluster Manager [STARTED]
Service Manager is running
Getting list of all services
>> Return code = 0
A Cisco DB[STARTED]
A Cisco DB Replicator[STARTED]
Cisco AMC Service[STARTED]
Cisco Audit Event Service[STARTED]
Cisco CDP[STARTED]
Cisco CDP Agent[STARTED]
Cisco Certificate Change Notification[STARTED]
Cisco Certificate Expiry Monitor[STARTED]
Cisco DRF Local[STARTED]
Cisco DRF Master[STARTED]
Cisco Database Layer Monitor[STARTED]
Cisco Desktop Administrator Service[STARTED]
Cisco Desktop Agent E-Mail Service[STARTED]
Cisco Desktop Browser and IP Phone Agent Service[STARTED]
Cisco Desktop Call/Chat Service[STARTED]
Cisco Desktop Enterprise Service[STARTED]
Cisco Desktop LDAP Monitor Service[STARTED]
Cisco Desktop License and Resource Manager Service[STARTED]
Cisco Desktop Recording and Playback Service[STARTED]
Cisco Desktop Recording and Statistics Service[STARTED]
Cisco Desktop Sync Service[STARTED]
Cisco Desktop VoIP Monitor Service[STARTED]
Cisco Log Partition Monitoring Tool[STARTED]
Cisco RIS Data Collector[STARTED]
Cisco RTMT Reporter Servlet[STARTED]
Cisco Serviceability Reporter[STARTED]
Cisco Syslog Agent[STARTED]
Cisco Tomcat[STARTED]
Cisco Tomcat Stats Servlet[STARTED]
Cisco Trace Collection Service[STARTED]
Cisco Trace Collection Servlet[STARTED]
Cisco Unified CCX Administration[STARTED]
Cisco Unified CCX CVD Dependent Webapp[STARTED]
Cisco Unified CCX Cluster View Daemon[STARTED]
Cisco Unified CCX Configuration API[STARTED]
Cisco Unified CCX DB Perfmon Counter Service[STARTED]
Cisco Unified CCX Database[STARTED]
Cisco Unified CCX Engine[STOPPED] Component is not running
Cisco Unified CCX Notification Service[STARTED]
Cisco Unified CCX Perfmon Counter Service[STARTED]
Cisco Unified CCX SNMP Java Adapter[STARTED]
Cisco Unified CCX Serviceability[STARTED]
Cisco Unified CCX Voice Subagent[STARTED]
Cisco Unified CCX WebServices[STARTED]
Cisco Unified Intelligence Center Reporting Service[STARTED]
Cisco Unified Intelligence Center Serviceability Service[STARTED]
Cisco Unified Serviceability[STARTED]
Cisco Unified Serviceability RTMT[STARTED]
Host Resources Agent[STARTED]
MIB2 Agent[STARTED]
Platform Administrative Web Service[STARTED]
SNMP Master Agent[STARTED]
SOAP -Log Collection APIs[STARTED]
SOAP -Performance Monitoring APIs[STARTED]
SOAP -Real-Time Service APIs[STARTED]
System Application Agent[STARTED]
Primary Node =true

[D_Lebedev]

Hi,

In this case you'll have to open TAC case for better support and solution.

Another things you can to check:

1) When did you face with this issue?Was there any minor or major upgrade in the past?

2) Try srart same service from CLI  admin: utils service start "Cisco Unified CCX Engine"

3) Try inspect trace files and logs,there will be indicated root cause of your problem.

Without exact root cause from your logs it will be very very hard troubleshoot CCX engine failure reason.

It can be caused by expired .pem certificate "tomcat.pem" as you provided info SOAP-log collection service also stoped for example.It also can be caused by If another process is using the Cisco Unified CCX Engine default port 1099, the Cisco Unified CCX Engine will not start. and etc..There is a lot of causes,which wil be hard to determine without log inspection.

4) If you have valid service contract,you can try apply the latest SU update to your UCCX. As it can be so,you're facing with some bug for example.