CUCM replication mismatch (deep research) - Page 2

GlKlondike · ‎07-19-2017

Hi,

We have two servers Pub and Sub

After replication problem, execute reset replication cluster, but setup not completed

DB and Replication Services: ALL RUNNING

DB CLI Status: No other dbreplication CLI is running...

Cluster Replication State: BROADCAST SYNC Completed on 1 servers at: 2017-07-19-17-55
Last Sync Result: SYNC COMPLETED 603 tables sync'ed out of 603
Sync Errors: 4 Errors or failures were found
Use this command to see the output- file view activelog /cm/trace/dbl/2017_07_19_17_53_02_dbl_repl_cdr_Broadcast.log

DB Version: ccm9_1_2_11900_12
Repltimeout set to: 300s
PROCESS option set to: 1

Cluster Detailed View from callmanager1 (2 Servers):

PING CDR Server REPL. DBver& REPL. REPLICATION SETUP
SERVER-NAME IP ADDRESS (msec) RPC? (ID) & STATUS QUEUE TABLES LOOP? (RTMT) & details
----------- ------------ ------ ---- -------------- ----- ------- ----- -----------------
callmanager1 10.0.3.11 0.051 Yes (2) Connected 0 match No (2) PUB Setup Completed
callmanager2 10.0.4.11 4.861 Yes (4) Connected 0 match N/A (4) Setup Completed

We have 4 sync errors, for example one of this:

Jul 18 2017 15:44:01 ------ Table scan for
ccmdbtemplate_g_callmanager1_ccm9_1_2_11900_12_1_243_replicationdynamic start --------

Node Rows Extra Missing Mismatch Processed
---------------- --------- --------- --------- --------- ---------
g_callmanager1_ccm9_1_2_11900_12 1 0 0 0 1
g_callmanager2_ccm9_1_2_11900_12 1 0 0 1 0

The repair operation completed. Validating the repaired rows ...
Validation failed for the following rows:
data mismatch
key: pkid:abcdd7a0-4c70-46f6-9e58-d56bf1d409ff
------------------------------------------------------------------
WARNING: replicate is not in sync

Ok, i understand, problem in replicationdynamic table.

Now show this table on Pub, execute run sql select * from replicationdynamic

admin:run sql select * from replicationdynamic
pkid fkprocessnode datetimestamp
==================================== ==================================== =============
abcdd7a0-4c70-46f6-9e58-d56bf1d409ff b5efd7a0-4c70-46f6-9e58-d56bf1d409ff 1497420013

Same command on Sub

admin:run sql select * from replicationdynamic
pkid fkprocessnode datetimestamp
==================================== ==================================== =============
abcdd7a0-4c70-46f6-9e58-d56bf1d409ff b5efd7a0-4c70-46f6-9e58-d56bf1d409ff 1497420013

This value the same on Pub and Sub, why cucm not sync?

Plz help, i dont have a choice to call TAC

GlKlondike · ‎07-20-2017

Yes, i do every step, all this table in good sync status, no error.

runtemstate show last setup try

admin:utils dbreplication runtimestate

DB and Replication Services: ALL RUNNING

DB CLI Status: No other dbreplication CLI is running...

Cluster Replication State: BROADCAST SYNC Completed on 1 servers at: 2017-07-20-15-04
Last Sync Result: SYNC COMPLETED 603 tables sync'ed out of 603
Sync Errors: 4 Errors or failures were found
Use this command to see the output- file view activelog /cm/trace/dbl/2017_07_20_15_02_14_dbl_repl_cdr_Broadcast.log

DB Version: ccm9_1_2_11900_12
Repltimeout set to: 300s
PROCESS option set to: 1

Cluster Detailed View from callmanager1 (2 Servers):

PING CDR Server REPL. DBver& REPL. REPLICATION SETUP
SERVER-NAME IP ADDRESS (msec) RPC? (ID) & STATUS QUEUE TABLES LOOP? (RTMT) & details
----------- ------------ ------ ---- -------------- ----- ------- ----- -----------------
callmanager1 10.0.3.11 0.054 Yes (2) Connected 0 match No (2) PUB Setup Completed
callmanager2 10.0.4.11 5.034 Yes (4) Connected 0 match N/A (4) Setup Completed

Everything showing problem with Application user "axluser", maybe his corrupted on Pub, i can't login (login/password) error, but on Sub login ok.

From stepping off of it field timelastaccessed datetimestamp lastsuccessfullogintime in credentialdynamic table on Application user "axluser" showing different results on Pub and Sub and sync not completed, how i write in message above.

I don't know what can i do with "axluser", i can't not delete (error Delete failed. Could not delete a row.) i can't not change password (error [2654] IMS_808)

Oh, what to do

Jitender Bhandari · ‎07-20-2017

Hi,

It might be possible to change the table settings through root but no option on GUI \ CLI through which we can do this.

If TAC is not a possibility then we can try running recovery CD.

http://www.cisco.com/c/en/us/support/docs/unified-communications/unified-communications-manager-callmanager/118948-technote-cucm-00.html

Else we will have to rebuild and restore from backup to fix it.

Sorry was not able to help you fix this issue.

JB

GlKlondike · ‎07-20-2017

Thank you very much for helping, if not hard, plz answer for this questions:

We can just forgot about this issue and just use CUCM, before upgrade or something else?

In this situation we can do backup or not?

Recovery CD for root access and manual change field on database table?

Jitender Bhandari · ‎07-20-2017

Hi,

Yes i don't see this issue expanding to other parts of CUCM and causing major issue like call routing.

Also there are more chances that upgrade to a higher version might also fix the issue.

You can still take backup with no issue.

"Recovery CD for root access and manual change field on database table?"

That is something i would not recommend as it would through your CUCM in unsupported configuration state, which means TAC would not support you going forward, even if you have valid contract to engage TAC.

JB