There were several reboots of Primary and Secondary SUBS. I noticed that Primary SUBS couldn’t authenticate on PUB. I tried to restore system from backup, but I cannot restore Primary SUBS.
Primary SUBS now serves ~3500 phones.
Finally, I have strange situation:
admin:utils dbreplication runtimestate
Server Time: Wed Apr 7 18:49:32 +03 2021
Cluster Replication State: BROADCAST SYNC ended at: 2021-04-07-17-57
Sync Result: SYNC COMPLETED on 706 tables out of 706
Sync Status: All Tables are in sync
Use CLI to see detail: 'file view activelog cm/trace/dbl/20210407_175345_dbl_repl_output_Broadcast.log'
DB Version: ccm11_5_1_13901_3
Repltimeout set to: 300s
PROCESS option set to: 1
Cluster Detailed View from SVRUA000101 (5 Servers):
PING DB/RPC/ REPL. Replication REPLICATION SETUP
SERVER-NAME IP ADDRESS (msec) DbMon? QUEUE Group ID (RTMT) & Details
----------- ---------- ------ ------- ----- ----------- ------------------
SVRUA000101 10.81.1.30 0.014 Y/Y/Y 0 (g_2) (2) Setup Completed
SVRUA000104 10.81.1.32 0.163 Y/Y/Y 0 (g_8) (2) Setup Completed
SVRUA000105 10.81.2.32 20.571 Y/Y/Y 0 (g_9) (2) Setup Completed
svrua102-103-group 10.81.1.31 0.298 Y/Y/Y -- (-) (-) Not Requested
SVRUA000103 10.81.2.31 0.255 Y/Y/Y 0 (g_10) (2) Setup Completed
svrua102-103-group it’s a name of Cisco Unified Communications Manager Group and it has correct Primary and Secondary SUBS. Moreover, everything is right in PUB Web Administration.
There should be SVRUA000102 (Primary SUBS)
admin:show network cluster
10.81.2.32 SVRUA000105.acme.net SVRUA000105 Subscriber callmanager DBSub authenticated using TCP since Wed Apr 7 18:45:09 2021
10.81.2.31 SVRUA000103.acme.net SVRUA000103 Subscriber callmanager DBSub authenticated using TCP since Wed Apr 7 18:18:17 2021
10.81.1.32 SVRUA000104.acme.net SVRUA000104 Subscriber callmanager DBSub authenticated using TCP since Wed Apr 7 18:28:38 2021
10.81.1.51 SVRUA000122.acme.net SVRUA000122 Subscriber cups DBSub not authenticated - INITIATOR since Wed Apr 7 15:03:23 2021
10.81.1.50 SVRUA000121.acme.net SVRUA000121 Subscriber cups DBPub authenticated using TCP since Wed Apr 7 15:03:40 2021
10.81.1.30 SVRUA000101.acme.net SVRUA000101 Publisher callmanager DBPub authenticated
Server Table (processnode) Entries
----------------------------------
10.81.2.32
10.81.1.32
10.81.1.30
10.81.1.31
10.81.2.31
SVRUA000121.acme.net
10.81.1.51
There are no Primary SUBS 10.81.1.31 SVRUA000102.acme.net
admin:set network cluster subscriber details CUCM SVRUA000102 10.81.1.31 acme.net
"CUCM SVRUA000102" is a duplicate entry.
Unable to add: SVRUA000102
Executed command unsuccessfully
admin:run sql select * from processnode
pkid name mac systemnode description isactive nodeid tknodeusage ipv6name fklbmhubgroup tkprocessnoderole tkssomode
==================================== ====================== === ========== ==================== ======== ====== =========== ======== ============= ================= =========
00000000-1111-0000-0000-000000000000 EnterpriseWideData t t 1 1 NULL 1 0
935ac2d6-5323-cac2-e81a-4c5c95a1e5c7 10.81.2.32 f Secondary TFTP t 9 1 NULL 1 0
bae67e02-3ed9-44e2-fc8b-33c3dc7a907d 10.81.1.32 f t 8 1 NULL 1 0
d08f0d9f-f943-4708-bb9e-49b91a1b5e29 10.81.1.30 f t 2 0 NULL 1 0
e90abb6b-c0ad-6bb0-1564-a350081b2819 10.81.1.31 f Primary Subscriber t 11 1 NULL 1 0
ed68c92b-79fd-db0f-e501-8d4c4516e3e6 10.81.2.31 f Secondary Subscriber t 10 1 NULL 1 0
1474eb6e-f9ea-4090-b185-41c9ed6ab413 SVRUA000121.acme.net f t 13 0 NULL 2 0
b2338c6d-ac07-4f03-adfe-4db1e25de703 10.81.1.51 f t 14 1 NULL 2 0
Please help to get out of this complicated situation.
