Re: Unable to contact server. Master or Local Agent could be down.

GilbertoNiño · ‎08-24-2022

Hi everyone,

I management a CUCM 10.5.1 System version: 10.5.1.10000-7 Unrestricted.

I have an error in the backup process, when the backup finished, the system show me this message:

“WARNING: Unable to contact server. Master or Local Agent could be down, Backup Completed...”

I read in another post that this error is solved executing these process.

Renegerate the IPSEC certificates in PUB and SUSC CUCM.
Restart the DRF Local/Master agent on PUB and SUSB CUCM. (This process I did, but the error continue)

My questions is, if the CUCM don´t have a active license, I can regenerate the certificates ipsec in the CUCM?

This regenerate of certificates don´t ocurr something problems with the CUCM off license?, because it's in production.

I hope that somebody you can help me in this topic.

Thanks a lot

Regards.

Roger Kallberg · ‎08-24-2022

From what I know there would be no problem with regeneration of this certificate even if your system is not in compliance state for the licensing. For additional information on how to renew certificate have a look at this document that I wrote awhile back. Cisco UC Certificates Renewal Guide

GilbertoNiño · ‎08-26-2022

Hi Roger,

Thanks for your comment, I follow review to sure that the regenerate these certificates don´t generate any problem with the CUCM operating. I need be sure.

I viewed your post about how to generate the certificates and other processes, very good that content, thank for your shared.

Roger Kallberg · ‎08-26-2022

One note, do not regenerate any certificate(s) that is not close to or has already expired as that would just cause you extra work.

GilbertoNiño · ‎08-26-2022

Ok, I review the date validate, and the ipsec.pem show this information.

Validity From: Wed Nov 20 12:54:56 VET 2019
To: Mon Nov 18 12:54:55 VET 2024

About to your comment, this certification isn't closed or expired

Roger Kallberg · ‎08-26-2022

That’s correct. So if this is present on all your nodes in the CM/IMP cluster there should be no need to renew it. This specific certificate has no impact on any other services than backups, so if you do select to renew it there should be no negative side effects on anything else.

GilbertoNiño · ‎08-26-2022

Perfect Roger, my next step will go to talk with the operation management, expose this topic, propose regenerate this certificate and run the backup againg, validate results.

Thanks for your collaborate.

Stephanie Knoop · ‎08-24-2022

Gilberto,
I would not immediately jump to the conclusion that the certificates need to be regenerated unless they are expired or if there are other services also not working on the cluster. I have had this error several times on my cluster and 2 things usually correct it:

1. Validate that there is good connectivity between the cluster members
2. Restart the following services:
- On the subscriber(s) the "Cisco DRF Local" service
- On the publisher the "Cisco DRF Local" and "Cisco DRF Master"
Those actions are a safer first attempt than regenerating certificates and only affect the backup services.

GilbertoNiño · ‎08-26-2022

Hi Stephanie,

Thanks for your feedback.

I reviewed the connectivity both members, physically are correct, it's into the same network, the ping is successful between they.

The other activity suggested, I did it. I restared these services, but the system continue show the error mesagge.

GN

Robert Profit · ‎09-01-2022

@GilbertoNiño If the DRF Master and Local services are started on the Publisher, and the DRF Local service on all Subscribers is started, the message you are seeing will not clear until the next time a backup is attempted. Additionally, you can navigate to DRF->Backup->Current Status to find the server in your cluster in which the backup failed on.

GilbertoNiño · ‎09-26-2022

Hi everyone,

Regards, I continuing with this theme, because after of did a large investigation and revision on both servers (PUB/SUSC) I found this:

Cisco Unified CM IM and Presence Administration/PUBLISHER

Connection to the Server cannot be established (unknown error) I try to call the same server PUB

Connection to the Server cannot be established (Certificate Exception) I try to call the Suscriber server

From CLI server PUBLISHER:

admin:utils dbreplication status

Replication status check is now running in background.

Use command 'utils dbreplication runtimestate' to check its progress

The final output will be in file cm/trace/dbl/sdi/ReplicationStatus.2022_09_26_08_02_34.out

Please use "file view activelog cm/trace/dbl/sdi/ReplicationStatus.2022_09_26_08_02_34.out " command to see the output

admin:utils dbreplication runtimestate

Server Time: Mon Sep 26 08:07:22 VET 2022

Cluster Replication State: Replication status command started at: 2022-09-26-08-03

Traceback (most recent call last):

File "/usr/local/cm/bin/DbReplRTstate.py", line 1225, in <module>

(cTables,nerr,lastTab) = getCheckProg(cfile)

File "/usr/local/cm/bin/DbReplRTstate.py", line 825, in getCheckProg

return ntab, int(nmis),curT[0]

IndexError: string index out of range

Cisco Unified CM IM and Presence Administration/SUSCRIBER

Database communication error. I can't into the web service.

From CLI server SUSCRIBER:

admin:utils dbreplication status

Database is down; please restart the database before reattempting a status check

Use this command to see the output - file view activelog cm/trace/dbl/sdi/ReplicationStatus.2022_09_26_08_13_13.out

admin:utils dbreplication runtimestate

Cisco Database Layer Monitor is not running

Cisco DB is not running

admin:utils service restart A Cisco DB
Don't press Ctrl-c while the service is getting RESTARTED.If Service has not Restarted Properly, execute the same Command Again
Service Manager is running
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]
A Cisco DB[STARTING]

Control-C pressed (This output not finished, I will press Control+C to stopped)

I observed that the Databases of Publisher and Suscriber Servers, both are fail present.

How I can to do to resolved this incident with the Databases? Any other command to restablisher this services?

Is possible to do formating from the servers only this module of Cisco Unified CM IM and Presence Administration, without affecting the CUCM operating?

I hope that you can help me in this theme

Regards

GilbertoNiño · ‎09-26-2022

Hi Everyone, I hope that you be fine.

I continuing for this theme, after to did a large investigation and reviewed both Servers Publisher/Suscriber, I found this scene:

Cisco Unified CM IM and Presence Administration PUBLISHER

To try ingress on web services and executed from of the "Serviceability" -> "Tools" -> "Control Center - Network Services" -> and select the itself server (the same IP server PUB) the system show this message error:

If you try call the suscriber server, show this error

CLI from Publisher Server

admin:utils dbreplication status

Replication status check is now running in background.

Use command 'utils dbreplication runtimestate' to check its progress

The final output will be in file cm/trace/dbl/sdi/ReplicationStatus.2022_09_26_08_02_34.out

Please use "file view activelog cm/trace/dbl/sdi/ReplicationStatus.2022_09_26_08_02_34.out " command to see the output

admin:utils dbreplication runtimestate

Server Time: Mon Sep 26 08:07:22 VET 2022

Cluster Replication State: Replication status command started at: 2022-09-26-08-03

Traceback (most recent call last):

File "/usr/local/cm/bin/DbReplRTstate.py", line 1225, in <module>

(cTables,nerr,lastTab) = getCheckProg(cfile)

File "/usr/local/cm/bin/DbReplRTstate.py", line 825, in getCheckProg

return ntab, int(nmis),curT[0]

IndexError: string index out of range

Cisco Unified CM IM and Presence Administration SUSCRIBER

When you try ingress to web services on Suscriber server, the error is "Database communication error"

CLI from Suscriber Server

admin:utils dbreplication status

Database is down; please restart the database before reattempting a status check

Use this command to see the output - file view activelog cm/trace/dbl/sdi/ReplicationStatus.2022_09_26_08_13_13.out

admin:utils dbreplication runtimestate

Cisco Database Layer Monitor is not running

Cisco DB is not running

I try ti restart the DB service.

admin:utils service restart A Cisco DB

Don't press Ctrl-c while the service is getting RESTARTED.If Service has not Restarted Properly, execute the same Command Again

Service Manager is running

A Cisco DB[STARTING]

This output don't finished, I press Control+C to stopped.

I hope that you can help me with this incident.

Any other commands or action to resolved the database errors?

Is possible to do the formating of the both virtual servers (PUB/SUSC) only this module "Cisco Unified IM and Presence Serviceability" without affecting to CUCM operations? Formating this servers, loading the IOS, is possible to do this?

I wait to your feedback?

Thanks a lot.