CUCM back up issue : unable to contact server. master or local agent could be down

[007 Believe]

Great.... and ...Thank you...... restarting service resolve the issue...

[Habeeb Mohammed]

Restarting Resolved the issue. Gr8

[Jagpreet Barmi]

Hi Deepesh,

As we can see in the screenshot, you are facing the issue with CUCM Subscriber.

The most probable cause of the issue is that the IPSec-trust doesn’t match with the Publisher and the Subscriber.

To start with, check if the Subscriber is not in read-only mode (can be verified by accessing the CLI/SSH of the subscriber. It should not show any java error as soon as you login)

verify if the IPSec.pem and IPSec-trust certificates are valid and not expired on the Publisher and the Subscriber.

If that is fine, verify that the certificate Serial Number for matches for the following certs.

* IPSec.pem on the Publisher.

* IPSec-trust on the Publisher.

* IPSec-trust on the Subscriber(s).

NOTE: IPSec.pem on the Sub would have a different serial number but IPSec-trust on the Sub should match with Publisher.

If they do not match, you can download the IPSec.pem from the Publisher and upload it as IPSec-trust on the Subscriber and remove the mismatched IPSec-trust cert.

Steps:

1. Log on to CUCM OS Admin page of affected node.

2. Choose Security > Certificate Management. The Certificate List window displays.

3. You can use the Find controls in order to filter the certificate.

4. Click on ipsec.pem file and download that certificate from the Publisher.

5. Find the existing ipsec-trust with the filename of the hostname of the publisher, click on the file name and Delete.

6. Upload the downloaded ipsec.pem file with the caption ipsec-trust.

7. Restart the DRF Master Agent(MA)/DRF Local Agent (LA).

HTH,

Jagpreet Singh Barmi

[Aman Soi]

Hi Jagpreet,

I think u are talikng about the bug which was found in 8.6.1

CSCtq70900 Bug Details

Tomcat and/or IPSEC Key store corruption causes DRF backup failures

Symptom: Unable to backup one or more nodes in a cluster Tomcat unable to start with exceptions in catalina.out:org.apache.tomcat.util.net.jsse.JSSESocketFactory getStore SEVERE: Failed to load keystore type PKCS12 with path /usr/local/platform/.security/tomcat/trust-certs/tomcat-trust.keystore due to Error decoding PKCS 12 input. java.io.IOException: Error decoding PKCS 12 input. at com.rsa.cryptoj.f.eS.engineLoad(Unknown Source) at java.security.KeyStore.load(KeyStore.java:1185) Conditions: Corruption of tomcat or ipsec keystoresWorkaround: Special root-based commands to re-generate the certificates either on Pub or Sub(s) (ie tomcat certs have to be generated on a per-Sub basis and ipsec certs have to be generated on the Pub)

regds,

aman

[Jagpreet Barmi]

Hi Aman,

This bug was filed for similar conditions and errors, however, the cause is different and the version as well.

As per the bug, the certs would not be expired and there will be no mismatch. Also, regenerating the certs would not help as the ipsec-trust.keystore exists in the database but with size zero showing. The only way to clear it is through the root access.

I would suggest to start with the verification of the certs expiration date and any mismatch between the certs on the Pub and Sub before we proceed further.

Regards,

Jagpreet Singh Barmi

[Vaijanath Sonvane]

Thanks Jagpreet, this resolved my issue.

Thanks,

Vaijanath

[john.gottstein]

Should both the publisher and subscriber have a DRF Master and a DRF Local?

[Imthiyas Khan K I]

Also, Please check the ipsec and ipsec-trust certificates and it's expiry dates in Pub and Sub's

[hostasaurus]

Thanks; just ran into the same issue with the ipsec cert having expired after five years.  Problem solved.