CUCM DRS menu options are not loading and schedule backups are failing

[DavidImmanuelSamDaniel8838]

Hello All, I am not able to access the DRS Backup and Restore menu, including the backup device. GUI loading for very long time and fails until the server logout through timeout. I tried to access through CLI using the command utils disaster_recovery device list to find unused backup devices and delete them, but the CLI also stuck.  CUCM version is 14.0.1.12900-161. Please find the troubleshooting steps completed 

1) Restarted DRF Master on Pub and DRF local on all servers. 

2) Regenerated IPSec certificate on all the servers of the cluster and uploaded PUB IPSEC certificate as IPSEC-trust on all SUBs. Also, restarted DRF Master on Pub and DRF local on all servers. 

3) Rebooted CUCM PUB.

4) Validated all Tomcat certificates on call servers, DB replication, NTP and diag test 

5) UCCX and CUCM both use the same SFTP server for DRS backup, UCCX works fine. 

6) Tried different browsers.

7) No DRF failure from RTMT alert central. 

Has anyone faced this issue before? Can you please help with this? am I missing any troubleshooting steps? 

Regards,

David

[Brad Magnani · Accepted solution]

I'd suggest enabling Debug level DRF Local and Master in Unified Serviceability, if not already.  Then reproduce the blank page load and note the timestamp and pull the traces from RTMT:

Cisco DRF Local
Cisco DRF Master
Cisco Tomcat
Cisco Tomcat Security
EventViewer-Application Logs
EventViewer-System Logs

[DavidImmanuelSamDaniel8838]

Thanks Brad. 

[DavidImmanuelSamDaniel8838 · Accepted solution]

I pulled the DRF master and local found multiple errors related to DRF. However, error "drfNetServerWorker.drfNetServerWorker: Unable to create input/output stream to client org.bouncycastle.tls.TlsFatalAlertReceived: certificate_unknown(46)" looked more relevant in the log. Checked on the Cisco bug searching tool and found a relevant bug CSCwf42793. This bug is actually for UCCX, however, CUCM is also affected by this. 

---

https://quickview.cloudapps.cisco.com/quickview/bug/CSCwf42793

tomcat-ECDSA self-signed certificate had expired 2 months ago, regenerated the tomcat-ECDSA on all the servers and uploaded the pub tomcat-ECDSA on all the other servers as tomcat-trust. Restarted the Tomcat service, DRF master and local. 

Issue fixed now. 

[Brad Magnani]

Awesome, glad to hear!