So, the initial problem was receiving the following error when trying to access the subscriber from the servicability drop down on the publisher:
I found this solution which advised to regenerate the tomcat and ipsec certificates, which I did:
It fixed the initial problem, but introduced a 'host not found' problem on 7962 phones when trying to access the corporate directory. Corporate directory does still work with IPC however. Just to note, directory URLs in Enterprise parameters are defined with IP addresses, not hostnames, so no DNS worries. Checking an affected phone's logs, I see:
339: ERR 08:53:39.390812 SECD: EROR:clpState: SSL3 alert write:fatal:handshake failure:<10.111.10.11>
1340: ERR 08:53:39.391810 SECD: EROR:clpSetupSsl: ** SSL handshake failed, <10.111.10.11> c:9 s:10
1341: ERR 08:53:39.392489 SECD: EROR:clpSetupSsl: SSL/TLS handshake failed, <10.111.10.11> c:9 s:10
1342: ERR 08:53:39.393385 SECD: EROR:clpSetupSsl: SSL/TLS setup failed, <10.111.10.11> c:9 s:10
1343: ERR 08:53:39.394002 SECD: EROR:clpSndStatus: SSL CLNT ERR, srvr<10.111.10.11>
1344: ERR 08:53:39.394790 SECD: EROR:secErr_errStr: *** bad err table ***
1345: ERR 08:53:39.395435 SECD: EROR:secErr_errStr: ** SEC-ERR: code:3(N/A) subcode:9(UNKNOWN_CERT)
1346: ERR 08:53:39.396243 SECD: EROR:clpSndStatus: ** SEC-ERR: desc <HTTPS cert failed auth via TVS>
1347: WRN 08:53:39.408913 JVM: Startup Module Loader|cip.http.ae:? - listener.httpFailed
1348: NOT 08:53:39.437862 SECD: clpDelClnt: closing conn to <10.111.10.11>, c:9, s:10
1349: NOT 08:53:39.439919 SECD: clpDelClnt: Adding a one second delay before we close the local socket
I restarted TVS on both nodes; no change.
We deleted the ITL file from the phone; no change.
I ran a 3rd party ITL scanner which found no ITL issues.
Appreciate any thoughts or suggestions.
Are you getting the error when pressing the services button or you see corporate directory and then when you press it, you get the error?
Did you regenerate tomcat and ipsec on all the nodes? Was the tomcat and TFTP service restarted on all the nodes? Try restarting Cisco tomcat, TVS and TFTP in order and then test.
Please share the output of show itl.
What is the value of "Prepare Cluster for Rollback to pre 8.0" in enterprise parameters?
Were the certificates CA signed or self signed?
It's most likely an ITL issue, since you recreated the Tomcat cert
This example shows what happens when the Directories button on the IP phone is pressed. The Directories URL is configured for HTTPS, so the phone is presented with the Tomcat web certificate from the Directories server. This Tomcat web certificate (tomcat.pem in OS Administration) is not loaded in the phone, so the phone must contact TVS in order to authenticate the certificate.
Make sure the ITL on the phone, matches the ITL on CUCM, you can try to do a factory reset of the device, if it works, it's pretty sure because of ITL.
Jamie and Ronak, thank you for your responses. We had to do a cluster reboot to try to clear a SIP trunking issue and tried the directory afterwards and it unexpectedly worked. I'm not sure why that worked, but I'll take it.