IM&P v11.5 Cluster Stuck In Failover State

[Chris M]

I noticed new users are not being assigned to IM&P servers.  When trying to assign them to a presence server in CUCM I received an error stating "Update failed. Cannot assign a user to a server that is not in a valid state".

On the IM&P servers there is an error stating "An automatic failover has been initiated due to the peer node being down" on the same day there was a network hardware failure in the core. In CUCM > System > Presence Redundancy Groups for IM&P it shows node 1 as "Running in Backup Mode" and node 2 as "Taking Over".  Both reasons state "Peer Down".

They appear to be stuck in this state, and I can't find a document on how to best manually force them back.  I assume it would be as simple as restarting some service(s) but I'm not sure which one(s).

[Chris M]

Update: Rebooted the publisher IM&P but Cisco Tomcat will not start now, despite "utils system start Cisco Tomcat". Creating a TAC case.

[Hari Prasad]

Hello Chris,

Could you please share outputs.

CUCM PUB>>

utils dbreplication runtimestate

Show network cluster

utils diagnose test.

Utils service list

Presence Node PUB->>

utils diagnose test

utils service list

[Chris M · Accepted solution]

Thanks Hari Prasad for the reply.
I had contacted TAC with the tomcat logs.
After reviewing replication and finding no issues I was instructed to attempt to force HA failover in the CLI but this did not work as the devices were not in a working state.
What worked was going into CUCM > System > Presence Redundancy Groups, select the group and disable HA. Then I restarted both presence servers. When they came back the Tomcat services were running fine. Once all services were up, I enabled HA in CUCM and tested adding a user successfully. Users can also log in to Jabber.