Strange problem here and hoping someone can shed some light on this issue.
I've installed a VCS cluster with VCS ver. X8.9.2 and have both VCS's licensed with release key and options. Everything seemed to work fine. At some point, my slave VCS went offline probably due to someone shutting down the VM for whatever reason. When I investigated and found it was was turned off, I powered it back on. It was now in a partitioned state and no clustering or synchronizing was taking place.
I checked the logs on the master and saw a ton of errors stating 'gatekeeper timed out' and External Server Communications Failure', which is understandable given the slave was powered down. However now we have this issue of them not returning to its clustered state. Other log messages are: "Cluster communication failure: The system is unable to communicate with one or more of the cluster peers" and also I see: Cluster peer x.x.x.x has been unavailable for more than 14 dyas. Its licenses have been removed from the total available for use across the cluster".
What does this mean? Both servers still have their respective option key release, and proper option keys installed. I've tried restarting both, taking the slave out of cluster configuration, and re-adding, restarting, etc. Certificates have been set to factory default and the requirement to use certificates for client security is set to not required.
Both VCS's are pingable to each other so connectivity is verified good. Is there anything else I can check or fix other than getting new licenses? I find it hard to believe that they would require new licenses if one of the peers were to go down for a while. Any help would be appreciated, thanks!
