UCCX 10.6 Cannot add an HA node

[christos.georgiadis]

Hi all,

I try to install a second UCCX node. Installation is fine, it can contact the primary server just fine. But HA does not seem to be working. I reinstalled the secondary node but the situation is still the same

You can see that the status for the secondary node is UNKNOWN.

In the serviceability I can only see the primary node in the drop down menu when selecting a server

I cannot login to the web gui of the secondary node (i get username password wrong)

I can only ssh to it and I can see the following when I do utils service list

admin:utils service list page

Requesting service status, please wait...
System SSH [STARTED]
Cluster Manager [STARTED]
Cisco SCSI Watchdog [STARTED]
Service Manager [STARTED]
Service Manager is running
Getting list of all services
>> Return code = 0
A Cisco DB[STARTED]
A Cisco DB Replicator[STARTED]
Cisco AMC Service[STARTED]
Cisco Audit Event Service[STARTED]
Cisco CDP[STARTED]
Cisco CDP Agent[STARTED]
Cisco CallManager Serviceability[STARTED]
Cisco Certificate Change Notification[STARTED]
Cisco Certificate Expiry Monitor[STARTED]
Cisco DRF Local[STARTED]
Cisco DRF Master[STOPPED]  Commanded Out of Service
Cisco Database Layer Monitor[STARTED]
Cisco Log Partition Monitoring Tool[STARTED]
Cisco RIS Data Collector[STARTED]
Cisco RTMT Reporter Servlet[STARTED]
Cisco Syslog Agent[STARTED]
Cisco Tomcat[STARTED]
Cisco Tomcat Stats Servlet[STARTED]
Cisco Trace Collection Service[STARTED]
Cisco Trace Collection Servlet[STARTED]
Cisco Unified CCX Administration[STARTED]
Cisco Unified CCX CVD Dependent Webapp[STARTED]
Cisco Unified CCX Cluster View Daemon[STARTED]
Cisco Unified CCX Configuration API[STARTED]
Cisco Unified CCX DB Perfmon Counter Service[STARTED]
Cisco Unified CCX Database[STARTED]
Cisco Unified CCX Perfmon Counter Service[STARTED]
Cisco Unified CCX SNMP Java Adapter[STARTED]
Cisco Unified CCX Serviceability[STARTED]
Cisco Unified CCX Voice Subagent[STARTED]
Cisco Unified CCX WebServices[STARTED]
Cisco Unified Serviceability RTMT[STARTED]
Host Resources Agent[STARTED]
MIB2 Agent[STARTED]
Platform Administrative Web Service[STARTED]
SNMP Master Agent[STARTED]
SOAP -Log Collection APIs[STARTED]
SOAP -Performance Monitoring APIs[STARTED]
SOAP -Real-Time Service APIs[STARTED]
System Application Agent[STARTED]
Cisco Desktop Administrator Service[STOPPED]  Service Not Activated
Cisco Desktop Agent E-Mail Service[STOPPED]  Service Not Activated
Cisco Desktop Browser and IP Phone Agent Service[STOPPED]  Service Not Activated
Cisco Desktop Call/Chat Service[STOPPED]  Service Not Activated
Cisco Desktop Enterprise Service[STOPPED]  Service Not Activated
Cisco Desktop LDAP Monitor Service[STOPPED]  Service Not Activated
Cisco Desktop License and Resource Manager Service[STOPPED]  Service Not Activated
Cisco Desktop Recording and Playback Service[STOPPED]  Service Not Activated
Cisco Desktop Recording and Statistics Service[STOPPED]  Service Not Activated
Cisco Desktop Sync Service[STOPPED]  Service Not Activated
Cisco Desktop VoIP Monitor Service[STOPPED]  Service Not Activated
Cisco Finesse Tomcat[STOPPED]  Service Not Activated
Cisco Serviceability Reporter[STOPPED]  Service Not Activated
Cisco Unified CCX Engine[STOPPED]  Service Not Activated
Cisco Unified CCX Notification Service[STOPPED]  Service Not Activated
Cisco Unified Intelligence Center Reporting Service[STOPPED]  Service Not Activated
Cisco Unified Intelligence Center Serviceability Service[STOPPED]  Service Not Activated

admin: show network cluster
128.188.71.247 besnsuccx2.domain.com besnsuccx2 Subscriber contactcenterxpress DBSub authenticated
128.184.71.247 besnsuccx.domain.com besnsuccx Publisher callmanager DBPub authenticated using TCP since Fri Jun 19 01:21:02 2015
admin:

From the primary I see the following

admin:show network cluster

128.184.71.247 besnsuccx.domain.com besnsuccx Publisher callmanager DBPub authenticated
128.188.71.247 besnsuccx2.domain.com besnsuccx2 Subscriber contactcenterxpress DBSub authenticated using TCP since Fri Jun 19 01:20:52 2015

Server Table (processnode) Entries
----------------------------------
besnsuccx
besnsuccx2

Successful

admin: utils uccx database healthcheck

*******************************************************************************
*     healthcheck.sh v0.1 UCCX Health Report (Fri Jun 19 17:12:37 CEST 2015)      *
*******************************************************************************

Basic Configuration Information
-------------------------------------------------------------------------------
                         CONFIG                LOCAL                  REMOTE
-------------------------------------------------------------------------------
                       DBSERVER       besnsuccx_uccx         besnsuccx2_uccx                        not_set
              DB_SCHEMA_VERSION                    -                       -
                PRODUCT_VERSION                    -                       -
                    OLDEST_DATE                    -                       -
               CDS_PERCENT_FREE                    -                       -
               HDS_PERCENT_FREE                    -                       -
-------------------------------------------------------------------------------

*******************************************************************************
This is a single node setup. There is no remote DB to compare with.

Why is it saying this is a single node setup ?????? The communication is fine on the informix ports 1500, 1515 I checked with a sniffer capture

The second node is in the POLICY_INJECTED state so it has joined the cluster as I see in the cluster manager logs.

Can anybody tell me what might be the problem ?

What logs to check and how to verify that there is no network problem ? Are there any ports I need to verify that are open ?

Thanks!

Christos

[tonyperla]

Hi Christos,

Just making sure that your system is licensed for HA since there is no mention of it.  It is also strange that those services are all stopped on your secondary.  Do you see any appearance of the secondary when in the CCX Serviceability?  When installing UCCX on the second node, did you make sure that you said NO to it being the first node in the cluster?  

http://www.cisco.com/c/en/us/td/docs/voice_ip_comm/cust_contact/contact_center/crs/express_10_6/install/guide/UCCX_BK_C2956BDB_00_uccx-install-upgrade-guide-106/UCCX_BK_C2956BDB_00_cisco-unified-contact-center-express_chapter_010.html#UCCX_TK_IA00CE5...

Hope this helps

Tony

Please rate helpful posts

[christos.georgiadis]

Hey Tony,

Thanks for the reply. Yes I had the license but due to the fact that I made a config change it is now invalidated so I am with the demo. But when i see the license in the gui it still says HA enabled so I think I am ok. I have rebooted several times but no go.

In serviceability when I go to the service parameters I can see the second node but not in the control center.

Regards

[tonyperla]

I would get the licensing straightened out before you chase it too far.  When installing UCCX on second node did all of the communication tasks pass.  Can you reference the guide I posted to make sure you had the same experience?

Hope this helps

Tony

Please rate helpful posts

[christos.georgiadis]

Yes correct, I did not say it is the first node (I did many installation of UC software in my career so far) and also there was no error whatsoever during installation which means the communication to the first node was ok. Even the dbreplication is fine when I check from the cli.

I even installed it twice but with the same results. Now I am installing it on the same esxi host to rule out any connectivity issues.

[tonyperla]

I would get the licensing straighten out first.  It takes seconds for GLO to cut a new license or to rehost one.  I would try that before your next step you mentioned.

Thanks

[christos.georgiadis]

The installation is almost finished. i will check if I have the same issue and if yes then I will do what you say. It is indeed the next logical step

Thanks!

Christos

[christos.georgiadis]

Yes it is the same. I will get a new license and report back.

[christos.georgiadis]

No change with the new license. Still having the same issue

[andrew.chan.avnet]

I'm having the same issue after recently IP changes to both nodes.

Can you share the step/command so that i dont need to re-build the second nodes?

Thank you

[Chakshu Piplani · Accepted solution]

Hi Christos,

You are unable to see the second node because, the replication for the second node has not been initialized, that will began only after you login to CCX second node for the first time and go through the initial setup through web page, It will ask you if this setup is HA over LAN or WAN, ask you the credentials of appadmin login, and also go through activation of datastores, once thats completed only then you will see the second node in you dropdown when checking services.

For this to happen you need to login to second node using "Cisco Unified serviceability" credentials, (different that Cisco Unified "CCX" serviceabiility), these are the same credentials used for logging into RTMT tool.

You can use the CLI(SSH) to reset the credentials if you are unable to login.

Let me know if you need assistance on this. No need to keep rebuilding the server.

HTH

Chuck.

[richb1971]

Hey Chuck,

This was a life saver thank you! TAC spent hours playing with the CET tool before we read this!

Cheers

Rich

[richb1971]

So the Install and Upgrade guide is wrong and needs correcting. Can TAC correct this doc please!

Step 1 should read Cisco Unified Serviceability

Cisco Unified Contact Center Express Installation and Upgrade Guide, Release 10.6(1):

Add Second Node

Configure the IP address of the second node on the first node.

Procedure

---

Step 1	Log in to the Cisco Unified CCX Administration web interface of the first node.
Step 2	Choose System > Server.

[christos.georgiadis]

Thanks Chuck. That was exactly it! Figured it out by chance yesterday!