IP IVR v7 database restore fails on reinstall

[Clifford McGlamry]

Customer has a CCM6.1 cluster and two IP IVRs that are used by a UCCE system.  We are in the process of moving the customer systems over to UC on UCS.

Customer wants to change both host names and IP addresses on the systems when they go on the blades.  But as this is a production environment, we need to build a copy of the infrastructure off line to do all the upgrades, database migrations, etc. 

We have the callmanager cluster up and running  Restore into it worked fine.  The IP IVR's are running SR5.  Both are failing with the same issue. 

We load the software, apply the patch, and then start the initial configuration selecting "Restore" from the drop down box as the installation type (other choices are things like "first node", "single node", "add to cluster", etc.). 

It asks for the TAR file, and we point it at that, and it happily goes off and finds it.  It untars the file (you see it appear in the c:\svi directory).  It comes up and then tells you that it has this information (reads off the date stamp, the software level, etc.) and you're asked to confirm it.  When we hit the button to continue, it runs for about 10 seconds and then fails.  Both nodes exhibit the same behavior.  The screen tells me it can't connect to CallManager.  We know this isn't the case, but that's what it says (VERY long explaination why, but we do know it can).  Screen shot of the failure is attached along with the MCVD log.

It also tells me to check the MCVD logs, and the only thing where I see a failure is below:

892: Sep 10 18:08:40.587 EDT %MCVD-LIB_DATABASE-7-UNK:SQLServerAliasController reinit called
893: Sep 10 18:08:40.587 EDT %MCVD-LIB_DATABASE-7-UNK:unregisterArchiveRecoveredListener:removing archive recovery listener
894: Sep 10 18:08:40.587 EDT %MCVD-FILE_MGR-4-MASTER_RDS_NOT_FOUND:THREAD:MXXX_MGR_MGR_INIT2:Repository Datastore not found.: Reason=Repository Datastore - getting master failed.
895: Sep 10 18:08:40.587 EDT %MCVD-FILE_MGR-4-RDS_DETERMINATION_FAILED:THREAD:MXXX_MGR_MGR_INIT2:Repository Datastore cannot be initalized.: Exception=com.cisco.file.impl.DbManagerException: :could not get RDS Master Info from cluster
896: Sep 10 18:08:40.587 EDT %MCVD-FILE_MGR-4-EXCEPTION:THREAD:MXXX_MGR_MGR_INIT2:com.cisco.file.impl.DbManagerException: :could not get RDS Master Info from cluster
897: Sep 10 18:08:40.587 EDT %MCVD-FILE_MGR-4-EXCEPTION:THREAD:MXXX_MGR_MGR_INIT2: at com.cisco.file.impl.DbManager.register(DbManager.java:376)
898: Sep 10 18:08:40.587 EDT %MCVD-FILE_MGR-4-EXCEPTION:THREAD:MXXX_MGR_MGR_INIT2: at com.cisco.file.impl.FileManagerImpl.register(FileManagerImpl.java:1346)
899: Sep 10 18:08:40.587 EDT %MCVD-FILE_MGR-4-EXCEPTION:THREAD:MXXX_MGR_MGR_INIT2: at com.cisco.file.File.register(File.java:156)
900: Sep 10 18:08:40.587 EDT %MCVD-FILE_MGR-4-EXCEPTION:THREAD:MXXX_MGR_MGR_INIT2: at com.cisco.script.impl.ScriptManagerImpl.start(ScriptManagerImpl.java:1131)
901: Sep 10 18:08:40.587 EDT %MCVD-FILE_MGR-4-EXCEPTION:THREAD:MXXX_MGR_MGR_INIT2: at com.cisco.manager.Manager.start(Manager.java:130)
902: Sep 10 18:08:40.587 EDT %MCVD-FILE_MGR-4-EXCEPTION:THREAD:MXXX_MGR_MGR_INIT2: at com.cisco.manager.ManagerBridge.start(ManagerBridge.java:86)
903: Sep 10 18:08:40.587 EDT %MCVD-FILE_MGR-4-EXCEPTION:THREAD:MXXX_MGR_MGR_INIT2: at com.cisco.manager.impl.ManagerManagerImpl.startManagers(ManagerManagerImpl.java:3104)
904: Sep 10 18:08:40.587 EDT %MCVD-FILE_MGR-4-EXCEPTION:THREAD:MXXX_MGR_MGR_INIT2: at com.cisco.manager.impl.ManagerManagerImpl.access$2400(ManagerManagerImpl.java:196)
905: Sep 10 18:08:40.587 EDT %MCVD-FILE_MGR-4-EXCEPTION:THREAD:MXXX_MGR_MGR_INIT2: at com.cisco.manager.impl.ManagerManagerImpl$2.run(ManagerManagerImpl.java:895)
906: Sep 10 18:08:40.587 EDT %MCVD-SCRIPT_MGR-3-MGR_IN_SERVICE:Undefined mnemonic 'MGR_IN_SERVICE'
907: Sep 10 18:08:40.603 EDT %MCVD-LIB_DATABASE-7-UNK:IsPrimary for DS:CRS Historical Datastore is false on node:172.28.27.7
908: Sep 10 18:08:40.603 EDT %MCVD-LIB_DATABASE-7-UNK:IsValid for DS:CRS Historical Datastore is true on node:172.28.27.7
909: Sep 10 18:08:40.603 EDT %MCVD-LIB_DATABASE-7-UNK:IsEnabled for DS:CRS Historical Datastore is false on node:172.28.27.7
910: Sep 10 18:08:40.603 EDT %MCVD-LIB_DATABASE-7-UNK:isLocalNodeNonPrimaryAndInvalid returns false
911: Sep 10 18:08:40.603 EDT %MCVD-LIB_DATABASE-7-UNK:IsPrimary for DS:CRS Agent Datastore is false on node:172.28.27.7
912: Sep 10 18:08:40.603 EDT %MCVD-LIB_DATABASE-7-UNK:IsValid for DS:CRS Agent Datastore is true on node:172.28.27.7
913: Sep 10 18:08:40.603 EDT %MCVD-LIB_DATABASE-7-UNK:IsEnabled for DS:CRS Agent Datastore is false on node:172.28.27.7
914: Sep 10 18:08:40.603 EDT %MCVD-LIB_DATABASE-7-UNK:isLocalNodeNonPrimaryAndInvalid returns false
915: Sep 10 18:08:40.603 EDT %MCVD-LIB_DATABASE-7-UNK:IsPrimary for DS:CRS Repository Datastore is false on node:172.28.27.7
916: Sep 10 18:08:40.603 EDT %MCVD-LIB_DATABASE-7-UNK:IsValid for DS:CRS Repository Datastore is true on node:172.28.27.7
917: Sep 10 18:08:40.603 EDT %MCVD-LIB_DATABASE-7-UNK:IsEnabled for DS:CRS Repository Datastore is false on node:172.28.27.7
918: Sep 10 18:08:40.603 EDT %MCVD-LIB_DATABASE-7-UNK:isLocalNodeNonPrimaryAndInvalid returns false
919: Sep 10 18:08:40.603 EDT %MCVD-LIB_DATABASE-7-UNK:IsPrimary for DS:CRS Config Datastore is false on node:172.28.27.7
920: Sep 10 18:08:40.603 EDT %MCVD-LIB_DATABASE-7-UNK:IsValid for DS:CRS Config Datastore is true on node:172.28.27.7
921: Sep 10 18:08:40.603 EDT %MCVD-LIB_DATABASE-7-UNK:IsEnabled for DS:CRS Config Datastore is false on node:172.28.27.7
922: Sep 10 18:08:40.603 EDT %MCVD-LIB_DATABASE-7-UNK:isLocalNodeNonPrimaryAndInvalid returns false
923: Sep 10 18:08:40.603 EDT %MCVD-CVD-7-UNK:after synch SQLServerAliasController

Multiple folks looking at this, and we're getting no where.

Any ideas appreciated.

[Pablo Leiva · Accepted solution]

Hello Clifford,

As per the error "Fail to update CRS Config Datastore primary state in node 1', you can try enabling this manually and see if you can get passed the 33%

- Start> Run> cet

- Select the com.cisco.crs.cluster.config.ComponentConfig

- From the right pane double-click the CRS Config Datastore (node 1)

- Select tab com.cisco.crs.cluster.config.DBComponentConfig

- Check if the "Primary Datastore Flag" is set to false.

- If it is, try changing it to true, restart the CCX node manager and try the IP IVR restore again.

Hope that helps!

Pablo

[Clifford McGlamry]

That did it! The restore was able to successfully run to completion.

I think the root cause here was that the IVR was installed as a first node, instead of a stand alone node, even though it was just single node. Altering the Primary Data Store Flag to true was the solution.

Thanks!

[Pablo Leiva]

Yeah that makes sense. Glad to hear it worked!

[joesnyde]

Hi Clifford,

"Customer wants to  change both host names and IP addresses on the systems when they go on  the blades.  But as this is a production environment, we need to build a  copy of the infrastructure off line to do all the upgrades, database  migrations, etc."

Not sure if you realize, but changing the host name of a IPIVR is not supported, which is documented in the UCCX/IPIVR installation guide, page 3-7

http://www.cisco.com/en/US/docs/voice_ip_comm/cust_contact/contact_center/crs/express_7_0/installation/guide/uccx70ig.pdf

Regards,

Joe