Unity Connection refuses to upgrade from 8.0 to 8.6

Tracee Baddley · ‎03-28-2013

on our Subscriber. Publisher upgrades fine.

Our Publisher is a MCS7835-12-K9-UCB1A with 6GB of memory.

Our Subcriber is a MCS7835-13-K9-UCB2 with 8 GB of memory.

I installed the .cop file on both servers prior to upgrade.

The Publisher seems to have upgraded fine.

I followed the same process with the Subscriber and it APPEARED to upgrade.

Please see the attached document to see the perplexing issue that I am running in to.

Rob Huffman · ‎03-28-2013

Hi Tracee,

Have you tried re-installing 8.6 on the Sub? I'm just

wondering if you may have hit this restriction;

Caution

Do not start the upgrade on the subscriber server until after the upgrade of the publisher server is complete, or all Connection functionality on both servers will be disabled until the upgrade of the publisher server completes. In addition, the upgrade of the subscriber server will fail, and you will have to upgrade the subscriber server again

http://www.cisco.com/en/US/docs/voice_ip_comm/connection/8x/upgrade/guide/8xcucrug007.html#wp1070971

Cheers!

Rob

"Hours are like diamonds, don't let them waste
Time waits for no one, no favors has he"

- Stones

Tracee Baddley · ‎03-28-2013

Hi Rob, I have tried 3 times to install on the sub. I am wondering if I should just blow away both inactive partitions of 8.6 on both the Pub and Sub and start all over again.

The thing that confuses me is that the Pub runs fine on 8.6. I just can't cluster them, they both act like stand-alone servers with version mismatch and no database replication.

If I did have to remove the inactive partition, does anyone know how to do that?

Thanks and Cheers back to you! :

Tracee

yahsiel2004 · ‎03-28-2013

Tracee,

Can you place both the Pub and the Sub on version 8.0.3. Then either via RTMT or CLI verify that the dbreplication is ok. If the replication is not ok then you would need to fix the replication first and then re-upgrade afterwards.

Regards,

Yosh

HTH Regards, Yosh

Tracee Baddley · ‎03-28-2013

I have searched for two days to try and find out how to do that with Unity on RTMT with no luck.

From the CLI is appears that the replication is indeed bad:

From Pub:

admin:show perf query class "Number of Replicates Created and State of Replication"

==>query class :

- Perf class (Number of Replicates Created and State of Replication) has instances and values:

ReplicateCount -> Number of Replicates Created = 519

ReplicateCount -> Replicate_State = 3

From Sub:

admin:show perf query class "Number of Replicates Created and State of Replication"

==>query class :

- Perf class (Number of Replicates Created and State of Replication) has instances and values:

ReplicateCount -> Number of Replicates Created = 0

ReplicateCount -> Replicate_State = 4

Two questions:

Any idea on how to check this via RTMT?

What is the CLI command to repair the replication (and should it be done after work hours)?

Thanks,
Tracee

Rob Huffman · ‎03-28-2013

Hi Tracee,

If it were me, I would open a TAC case at this point to

resolve these issues before things get worse, but you also have

the opportunity right now to engage Aashish in this "Ask the expert" event;

https://supportforums.cisco.com/thread/2206238

Cheers!

Rob

"Hours are like diamonds, don't let them waste
Time waits for no one, no favors has he"

- Stones

Tracee Baddley · ‎03-28-2013

Thank you Sir. Calling TAC again now and also gonna ask Aashish.

yahsiel2004 · ‎03-28-2013

You will need to fix this before you upgrade. To fix this issue do the following:

Procedure

If the replication status is shown as 4, which indicates that the replication is not setup correctly, complete these steps:

Double check the current status of the publisher in order to ensure it can establish the agreement. For this you can export and provide the database report from http:///cucreports > System Reports > Unified CM Database Replication Debug or download the report from http:///cucreports > System Reports > Unified CM Database Status > Download report .
Re-initiate the agreement for each subscriber, then wait and verify the status. When the nodes start to communicate, you can reset the replication per node. In order to reset the replication in each subscriber, complete these steps:
1. Execute the utils dbreplication stop command on all subscribers.
  Note: Ensure that you had finished Step 2a on all subscribers and then only run the command mentioned in Step 2b on the publisher.
2. Execute the utils dbreplication stop command on the publisher.
3. Execute the utils dbreplication clusterreset command on the publisher.
  Note: This command resets database replication on an entire cluster. If there is replication issue with only one subscriber there is no need to execute the command. You may skip Step 2d and perform Step 2e.
4. If all subscriber servers need to be reset, issue the utils dbreplication reset all command.
5. At the publisher server, issue the utils dbreplication reset CLI command where is the hostname of the subscriber server that needs to be reset.

http://www.cisco.com/en/US/products/sw/voicesw/ps556/products_tech_note09186a00809643e8.shtml

After you fix the dbreplication and the status shows as 2 then re-upgrade the Pub and then the Sub.

In regards to viewing the dbreplication on RTMT go to CallManager>Service>Database Summary:

Hope this helps.

Regards,

Yosh

HTH Regards, Yosh

Rob Huffman · ‎03-28-2013

Hi Tracee,

Just as an FYI You can open a TAC case right from here @ CSC

https://supportforums.cisco.com/docs/DOC-13613

Cheers!

Rob

PS: +5 to my friend Yosh for his great tips here!

"Hours are like diamonds, don't let them waste
Time waits for no one, no favors has he"

- Stones

yahsiel2004 · ‎03-28-2013

Thanks Rob +5 for you as well for your much appreciated consitent high level support and guidance.

Yosh

HTH Regards, Yosh

Tracee Baddley · ‎03-28-2013

All of you guys are awesome! Thank you very much.

I was confused by some conflicting info from my Unity Publisher.

I'm still getting:

From Pub:

admin:show perf query class "Number of Replicates Created and State of Replication"

==>query class :

- Perf class (Number of Replicates Created and State of Replication) has instances and values:

ReplicateCount -> Number of Replicates Created = 519

ReplicateCount -> Replicate_State = 3

From Sub:

admin:show perf query class "Number of Replicates Created and State of Replication"

==>query class :

- Perf class (Number of Replicates Created and State of Replication) has instances and values:

ReplicateCount -> Number of Replicates Created = 0

ReplicateCount -> Replicate_State = 4

But if we look at the service reports, it says DB Rep Status is at 2 for both servers.

Rob Huffman · ‎03-28-2013

Hi Tracee,

That looks like the CUCM cluster perhaps

Can you show the following;

admin:show cuc cluster status

should look like this;

Server Name Member ID Server State Internal State Reason

----------- --------- ------------ -------------- ------

uc1 0 Primary Pri Active Normal

uc2 1 Secondary Sec Active Normal

SERVER ID STATE STATUS QUEUE CONNECTION CHANGED

-----------------------------------------------------------------------

g_ciscounity_uc1 100 Active Local 0

g_ciscounity_uc2 101 Active Connected 0 Jul 28 22:50:26

Cheers!

Rob

"Hours are like diamonds, don't let them waste
Time waits for no one, no favors has he"

- Stones

Tracee Baddley · ‎03-28-2013

Gladly sir!

From the Pub:

admin:show cuc cluster status

Server Name Member ID Server State Internal State Reason

----------- --------- ------------ -------------- ------

usxhqvuc 0 Primary Pri Active Normal

usxhqvuc2 1 Secondary Sec Active Normal

SERVER ID STATE STATUS QUEUE CONNECTION CHANGED

-----------------------------------------------------------------------

g_ciscounity_pub 100 Active Local 0

g_ciscounity_sub1 101 Active Connected 0 Mar 25 19:27:27

From the Sub:

admin:show cuc cluster status

Server Name Member ID Server State Internal State Reason

----------- --------- ------------ -------------- ------

usxhqvuc 0 Primary Pri Active Normal

usxhqvuc2 1 Secondary Sec Active Normal

SERVER ID STATE STATUS QUEUE CONNECTION CHANGED

-----------------------------------------------------------------------

g_ciscounity_pub 100 Active Connected 0 Mar 25 19:27:27

g_ciscounity_sub1 101 Active Local 0

admin:

These are the things that have me so perplexed!

Rob Huffman · ‎03-28-2013

I agree Tracee, that looks perfectly fine to me as well. I really

think the upgrade failure may be related to something else

altogether

There are a number of bugs that can hamper the refresh

upgrade process to 8.6 and I'm sure that TAC will be able

to help identify the issue with root access.

I wish I could be more help here, but I'm stumped.

Cheers!

Rob

"Hours are like diamonds, don't let them waste
Time waits for no one, no favors has he"

- Stones

Tracee Baddley · ‎03-28-2013

I have been trying to do this upgrade since Friday night so I am so befuddled and nervewracked by now that it's almost funny.

Let me check my processes with you guys as a sanity check as I prepare a fresh start:

I'm going to get the databases repaired and in status 2 by checking from the CLI.

I'm going to then reinstall the .cop file on both servers.

Both servers will be active and running at 8.0 with the Publisher as primary and the Subcriber as secondary.

I am then going to install 8.6 fresh on the Publisher. After it completes it will do a final reboot and be running at 8.6.

Do I need to leave it running at 8.6 as I start the upgrade on the Subcriber? Or do I need to switch running versions on the Publisher back to 8.0 and then start the upgrade to the Subsciber?

Last time I just let the Pub run at 8.6 while the Sub was upgrading. Do you guys think that might have been a mistake?

Thanks again for all of your help and generally helping me check my sanity.