UCM 6 on MCS to UCM 9 on UCS

[Brian Houston]

I'm planning an upgrade of UCM6 on MCS platforms to UCM9 on UCS and I'd appreciate any insight on the best method of migration.

Its a 7 node UCM cluster supporting 8500 phones, integrated with a 2 node Unity Connection cluster and full blown Contact Center Enterprise. Unity Connection will also be upgraded to version 9 and Contact Center Enterprise will be upgraded by a 3rd party.

My focus rught now is on how best to migrate phones and users across to the new UCM9 cluster on UCS. I think that a "big bang" cutover of allphones/users at once would be impractical and very risky - unless someone can advise me differently of course. My current plan is as follows:

• Build the new UCM9 cluster on the UCS platforms. (The new cluster will have new IPaddresses and will co-exist with the production cluster.)

• Upgrade all phones on UCM6 production cluster to latest firmware supported by UCM9.
• Using a spare MCS server in our lab, build UCM with the same UCM6 version as the production cluster. (Use same hostname, IP address, security password etc at this point)
• Take a DRS import from the UCM6 production cluster.
• Import the production DRS into the lab UCM6. I now have an exact copy of the production Publisher. A change freeze would now be required on the production cluster, which is not ideal but unavoidable as far as I know.
• Change IP addresses and hostnames of the lab UCM6 cluster.
• Upgrade in stages from UCM6 (via 7 and/or 8) to UCM9 on the lab MCS.
• Take a DRS export from the newly upgraded lab UCM9

• Import the lab UCM9 DRS into the newly built UCM9 on UCS. I now have a UCM9 copy of the existing UCM6 cluster (with the obvious exception of IP addresses and host names)
• Create an inter cluster trunk between the production UCM6 cluster and the new UCM9 cluster.
• Configure call routing between the two clusters.
• Perform phased migration of phones and users from the productionUCM6 cluster to the UCM9 cluster.

These are my first thoughts on this, never having performed such a large upgrade of migration in the past. I'd welcome any feedback on Cisco best practices or anyone's own experience.

[Jonathan Schulenberg]

To save one of the TAC engineers saying this: the upgrade on-the-side process with spare hardware is entirely unsupported by TAC. Whether it should be or not is a debate for another day; at the moment they will tell you to upgrade the real cluster.

With that out of the way the real problem here is that your on-the-side MCS cluster is missing all of the subscriber nodes. There has been some debate recently on this topic but the documentation is very clear. Your DRS backup will fail to complete without the subscriber nodes online at the time of backup. Additionally, without the DRS restore operation on each subscriber it will appear as a new node within the product database and other behind-the-curtain configurations. The whole cluster needs to be backed up, restored (temporary MCS), upgraded, backed up, and restored (UCS) in this scenario.

Create an inter cluster trunk between the production UCM6 cluster and the new UCM9 cluster.
Configure call routing between the two clusters.
Perform phased migration of phones and users from the productionUCM6 cluster to the UCM9 cluster.

This seems like a horrible idea; you'll get stuck midway between clusters and never finish the migration. Additionally, CTI applications such as Jabber/CUPC or UCCE will not be able to control phones on both clusters. Stick to having one cluster and upgrading it; not spawning a clone.

Also, UCCE is rather version-locked to the CUCM version. Unless you have an ICT to a seperate CUCM cluster for UCCE you must work with the partner who will be doing that upgrade. You two will need to coordinate your actions to say the least.

If you feel that upgrading straight to 9.0 is unrealistic in a single change window you should look at splitting it into two: taking the cluster (and possibly UCCE if it's using the same CUCM cluster) to 7.x or 8.x one weekend, then to UCS and 9.0 a second weekend.

Please remember to rate helpful responses and identify helpful or correct answers.

[Gordon Ross]

I agree with Jonathan. Your plan is horribly complicated. The more complicated the plan, the more likely it is to go wrong.

Upgrade to 8.6 on your MCS boxes, then look to move to CUCM 9 on UCS.

GTG

[Brian Houston]

Thanks Jonathan for your feedback; its much appreciated.

I take your points with regard to the complexity of migrating users across from the production cluster to the new "cloned" cluster and if I can avoid this, I most certainly will. Given the age of the production servers, I admit it hadn't occurred to me that I could perform the upgrade on the them. I've checked the compatability of the servers and it appears that they are Type3 and I can perform a bridged upgrade, albeit a couple of upgrade steps will be involved. With this in mind, the cleaner solution may be along the following lines:

• Upgrade phone firmware on existing 6.1(3b)SU1 cluster.
  
• Upgrade  servers from 6.1.(3b)SU1 to 7.1(5b)SU5. Version 7.1(5b)SU5 can run in production mode on the servers therefore downtime will be limited to duration of reboots.
  
• Upgrade from 7.1(5b)SU5 to 9.0(1), switch active partition so that 9.0(1) becomes the active partition- this can run only in bridge mode and therefore will incur significant downtime during DRS phase.
  
• Take 9.0(1) DRS export.
  
• On new UCS platform running 9.0(1) , import DRS taken from 9.0(1) running in bridged mode on old server.
  
• Install new licenses.
  
  

This sounds clear cut enough but my concern  is that in upgrading,  I'm twice doing a flash cut of 8,500 phones. I can split this over a couple of weekends, 6.1 to 7.1 one weekend then 7.1 to 9.0 on another weekend.

In my experience this is a lot of phones and an afwul lot of unhappy users if things go badly during the cutover. Perhaps I'm worrying unduly and in fact this type of cutover for such user numbers is common practice?

Assuming there were problems, I'd have to have a solid plan in place to be able to rollback from 9.0 to 7.1.

[Jonathan Schulenberg]

In my experience this is a lot of phones and an afwul lot of unhappy users if things go badly during the cutover.

Agreed. At this scale I would expect that you have a non-production lab environment that you can install the upgraded software versions in and run through an test/acceptance plan before a change this critical gains approval. Every Cisco software build has defects; you'll want to test the features/products/scripts (in the case of UCCE) before you do it for real.

Upgrade from 7.1(5b)SU5 to 9.0(1), switch active partition so that 9.0(1) becomes the active partition- this can run only in bridge mode and therefore will incur significant downtime during DRS phase.
Take 9.0(1) DRS export.
On new UCS platform running 9.0(1) , import DRS taken from 9.0(1) running in bridged mode on old server.

In this case you could actually revert back to 7.1 and reset database replication as soon as you take the DRS backup. This would keep the phones online longer. Once the 7.1 cluster is back up you could power off the 7.1 nodes one at a time, power up the 9.0 UCS node and do the restore. Example order:

1. Publisher
2. All TFTP/IPVMSA nodes
3. All backup call processing subscribers (assuming a clean 1:1 CMG design)
4. All primary call processing subscribers

Please remember to rate helpful responses and identify helpful or correct answers.

[Brian Houston]

Despite the large number of phones involved, I'm now leaning towards the upgrade process we've discussed. Thanks for your input.

A final question; are there any Cisco white papers or best practice documents available  for this kind of scenario?

[amansin74]

Dear Brian,

If your upgrade is success full Kindly let us know your experience,

I have more or less similar problem.

I had a question :

Can we upgrade half of subscriber today and half tomorrow (moving from 6.4 to 8.5 on MCS) and how will that effect out environment,

[Gordon Ross]

amansin74 wrote:
I had a question :
Can we upgrade half of subscriber today and half tomorrow (moving from 6.4 to 8.5 on MCS) and how will that effect out environment,

No. All servers in a CUCM cluster must be running the exact same version of software. Servers on different versions of software refuse to talk to each other.

GTG