CUCM 8.6.1 Refresh upgrade to 10.5 fails with Internal Error (Function: ipmReadNormalizedInputLine) [SOLVED]

[Gianstefano Monni]

Hi,

we are upgrading our CUCM 8.6.1 to 10.5.1 on VMWare Enviroment (vSphere 5.5 running on UCS). 
We are running this upgrade in a sandbox, in order to avoid any problem in the production cluster.

We followed the refresh upgrade guide ( http://www.cisco.com/c/en/us/td/docs/voice_ip_comm/cucm/upgrade/10_0_1/CUCM_BK_U4214F9D_00_upgrade-guide-cucm-100.pdf)

Pre-configuration steps

1. Take a clone of the running publisher and put it in the sandbox
2. Remove DNS servers (they are not available in the sandbox)
3. Change the NTP Server with a ntp that is recheable from the sandbox
4. Check that Gateway and NTP connectivity are OK for the CUCM

Pre-upgrade:

1. Run the cop files to update keys and increase disk space (log partition is 80% full)
2. Shutdown the CUCM VM and
   1. Increase disk space on the VM configuration
   2. change VM OS from RedHat5 (32bit) to RedHat 6  (64bits)
   3. change VM nic from flexible to VMNIX3
3. Restart the VM

Upgrade
1. mount the iso
2. connect to OS Administration web interface and perform the Software installation from CD/DVD
3. upgrade process works fine up to the post-installation task

During post-installation tasks (after completing the DB Load and Transforming Copy) the Upgrade process always fails fails with an internal error

IPM|Internal Error, File:ipm.c:2011, Function: ipmReadNormalizedInputLine(), "/usr/local/cm/script/cm-dbl-install RU PostInstall 10.5.1.10000-7 8.6.1.21019-1 /usr/local/cm/ /common/component/database /common/log/install/capture.txt " failed (1)|<LVL::Critical>

IPM|  end-of-session "Installing database component": 1109.369 secs.|<LVL::Info>

IPM|Close progress meter "Component Install"|<LVL::Info>

We also tried the intermediate steps:

- upgrade from 8.6.1 to 9.1 : worked

- from 9.1 to 10.5: Failed with the same error

- from 9.1 to 10.1: Failed with the same error

any help will be greatly appreciated, thanks
Stefano

[Aman Soi]

Hi Stefano,

Since you are able to upgrade successfully from 8.6 to 9.1 but is failing to 10.x , the only difference I could find for OVA template.

which OVA template u are using ?

regds,

aman

[Gianstefano Monni]

hi Aman

thanks for your message, but actually the upgrade from 8.6.1 to 9.1 is a L2 upgrade. the upgrade from 8.6.1 (or from 9.1) to 10.x is a Refresh upgrade (RU) because it changes the backend operatings system.

on both CUCM 8.6 and 9.1 the OS is Redhat 5 on cucm 10.x is redhat 6 - 64bits, so it is a major uprgade. The ova file that i am using is the same i use for 8.6 because it is an upgrade, but as per documentation, i changed the NIC type (to VMNIX3), the disk size and the VM OS (to Redhat 6).

kind regards

stefano

[Aman Soi]

Hi Stefano,

I could not find any more info related to this error.

we can wait for more replies if someone has faced this issue earlier or if u are stuck up in middle of upgradation, go ahead with opening of TAC .

regds,

aman

[Gianstefano Monni]

Dear Aman,

thanks for your message.

I found a reference here https://supportforums.cisco.com/discussion/12334191/upgrade-cucm-1051-fails (same error, same line of code)  and it seems to point to Dial Plan installation.

I removed the dial plan (from route filters, route pattern and uninstalled the dial plan) and re-tried an upgrade but it did not fix (failed with the same error)

thanks for your feedback,

stefano

[mutte jyothandranath]

We are involving TAC to investigate on the issue.

[Yorick Petey]

Hi Mutte,

I have the exact same error message while doing the RU from a 8.6.2 to 10.5.2.

What is the result of your troubleshooting with the TAC?

Thank you for your help!

Yorick

[Gianstefano Monni]

hi Yourick

yesterday we repeated the RU procedure with TAC, and they confirmed procedure is correct. We received the same error in subject, and now they are further investigating on the logs.

kind regards

stefano

[Gianstefano Monni]

Issue was related to Dial Plans, the logs showed

16:45:46.204 |   DBUtil::BlockCopyTable ### *ERROR* ###:  (diagnosis):  Bulk Data Migration for table availdialplan failed due to data constraint issue, (-971).

16:45:46.611 |   DBUtil::BlockCopyTable *

16:46:16.555 |*ERROR* Error fetching column information: [Informix][Informix ODBC Driver][Informix]Could not position within a table (informix.systables).

16:46:16.588 |   DataConverter_5to6::CopyTables *ERROR* BlockCopyTable failed to process table (availdialplan)

16:47:52.395 |   installFull *ERROR* Prior Cancel or Error Processing convert5to6()

Cisco TAC fixed by updating the primary key in the DB.

After the fix we were able to perform the RU as expected.

[Aman Soi]

thanks for update[+5]

regds,

aman

[Jose Ignacio Leite Neto]

Hello Gianstefano,

This have been two year now, but i'm getting-the same issue on a upgrade from 8.6.1 to 11.5

what was the final solution for this issue?

I know that your said that this was related to dial plan, may you provide further details?

[Jose Ignacio Leite Neto]

i found the solution my problem here

https://supportforums.cisco.com/discussion/12334191/upgrade-cucm-1051-fails

basically there is a SIP profile called "Standard SIP Profile for Cisco VCS "that is created during the upgrade (version 9 and up), if you have that already configured BEFORE  the upgrade, it will fail since its a duplicate entry.

the failure happens on item 6 of the database component installation. 83%

Change the name for the SIP profile to something else and try it again.

it worked for me.

[Aman Soi]

[+5] thanks for update.

regds,

aman

[Jaime Valencia]

FYI, the OS and vNIC change are to be done AFTER the upgrade, not before it.

[lior look]

Hey Jaime,

what do u mean before, because in your youtube you did it before upgrade!

https://www.youtube.com/watch?v=Ml_z4YUgsBw