PCD 14 SU2 - Migrate CUCM 11 to 14 - PCD not pulling PlatformConfig

[steeda]

Got a fun one here. Migration failed and I restarted the source VMs running 11. It failed because the nodes could not contact the publisher so they just stalled out forever. Why couldn't they contact the Pub? Come to find out hey had no domain name so couldn't resolve it. 

My cluster is DNS based, and DNS is fine. I discovered PCD is *not* getting the Domain Name during the Cluster Discovery task. I verified this by looking in the /export directory by SFTPing into the PCD. The Platformconfig XMLs have every setting except for the local domain section. The PCD Migrate task uses this file and some other others to create a Floppy Image used during install of the new VMs. It's the answer file. 

I went ahead an logged into another PCD I used for a previous migration, a 12.5 one. The /export Platformconfig XMLs *have* the local domain section. That PCD had no problem.

I have a TAC case open on this - they're clueless and provided no resolution to the issue or even a suggestion. SO - I am having to figure out a way to fix this myself instead. Fun.

I am wondering if there would be any issue if, prior to the Migration task, I manually edited these PCD-derived Platformconfig XMLs - adding the localdomain section back in - and then starting the task. I am hoping the correct Floppy Images will then be created for the migration.

Anyone hit this yet?

Here is the end of the PCD 14 files it pulls. IPs changed.

<LocalHostDnsPrimary>
<ParamValue>1.1.1.1</ParamValue>
<ParamDefaultValue>0.0.0.0</ParamDefaultValue>
</LocalHostDnsPrimary>
<LocalHostDnsSecondary>
<ParamValue>1.1.1.2</ParamValue>
<ParamDefaultValue>0.0.0.0</ParamDefaultValue>
</LocalHostDnsSecondary>
</PlatformData>

Here is the end of the files pulled by PCD 12.5 on a different migration:

<LocalHostDomain>
<ParamNameText>Domain name for this machine</ParamNameText>
<ParamDefaultValue>cisco.com</ParamDefaultValue>
<ParamValue>fake.com</ParamValue>
</LocalHostDomain>

I assure you DNS and domains are populated properly and working in the Source Cluster - PCD is not gathering the info.

Think I can just manually add that section?

[Nithin Eluvathingal]

Which node got failed ?

if the reverse and forward lookup doesn’t work PCD migration will fail 

what you see on the console of new node ?

[steeda]

All of them except the Publisher, since it knows where itself is

DNS is fine. Forward and reverse. Has been fine since 2012 when I changed the install to DNS based. 

The problem is the Platformconfig's pulled from the nodes by PCD using its cop files are not including the Domain Name - this causes the nodes to fail during install. 

[Robert Profit]

Hello @steeda 

For your PCD migration, were you modifying names or IPs? I know you can modify those settings to include DNS during the task creation. If not, then PCD should just use the source servers' hostname/ip/dns info. Very odd that it would not pull information from the source cluster.

I am assuming you have tried deleting and re-creating/re-discovering the source cluster?

I have done a few migrations from 11x to 14 with PCD 14 and have had no issues.

Would you care to share your case number?

V/R,

Rob Profit

[steeda]

No mods - same IPs. Not doing Network Migration, just Simple Migration.

Steps taken:

Reinstalled PCD 14 and applied SU2. Naturally, doing this means I also rediscovered the cluster. The same PlatformConfig.xml files then appear in the /export directory, missing the Domain name which is required during install otherwise the Subscribers cannot contact the Publisher since it is DNS based. Without the domain, they'll never find it. 

Case is 695052199. I've escalated it 3x since the 11th and currently have requested a new TAC engineer as the current one only messages me once per day at 5am when I am asleep.

[steeda]

Here is a pic of what PCD generates.

If I Discover the Cluster using PCD 12.5 - the platformConfig's come out clean WITH the domain. 14 and 14 SU2 do not include it. Successful installation is impossible without the domain name.

[steeda]

SU2 actually changes things up a bit, sorry, it has the section, just no population:

<LocalHostDomain>
<ParamNameText>Domain name for this machine</ParamNameText><ParamDefaultValue>cisco.com</ParamDefaultValue>
<ParamValue></ParamValue>

SSH into Pub:

admin:show network eth0
Ethernet 0
DHCP : disabled Status : up
IP Address : 10.x.x.20 IP Mask : 255.255.255.000
Link Detected: yes Mode : Auto disabled, Full, 10000 Mbits/s
Duplicate IP : no

DNS
Primary : 10.x.x.1 Secondary : 10.x.x.2
Options : timeout:5 attempts:2
Domain : edited.local
Gateway : 10.x.x.1 on Ethernet 0

All nodes have the exact same. Forward and reverse DNS works. PCD is not pulling the domain out and adding it to platformConfig. Logs show nothing.

So - think I just get away with editing those prior to the Migration task?

Attaching pic showing servers with DNS populated.

[steeda]

I manually edited the platformConfig XMLs, populating the Domain Name. 

I created a migration task with a Pause prior to Publisher being hit. This will let me inspect the virtual floppy PCD maps to the Publisher. I will be able to see if it retained the manually added Domain Name.

I will update. There is clearly no "user error" possible here. There is no "user operation" that will cause PCD's discovery to miss the clearly defined Domain Names on all nodes. TAC won't acknowledge it as a defect, super frustrating.

[steeda]

Did not work. PCD once again DELETES the domain:

<LocalHostDomain>
<ParamNameText>Domain name for this machine</ParamNameText>
<ParamDefaultValue>cisco.com</ParamDefaultValue>
<ParamValue/>
</LocalHostDomain>

[steeda]

Here is TAC in 2023. Attached.

[steeda]

No joke....on phone with TAC right now. He pastes some steps to root into PCD, mount the floppies, and edit them. Cool!!! Getting somewhere! Then he asks me to complete the steps. I said " ok...can you do the remote account and gain root? ". He doesn't know what I am talking about. I tell him over and over that Cisco TAC gains root....customers cannot. He says the customer must complete these steps.

I plead with him and say 'listen, the mounting the floppy from root is a great find....but only you can root into PCD using the remote support account. ' He puts me on hold and comes back and says " my subject matter expert says we cannot root into a customer's server , you must collect pcd logs and send us '

What planet am I on?

[Jonathan Schulenberg]

Good grief! From an escalation perspective, have you:

• Called that engineer’s manager
• Called TAC front line and insist on escalating to the duty manager. (They will deny this if you haven’t already gone to the manager and given them a chance.)
• Called your Cisco AM and imply that this is jeopardizing future business

That’s my strategy: make noise until I get someone competent.

[steeda]

Got absolutely nowhere with TAC. They seem to have a procedure as I said earlier to root into PCD and VI the FLP files. This would WORK!!!!! He insists his "SME" says TAC cannot root into customer servers. Giving up on that route. Cisco cannot support UCM. Got it.

Using Winimage and other older tools I cannot seem to delete the platformConfig.xml from the FLP and reinject a modified one - just doesn't work. 

I am going to try a new option in the hopes the problem is 14 "reading" CUCM 11.0 SU2. Tonight I am going to upgrade using OS Admin to 11.5SU11 in place on the old hosts and the CUCM 80GB VMs. After that's all sorted, I will see if PCD 14 can work properly, read the Domain Name, and upgrade to 14. Stay tuned!

[steeda]

Revved the entire cluster to 11.5 SU11 - same. Deleted cluster, esxi hosts, everything from PCD. Shutdown PCD, deleted the VM from disk. Installing brand new PCD 14 now. Will rev to SU2 prior to discovering the 11.5 cluster. If this doesn't work, next plan is to install PCD 12.5 and rev the cluster to 12.5. This will allow me to create the 110GB CUCM VMs and from there I can OS Admin upgrade to 14. Good lord....

TAC = no responses to emails all day. 0 help. My impression of today's TAC is all they do is "Google" their internal knowledgebase and case files. They have no understanding of the products or how they work. If they can't find a "Google" hit, they give up.

[steeda]

Fresh PCD 14 install, patched to SU2 - discover cluster, go to define migration cluster - PCD deletes Domain for CUCM nodes, keeps it for IMP nodes.

Next step trying PCD 12.5.

TAC has not responded in nearly 24 hours even with an escalation request.