Unable to SSH to 8.6.2 pub after 21900 update

[Jeff Bankston]

I have a problem I have never seen before:

I did an upgrade of 8.6.2 base to 21900 patch level, went fine, rebooted fine, and browser into all functions fine without issue. But, when I tried to ssh into the cli, I got this wonderful message:

Command Line Interface is starting up, please wait ...

[Fatal Error] cliCommands.xml:4393:1: XML document structures must start and end within the same entity.

cannot find any commands in /usr/local/platform/conf/cliCommands.xml Cannot start

Short of a full cluster rebuild, is there anyway to fix this unless TAC has to root into it? Even that won't work - can't cli into it!

Thanks, Jeff

[Markus Schneider]

They might be able to rebuild the cliCommands.xml file (assuming that's what's corrupt) via the rescue disk.  I'd open up a TAC case and have them take a look before rebuilding.

[Jeff Bankston]

Naaaa, its toast - I pretty much debugged it myself and found that the patch was started on the subscribers before the pub was fully rebooted; looks like the subs came back online before the pub went down, and something happened to the pub since the pub was technically still running base 8.6.2a and the subs were already on 8.6.2-21900-5, the xml files must have gotten munged in a version mismatch.

I've seen this happen with Presence, never before with CCM, first time for everything I suppose.

Rebuild is in full swing, pub is already back online and working.

Thanks, Jeff