lab upgrade from 11.6 to 12.5(1) SU1 - post upgrade weirdness

[Clifford McGlamry]

Did a lab upgrade from 11.6 to 12.5(1) SU1.  Post upgrade, I'm seeing some strange behaviors.

IVR works okay (so far).  Agents can log in, but then it starts popping up a login dialog asking for user name and password.  You enter it, and it just keeps coming back.  If you just close out of it, you get an error message on the gadets that says "Failed to load the gadget.  Authentication endpoint is not reachable.  Please contact your administrator".  In other words, not terribly helpful.  UCCX can obviously authenticate, as the agent can log in.  But the agent cannot log in to CUIC (they could before).  

Tried deleting the agent and adding them back.  Nope....that doesn't work either.  

I would love it if someone had a suggestion that could point me at how to resolve this.  

[TomMar1]

Are you using Finesse?  Perhaps it is a browser certificate(s) you need to accept.

[Clifford McGlamry]

No, I'm very familiar with that one. This is an authentication problem between CUIC and UCCX

[TomMar1]

Finesse is the web based application that call center agents would use.  Is this only an issue using CUIC?  Can you get a screen shot of the pop up?  You might just need to click through those pop ups and accept/allow those browser exceptions the first time the agent connects. 

[jim-j]

---

@Clifford McGlamry wrote:

Did a lab upgrade from 11.6 to 12.5(1) SU1.  Post upgrade, I'm seeing some strange behaviors.

...

I would love it if someone had a suggestion that could point me at how to resolve this.  

---

I did a lab upgrade from UCCX 12.5.1 to 12.5.1 SU1 and I'm seeing the exact same thing.  Have you made any progress on this?

In my testing this issue only occurs with IE.  Chrome and Firefox both work correctly.

One other thing I've noticed, I can use IE to log into CUIC directly and all reports (historical and live data) work fine..  It's only loading live gadgets within Finesse that fails.

[Clifford McGlamry]

I have not, but I have had an idea.
If you do a new install, you MUST load the CUCM Tomcat cert chain (i.e. the AXL provider) from the pub onto UCCX as a tomcat trust and then reboot UCCX (cluster) before you can set it up. Nothing prompts you about this on an upgrade.
I'm thinking this may be the issue, but I still have to vet my idea. But that's what I'd try next.

[Clifford McGlamry · Accepted solution]

Ok....I have it working now.  Here's what I had to do:

1. Make sure your database replication is clean....if you have an HA box.

2. If you DON'T have an HA box, make sure you don't have one configured under system ==> Servers

3. Download the tomcat certificate from CUCM, and load it to the UCCX trust store as a tomcat-trust cert

Then run your upgrade.  My certs were self signed 2048 bit certs.  Hope this helps y'all!

[jim-j]

Things just got weirder for me.  I spent an hour this morning reviewing certs on my prod and non-prod clusters and didn't find any significant difference.  I then (without making any changes to IE or my UCCX cluster) tried signing in again and it worked without any issues!  My best theory on why it's working now is that that something in IE timed out (cache, cookie, setting, etc.) overnight which allowed it to work today.  Note I did clear IE's cache yesterday during my testing.

Actually, I did reboot my non-prod (problem) UCCX cluster at the end of the day yesterday (I actually shut both servers down, so they were both off at the same time so I could take snapshots in case I needed to unmuck changes).  I doubt this caused IE to start working, but it is a change in the environment.

[jim-j]

To continue the weirdness. I upgraded our production UCCX cluster over the weekend from 12.5.1 to 12.5.1 SU1. The upgrade went fine, but I ran into this IE issue again. Like last time I tried clearing IE's cache, but it didn't help. The last thing I did in IE was sign into Finesse as a different call center user, had the issue, signed back in as my original user and the problem had gone away. I doubt changing the user sign in helped, so I still have no idea what causes and resolves this. The two times I've encountered this it's only temporary, no one else at my company has ever experienced this, it doesn't stop the delivery of calls (just live gadgets don't work) and there's an easy workaround (use a different browser).