UCCX RTMT Authorisation Failures - Bug CSCvh85822

[Christopher Jones]

Hi all, 

Since upgrading our CUCM/UCCX clusters to 12.5 we've been inundated with RTMT alerts from our UCCX cluster complaining that the smart license renewal has encountered an auth failure - "%UC_LIB_LICENSE-1-SMART_LICENSE_RENEW_AUTH_FAILURE:"

TAC have confirmed that this is only cosmetic and related to bug - CSCvh85822

The TAC engineers suggestion to stop receiving the RTMT alerts was to disable local syslogs on the UCCX publisher from generating alarms to RTMT however this would render us blind to any other legitimate syslog alarms and I'm not keen to do that. 
Short of a mailbox rule to delete these specific alerts for the time being does anyone have any experience with this or ideas as to how we could silence this specific alert from being generated until a fix is available?

Thanks
Chris.

[Roger Kallberg]

You did not write if your using an on-prem SSM or cloud SSM. If you're like us using an on-prem SSM I think that with the latest release, 8-202201, you wouldn't get the faulty alerts.

[Christopher Jones]

Hi Roger, 

Thanks for your reply. 

We're using the Cloud SSM. Any idea if it's possible to silence alerts for this specific issue? 

Thanks
Chris

[Frank den Haan]

Hi Chris,

I had the exact same issue as you. Since going to UCCX 12.5.1.11001-348 (and ES03 shortly after), I got 70-80 RTMT alerts daily regarding %UC_LIB_LICENSE-1-SMART_LICENSE_RENEW_AUTH_FAILURE. TAC suggested I disable them as well, but unfortunately, UCCX decided to include these under the "SyslogSeverityMatchFound" category rather than a separate Smart License alert category, so like you, I cannot disable them as I'd miss potentially critical alerts. I had to create a mailbox rule sending them to a dedicated folder to minimize the annoyance. Cloud SSM as well.

So I had another issue where beyond these RTMT alerts where the UCCX Licensing page suddenly wouldn't register/authorize with the Smart Licensing Portal. Turned out to be bug CSCwa92591. TAC had to get root access to the server and apply a certificate to resolve. Interestingly, the %UC_LIB_LICENSE-1-SMART_LICENSE_RENEW_AUTH_FAILURE alerts suddenly went from 70-80 daily down to 5-9 alerts daily. Not perfect, but better.

- Frank

[lleweyiss]

Just upgraded to UCCX 12.5.1.11002-481 (su2-ES04)  - This issues still exists ! Opened a TAC before I saw this post.  The BUG shows:

Workaround: - None - Authorization should be successful in consecutive attempts - Alternatively, one can manually invoke Authorization/Registration attempt.

The "should be successful in consecutive attempts"  ... not so much - RTMT alert emails 24/7 still continuing after 2 weeks of moving the licenses to Smart Licenses.  Does anyone see their RTMT alerts ever stop ?

Since all my licenses on the Cisco site show registered , not sure  I could 'Manually invoke Authorization/Registration"  for the UC_LIB_LICENSE .  Is this even possible?

[j.burwell]

Just upgraded to 12.5.1SU2ES04 and are hitting the same bug. Was told to just ignore the alerts or set up an email filter to send them to trash. Sounds like SU3 will be released early March. Not sure if it will fix this bug or not. We are also hitting Bug CSCwe21416 which auto logs agents back in from not-ready after they receive a direct call.

[Brian Carscadden]

Just ran into this bug. Many hours with TAC only to reach the same conclusion as the others here. Still no word from Cisco on a permanent fix.

[layla.abunejem]

Hitting the same bug here too

[richard-ferguson]

We have same issue on CCX 12.5.2 SU2 ES04 - very irritating. By contrast the WebUI always shows Authorized and Registered AOK. Frank den Haan's post about certificates is ringing bells: The IdenTrust Commercial Root CA 1 with Serial Number 0a0142800000014523c844b500000002 is in tomcat-trust but are there other root certs required as well?

We also get irritating seemingly unresolvable 'cosmetic' daily "Database not in sync on both nodes" that aligns with the daily DB purge schedule. And they started to occur out of the blue at similar time to the Smart Lic alerts. Cosmetic they may be, but they clutter up RTMT and mask real events.

[parram2.hhc]

I have a TAC case and got the following "workaround", I am about to test it, please make sure you do this via the UCCX RTMT (download RTMT  from UCCX and use that one, with a CUCM RTMT install in your PC the UCCX tab won't show up)

- Adjusted UCCX RenewAuthorizationFailure alarm threshold in RTMT

1. Open RTMT, goto system > Alert Central > click on the Unified CCX tab

2. Right lick on the RenewAuthorizationFailure alert and click on 'Set Alert/Properties...'

3. Click on Next twice.

4. Set the frequency to Trigger up to 3 alerts within 60 minutes.

* This will set so an alarm will only be raised if authentication fails 3 times within 60 minutes and can be adjusted accordingly to avoid unnecessary alerts being generated.

 In our case in the MIVR logs we could see that CCX sends the renew authorization request to the cloud, the cloud rejects it (no idea) CCX attempts again immediately and now the cloud does accept the authorization, but just that failure caused a notification to be sent via RTMT and also for our monitoring systems to trigger a ticket.

I think this should do the trick

[parram2.hhc]

Forgot to say that the TAC case indicated that this will be greatly reduced (the cloud is rejecting the connection of smartlicensing) when upgrading to 12.1.5 SU3 ES03.

[omemar]

Hello @Christopher Jones 

This alert is cosmetic and you can disable it from RTMT.