topic Re: WARNING: Ungraceful shutdown detected >>CUCM v12.5 SU4 in Collaboration Applications

WARNING: Ungraceful shutdown detected >>CUCM v12.5 SU4

Abdelrhman Elmenyawe — Thu, 03 Jun 2021 10:06:55 GMT

I have a CUCM cluster with IMP and CUC, the site had a power outage.

and then all server nodes have shown these alerts

WARNING: Ungraceful shutdown detected - A rebuild of this node is highly recommended to ensure no negative impact (such as configuration or file system corruption). For rebuild instructions, see the installation guide.

Do I need to rebuild all nodes again, or is there a workaround to eliminate these alerts?

Re: WARNING: Ungraceful shutdown detected >>CUCM v12.5 SU4

Nithin Eluvathingal — Thu, 03 Jun 2021 10:08:31 GMT

We used to have Ungraceful shutdown in our data centers, But never seen such a warning on our CUCM 12.5 SU1.

I Would recommend to contact TAC to verify your system and confirm everything is good.

Re: WARNING: Ungraceful shutdown detected >>CUCM v12.5 SU4

Jaime Valencia — Thu, 03 Jun 2021 13:28:32 GMT

See here

Enhancement for GUI and CLI Notification of Unexpected Shutdowns in CVOS

CSCva87117

Re: WARNING: Ungraceful shutdown detected >>CUCM v12.5 SU4

Vaijanath Sonvane — Thu, 03 Jun 2021 13:43:44 GMT

This is something new which is added in new versions. Recently I had unity connection publisher go down unexpectedly. After I powered it on, GUI didn't show this warning message but CLI did. I performed system verification and tests to make sure all the features that are configured working or not. All the database replication, diagnose test, cluster status etc. looked good and so far I did not hear any issues from users. In your case, make sure all the features and functionalities are working or not. And as @Nithin Eluvathingal mentioned, open Cisco TAC case if you want more clarification or recommendation from TAC Team.

Re: WARNING: Ungraceful shutdown detected >>CUCM v12.5 SU4

mHadi — Mon, 07 Jun 2021 10:09:57 GMT

hi dear @Abdelrhman Elmenyawe

if you have backup your server with Veeam-Backup-Solution,

i suggest you do delete currently cucm vm, and Restore your last backup,

this is best way to make sure your cucm will working properly again..

(HTH)

(if this helps, please rate..)

Re: WARNING: Ungraceful shutdown detected >>CUCM v12.5 SU4

Abdelrhman Elmenyawe — Sun, 13 Jun 2021 10:39:04 GMT

Thanks all of you for your answers,

I have opened a case with Cisco TAC and I will share his answers with you:

Q:But if the servers were working fine, could I continue with same installation without doing the process of rebuilding new servers?

A: No, you can not continue, by TAC it will be considered as unsupported.

Q:Also, can I clear these alerts that comes to GUI ?

A:no, not possible

Q:Last question, can I take a copy for the VMs and used them to be restored if the servers hit the ungraceful shutdown again ?

A: It is possible to take cold snapshot via vmware (after shutting down the system).

Re: WARNING: Ungraceful shutdown detected >>CUCM v12.5 SU4

jherlitzke — Thu, 22 Jul 2021 15:59:37 GMT

I have been noticing this with 3 of my customers now that have had un graceful shutdowns and I have to rebuild everything. Pushed them into finally replacing their UPS systems we have been recommending...

I wish this wasn't added, as I agree, everything is working as normal, but that message will never go away and then TAC won't support.

Good to read in other comments that this was new with the latest updates.

Re: WARNING: Ungraceful shutdown detected >>CUCM v12.5 SU4

Kathy N. — Thu, 22 Jul 2021 19:25:06 GMT

We have the same issue but TAC did NOT say it would not support us if we didn't rebuild the servers. We were told it is a message that was newly introduced in 12.5.1 SU4. Since rebuilding a system after a power outage has never been indicated previously and is a significant action to take when no problems appear to be present, we are not rebuilding our servers. We have an open enhancement request with Cisco TAC to provide a way to clear this alert and are living with the notice in the meantime.

Re: WARNING: Ungraceful shutdown detected >>CUCM v12.5 SU4

Jaime Valencia — Thu, 22 Jul 2021 21:50:20 GMT

There is a bug for a tool which would check the system and might provide the ability to remove the message

Tool to check for Database and File System integrity after Ungraceful Shutdown or Restart

CSCvy68211

Re: WARNING: Ungraceful shutdown detected >>CUCM v12.5 SU4

Kathy N. — Fri, 23 Jul 2021 13:05:46 GMT

Checked that link and it says a Tool is needed. The link above goes to a request to create a tool and says that currently there is no work around.

Re: WARNING: Ungraceful shutdown detected >>CUCM v12.5 SU4

jherlitzke — Fri, 23 Jul 2021 13:03:30 GMT

I looked up that bug, but it states there is currently no tool..

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvy68211

Re: WARNING: Ungraceful shutdown detected >>CUCM v12.5 SU4

jeremy.worden — Thu, 29 Jul 2021 17:08:09 GMT

FYI: I got the same error on our lab server running 14.0.1.10000-20. I just upgraded it to the latest beta, 14.0.1.11900-57 and the error message went a way.

Re: WARNING: Ungraceful shutdown detected >>CUCM v12.5 SU4

Nithin Eluvathingal — Fri, 30 Jul 2021 09:57:53 GMT

@jeremy.worden are we entitled to share such information on a public forum ? Hope you signed NDA when you applied for beta program.

Re: WARNING: Ungraceful shutdown detected >>CUCM v12.5 SU4

Adam Pawlowski — Mon, 02 Aug 2021 17:56:16 GMT

It has been indicated for Emergency Responder in documentation for some time, but the message hasn't been displayed.

The way I look at it is that enough problems have been caused to prompt them to add this message, not as likely that something in the architecture has changed to make it more susceptible to issues in case of a failure. Either way this seems like the sort of thing that would be an issue during a systems upgrade, and I'd rather rebuild the system when I have control over the timing of it.

I've had to do it before for ER, for support, so I'd rather just get it out of the way. It's also, at least for my org, a good way to review the DRF process, make sure it works, timing, etc.

Re: WARNING: Ungraceful shutdown detected >>CUCM v12.5 SU4

Adam Pawlowski — Mon, 02 Aug 2021 18:02:52 GMT

I don't believe Veeam would be supported, as a 3rd party backup solution.

Re: WARNING: Ungraceful shutdown detected >>CUCM v12.5 SU4

ziqex — Wed, 01 Dec 2021 15:20:21 GMT

Hi Kathy, what version did you upgrade to? Thanks

Re: WARNING: Ungraceful shutdown detected >>CUCM v12.5 SU4

RichardBooth3236 — Thu, 02 Dec 2021 20:21:45 GMT

I had this too. There still is no TAC tool to check for database issues.

The fix was do backup of Publisher (with error displaying)

Rebuild Publisher and Restore from backup and sync Subs - all clear

Re: WARNING: Ungraceful shutdown detected >>CUCM v12.5 SU4

Azuka Ihonor — Thu, 30 Dec 2021 12:30:16 GMT

Hi Richard, I am in a similar situation at my place of work where the CUCM Pub, IM&P Pub and a CUCM Sub all display the ungraceful shutdown following a data center engineer who tripped the power to the UCS Server.

We use Single Sign On (SSO), LDAP Syncronisation and all other services that requires the Tomcat Certificate and ADFS metadata.

Would you happen to know if the DRS restore will also restore the old Tomcat certificates. Basically I am trying to avoid creating a new metadata for ADFS. Your response or anyone from this thread will be greatly appreciated

Re: WARNING: Ungraceful shutdown detected >>CUCM v12.5 SU4

Adam Pawlowski — Thu, 30 Dec 2021 13:52:08 GMT

Yes, DRS backs up certificates, so they will come back when you restore.

Your metadata shouldn't change either unless you click the button to regenerate (node order might change in multi-node cluster, or at least it did for me :()

Re: WARNING: Ungraceful shutdown detected >>CUCM v12.5 SU4

Azuka Ihonor — Thu, 30 Dec 2021 14:02:00 GMT

Thanks Adam, Much appreciated.

One more question, I am planning on the following

1. Back up the cluster (including the nodes with the "Ungraceful Shutdown Message"

2. Shut down the publisher VM, build another publisher VM and do a fresh install.

3. Bring the new pub up to the same version (including cop files and device packs)

4. Restore to the new Publisher from the DRS backup.

Question 1, do I restore the Pub from Pub or Pub from a Sub without the message

Question 2, When building the new Pub, would I need to clone the MAC address of the old Pub ( I am concerned about smart licensing)

Thanks once again for your response.