WARNING: Ungraceful shutdown detected >>CUCM v12.5 SU4

[Abdelrhman Elmenyawe]

I have a CUCM cluster with IMP and CUC, the site had a power outage.

and then all server nodes have shown these alerts

"

WARNING: Ungraceful shutdown detected - A rebuild of this node is highly recommended to ensure no negative impact (such as configuration or file system corruption). For rebuild instructions, see the installation guide.

"

 Do I need to rebuild all nodes again, or is there a workaround to eliminate these alerts?

[Nithin Eluvathingal]

We used to have Ungraceful shutdown  in our data centers, But never seen such a warning on our CUCM  12.5 SU1.

I Would recommend to contact TAC to verify your system and confirm everything is good.

[Jaime Valencia]

See here

Enhancement for GUI and CLI Notification of Unexpected Shutdowns in CVOS

CSCva87117

[Flemming Ortvald]

Any indication, on how the result of the "show tech dbintegrity" would look, if there actually is a problem?
That is, what should I search in the resulting log file?

[Vaijanath Sonvane]

This is something new which is added in new versions. Recently I had unity connection publisher go down unexpectedly. After I powered it on, GUI didn't show this warning message but CLI did. I performed system verification and tests to make sure all the features that are configured working or not. All the database replication, diagnose test, cluster status etc. looked good and so far I did not hear any issues from users. In your case, make sure all the features and functionalities are working or not. And as @Nithin Eluvathingal mentioned, open Cisco TAC case if you want more clarification or recommendation from TAC Team.

[mHadi]

hi dear @Abdelrhman Elmenyawe 

if you have backup your server with Veeam-Backup-Solution,

i suggest you do delete currently cucm vm, and Restore your last backup,

this is best way to make sure your cucm will working properly again..

(HTH)

(if this helps, please rate..)

[Adam Pawlowski]

I don't believe Veeam would be supported, as a 3rd party backup solution.

[Abdelrhman Elmenyawe]

Thanks all of you for your answers,

I have opened a case with Cisco TAC and I will share his answers with you:

Q:But if the servers were working fine, could I continue with same installation without doing the process of rebuilding new servers?

A: No, you can not continue, by TAC it will be considered as unsupported.

Q:Also, can I clear these alerts that comes to GUI ?

A:no, not possible

Q:Last question, can I take a copy for the VMs and used them to be restored if the servers hit the ungraceful shutdown again ?

A: It is possible to take cold snapshot via vmware (after shutting down the system).

[RichardBooth3236]

I had this too. There still is no TAC tool to check for database issues.

The fix was do backup of Publisher (with error displaying)

Rebuild Publisher and Restore from backup and sync Subs - all clear

[Azuka Ihonor]

Hi Richard, I am in a similar situation at my place of work where the CUCM Pub, IM&P Pub and a CUCM Sub all display the ungraceful shutdown following a data center engineer who tripped the power to the UCS Server.

We use Single Sign On (SSO), LDAP Syncronisation and all other services that requires the Tomcat Certificate and ADFS metadata.

Would you happen to know if the DRS restore will also restore the old Tomcat certificates. Basically I am trying to avoid creating a new metadata for ADFS. Your response or anyone from this thread will be greatly appreciated 

[Adam Pawlowski]

Yes, DRS backs up certificates, so they will come back when you restore.



Your metadata shouldn't change either unless you click the button to regenerate (node order might change in multi-node cluster, or at least it did for me :()

[Azuka Ihonor]

Thanks Adam, Much appreciated.

One more question, I am planning on the following

1. Back up the cluster (including the nodes with the "Ungraceful Shutdown Message"

2. Shut down the publisher VM, build another publisher VM and do a fresh install.

3. Bring the new pub up to the same version (including cop files and device packs)

4. Restore to the new Publisher from the DRS backup.

Question 1, do I restore the Pub from Pub or Pub from a Sub without the message

Question 2, When building the new Pub, would I need to clone the MAC address of the old Pub ( I am concerned about smart licensing)

Thanks once again for your response.

[Kathy N.]

@Azuka Ihonor - Cisco has created a fix for this issue that doesn't require a rebuild of the system.  It is scheduled to be implemented soon with bug ID CSCvy68211.   Here's a link to the information.

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvy68211?emailclick=CNSemail 

[Roger Kallberg]

In DRS restore you restore the Pub from the backup of your Pub. There is no need to clone the MAC address.

[jherlitzke]

I have been noticing this with 3 of my customers now that have had un graceful shutdowns and I have to rebuild everything.  Pushed them into finally replacing their UPS systems we have been recommending...

I wish this wasn't added, as I agree, everything is working as normal, but that message will never go away and then TAC won't support.

Good to read in other comments that this was new with the latest updates.