CUCCX Services Auto Restarting

[silvakizir]

Hi to everyone.

We have a Standalone CCX version 10.0.1

All the services are automatically restarting every 30-40 minutes.

I'v done a full Restart of the CCX but nothing changed.

What can I do and does anyone know why is this happening??

[deepakcn]

Hi,

I believe you are running into defect ID: CSCuu97701
https://bst.cloudapps.cisco.com/bugsearch/bug/CSCuu97701/?referring_site=bugquickviewredir

Ways to verify this:

Run the below command and check the pattern of the reboot. You should find "Boot"events after every hour which is an indication that the server is rebooting on its own :

Command : file tail install system-history.log
Press CTRL + C to stop the tailing of the logs and come out.

Sample output :
07/06/2017 01:01:59 | root: Boot 10.0.1.11001-37 Start
07/06/2017 02:03:41 | root: Boot 10.0.1.11001-37 Start
07/06/2017 03:03:27 | root: Boot 10.0.1.11001-37 Start
07/06/2017 04:03:51 | root: Boot 10.0.1.11001-37 Start
07/06/2017 05:04:07 | root: Boot 10.0.1.11001-37 Start

Next collect System event viewer and application logs and look for the following :

Jul 6 05:02:05 wow-uccx daemon 5 auditd[7351]: Audit daemon rotating log files
Jul 6 05:02:08 wow-uccx daemon 6 auditd[7351]: The audit daemon is exiting.
Jul 6 05:02:13 wow-uccx daemon 5 auditd[7351]: Audit daemon rotating log files
Jul 6 05:04:08 wow-uccx syslog 6 syslogd 1.4.1: restart.

If you see the above lines in the logs, raise a TAC case to perform the WA for this defect.

Thanks and Regards,
Deepak Nair

[t-nms]

Dear Deepak,

             Now I face this issue but can't open case to CISCO TAC. You had any workaround to fix this issue?

[deepakcn]

Hi,

A TAC case needs to be opened as the workaround needs to be performed from the root. If you cannot raise a TAC case , a permanent fix would be to upgrade to a higher release preferably 10.6.1 SU3,11.5.1 SU1 or 11.6.

Thanks and Regards,
Deepak Nair

[t-nms]

Hi Deepak,
Thank You.
Can we upgrade without MA with Cisco?
T-NMS

[silvakizir]

Dear Deepack.

I can see the bellow output.

admin:file tail install system-history.log
12/18/2017 20:05:15 | root: Boot 10.0.1.10000-56 Start
12/18/2017 21:06:40 | root: Boot 10.0.1.10000-56 Start
12/18/2017 22:06:09 | root: Boot 10.0.1.10000-56 Start
12/18/2017 23:04:33 | root: Boot 10.0.1.10000-56 Start
12/19/2017 00:05:37 | root: Boot 10.0.1.10000-56 Start
12/19/2017 01:06:14 | root: Boot 10.0.1.10000-56 Start
12/19/2017 02:06:40 | root: Boot 10.0.1.10000-56 Start
12/19/2017 03:08:30 | root: Boot 10.0.1.10000-56 Start
12/19/2017 04:06:29 | root: Boot 10.0.1.10000-56 Start
12/19/2017 12:07:50 | root: Boot 10.0.1.10000-56 Start

In order to collect System event viewer and application logs I go to RTMT=>Trace & Log Central=>Collect Files=>Next=>I choose both the: Event Viewer-Application Log and Event-Viewer-System Log but I can't see the "Daemon" logs you wrote..

Am I doing something wrong?

Thank you.

[deepakcn]

Hi,

The logs you have collected are correct (Event Viewer-Application Log and Event-Viewer-System Log)

Now open the above logs and look for the following  events just before 12/19/2017 12:07:50

 wow-uccx daemon 5 auditd[7351]: Audit daemon rotating log files
wow-uccx daemon 6 auditd[7351]: The audit daemon is exiting.
wow-uccx daemon 5 auditd[7351]: Audit daemon rotating log files
wow-uccx syslog 6 syslogd 1.4.1: restart.

Also raise a TAC case for this issue as the workaround needs to be performed from the root. As a permanent fix, kindly upgrade to a higher release. Preferebaly 10.6.1 SU3,11.5.1 SU1 or 11.6.

Thanks and Regards,

Deepak Nair

[silvakizir]

Yes my friend I found the "daemon rotating log files" messages you describe.

Upgrading was the plan from the beginning but I wanted to see if I can fix it by someway.

Thanks a lot for the prompt replies.

[silvakizir · Accepted solution]

Dear all.

The problem was solved by upgrading the CUCCX.

In fact when I tried to upgrade the current version it crashed so I had to do a clean install again, restore it from the Backup and afterwards I did an upgrade.