Call Manager 11.5 CallManager Service Abruptly Stops

[Robert Shaw]

Hi Folks,

Having some problems with our Call Manager 11.5.1.12900-21.  Everything has been running fine since we updated around Summer of 2017.  Over the past couple of weeks we've had to setup nearly 300 Jabber Phones for staff working from home during lockdown.

Since then over the last few days we've been getting alerts to suggest that the CallManager Service abruptly stops on the Subscriber, this causes other alerts, such as 100% phones unregistered, Registered Media devices Reduced etc.  By the time I go to investigate everything appears to be registered back to the Subscriber, so the CallManager Service must be restarting.

We have two servers in the cluster the Subscriber does all of the call handling and phone registration.

It would been last rebooted in October when we applied the latest Daylight Saving patch so is due for it's 6 monthly reboot.  Other than having all of these additional Jabber phones on the go I can't see any reason for the box to be under that much more pressure.  A similar number of calls are still being handled, probably less in all honesty.

Could this be a sign of over-utilization or should we look at maybe a Patch Update?  Going to try a simple reboot tonight, but need a plan of action in case that doesn't work.

Any advice, greatly appreciated!

Thanks
Rob

[Scott Leport]

Hi,

If you download the RTMT plugin from
CUCM applications section on CUCM and install that may reveal a lot about what is happening. Do you know the call manager service is definitely restarting? CM Serviceability would tell you the uptime of said service. However in RTMT and then the Alert Central section will reveal a number of things / issues etc with the servers in your cluster, including any loss of phone registrations / services etc.

Good luck!

Scott

[Roger Kallberg]

Likely you are hitting one of many bugs that is around in the 11.5 version. I would recommend to upgrade to SU6 or newer if it exists.

To better understand your setup, are you only having the Call Manager service running on your subscriber? In a two node cluster that would mean you’d have no redundancy.

[Robert Shaw]

Hi Roger,

Thanks for your response.

No, we have the Call Manager Service running on both nodes.  I have now seen that when the Service stops on the Subscriber, everything flicks over to the Publisher for a few minutes while the Subscriber sorts itself out and transfers everything back.  So it does appear that the service is restarting again.

I gave the Subscriber a reboot last night so will monitor it over the next few days.  It does feel like a bug, but the system has been so stable for us over the past couple of years.  I just wondered if the sudden influx of Jabber phones would have had anything to do with it and maybe it needed some more resources.

Will take a look at deploying the latest Service Update in preparation.

Thanks for the advice!

Rob

[Scott Leport]

Hi Rob,

As I mentioned before, it would be worth opening up RTMT to find out what that says.

It will give you information on a number of issues which you may be having, e.g. if there is high utilization or memory issues. 

It would definitely flag up the CM service stopping.

Regards,

Scott

[Robert Shaw]

Hi Scott,

Thanks for your response.  Apologies for not responding sooner.

I have had a look through RTMT but all I'm seeing are the same alerts that have been emailed to me.  There are Crash Dump files available but I'm starting think along the same lines as Roger mentioned that it's likely a bug.

I checked all of the UCS Servers Performance tabs in VMWare and can see that the Subscribers CPU Spikes at the time of the crash but before that is running around the 25% mark.  Our Internal Expressway is running on the same physical box which is consuming more resources at the moment with the number of Jabber clients connecting externally.  The whole box hovers around 40-60% CPU, Memory around 30% so doesn't appear to be resource issue.

Thanks

Rob