RTMT Call Processing Node CPU Pegging Alerts

[paulmciver]

I am in the process of building a lab for some testing and have noticed quite early on some massive delays. A little more digging around and I have noticed that within RTMT there is a persistant error "CallProcessingNodeCpuPegging" Serverity Critical. Looking at the graph the CPU usage is spiking from nothing up to 100% fairly regularly.

If I look at the processes, there are a large amount going to 99% CPU

I'm sure I have missed something stupid but I have never come accross something like this and it's affecting the tests quite drastically.

[Manish Gogna]

Hi Paul,

First thing to check is if the CUCM version that you are running and the hardware platform that you are using, is it built according to MCS or UCS specs specified by Cisco. If yes, then we need to check if the CPU spike is happening at a specific time or randomly. If it is occuring randomly then we require detailed cucm traces as well as RisDC perfmon logs to check what is triggering the spike in CPU. If the alerts are happening at a specific time of the day then it could be due to some scheduled activity like DRS backup, MWI sync, CDR Load etc. Please check the output of the following commands to see if there are any core dumps or other resource issues

utils core active list

utils diagnose test

HTH

Manish

[isabelpgv1]

hello,

I'm having the same issue; and there is a pattern, it only happens at 00:06; this is the time the backup is running.   should I worry about this?, since only happen at this time when nobody is in the office and a backup is happening.  I would like to stop this alarm.

thanks,

[Manish Gogna]

Following link has some good explanation of this alert

http://www.cisco.com/en/US/products/sw/voicesw/ps556/products_tech_note09186a00808ef0f4.shtml

HTH

Manish

[paulmciver]

Hi Manish,

Thanks for this, have you got a a good guide for setting up the UCS box? Unfortunately this was all setup by somene who is no longer with the company so I have no real guide on how it should be configured?

This is certianly not a "time of the day" based issue as it is more constantly at 100% then not. I now have no devices at all registered and just going through a process of rebooting all nodes in the cluster.

Once all three node's are back up and left running for 10-15 minutes I will re-evaluate if that has made any difference.

[Manish Gogna]

Hi Paul,

The following guide contains a lot of relevant information about sizing and deploying UC applications on UCS.

http://docwiki.cisco.com/wiki/Unified_Communications_in_a_Virtualized_Environment

HTH

Manish

[Manish Gogna]

Hi Paul,

Is the issue taken care of after the reboot?

Manish

[paulmciver]

No, unfortunately not. At the moment I have taken this specific build and rebuilt it once on the same ESX box and it was the same, on another ESX box that has other call managers running ok and again the CPU spiking is present immediatly after install.

I also just fired up an older CUCM that was running on that ESX and it was completely fine so it certainly seems to be something wrong with that version. I am going to get a different release downloaded and fire that up, I have wasted far too much time trying to figure out if this was ESX or not that I just need something working.