Unity Connection server 100% CPU

[eschroedorica]

I have a virtualized Unity Connection server running OS 8.6.2.20000-76. I've had it up and running for about 8 months now, and I keep having this problem:

After starting the server, the instance runs for a few days/weeks, then the CPU spikes to 100% utilization. The only thing that I can do to get it to drop back to normal is reboot or restart the service management process, which kills all services and then brings them back up, so it takes call processing down and is pretty much the same effect as rebooting. Then the server operates normally for a while, until CPU utilization suddenly spikes to 100% again a week or two later.

There was a doc that addressed this issue for OS 7.1.5 or so, that had to do with unsupported IMAP clients hitting the server, but I followed those instructions and killed the IMAP support processes to no avail.

Has anyone else seen this problem, or know how to solve it?

[AskQ2Forum_2]

Hi,

have you seen this regarding unsupported imap clients

http://www.cisco.com/image/gif/paws/110587/cuc_credential_error.pdf

Jaginder

[eschroedorica]

Yes, this is the doc that I had seen before, the unsupported IMAP client does not seem to be what is causing my issue. Restarting (or even entirely disabling) the Connection IMAP Server service has no effect, CPU utilization remains at 100%.

[Joel Lallemand]

Hi

We've got exactly the same problem, but with Unity Connection 9.1.2.11900-12. The CPU also spikes to 100% utilization for a few secondes, fall back to normal and spikes again to 90-100% and so on.

During the 100% CPU utilisation, active calls will be stopped and only the half part of the voice message will be transmitted.

We see this behaviour in RTMT but on CLI (show process using-most cpu).

[Manish Gogna]

Hi Joel,

For CPU spike issues it is important to identify if it is being caused at any specific time of the day ( like during a DRS backup, peak hours or any other scheduled activity ).

Check for any core dumps using 'utils core active list'

Check the output of the following commands a few times with a gap of 5-10 minutes

show process load cpu

show process using-most cpu 5

Please enable following micro traces look at the following link which describe how to enable such traces

http://www.cisco.com/en/US/docs/voice_ip_comm/connection/9x/troubleshooting/guide/9xcuctsg010.html#wp1085339

CuImapSvr (all levels)
CML (all levels)
CsMalUmss (all levels)
MTA (all levels)

Then Collect traces following traces via RTMT
Connection IMAP Server
Connection SMTP Server
Connection Message Transfer Agent

HTH

Manish

[Joel Lallemand]

Hi Manish

Thanks for your fast reply.

I've checked the core dumps - there were no files.
The CPU Spikes are constantly, that means every 1-2 minutes. I've seen now in CLI that the Tomcat Process generates the spikes:

admin:show process load cpu num 2
top - 07:43:35 up 4 days, 17:56,  1 user,  load average: 0.79, 1.37, 1.61
Tasks: 176 total,   2 running, 174 sleeping,   0 stopped,   0 zombie
Cpu(s):  4.9%us,  3.1%sy,  0.0%ni, 66.9%id, 24.9%wa,  0.0%hi,  0.1%si,  0.0%st
Mem:   4016548k total,  3846004k used,   170544k free,   103164k buffers
Swap:  2064280k total,     8280k used,  2056000k free,  1964148k cached
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 2073 tomcat    18   0 1932m 835m  21m S 87.8 21.3 247:16.92 tomcat
 6552 ccmservi  16   0  328m  40m  19m S  1.3  1.0  65:00.70 RisDC

This Server isn't active at the moment. We switched to the subscriber server (Stop taking calls in servicability).

I've enabled the micro traces, but in rtmt, after selecting the CUC Services (IMAP,SMTP, Message Transfer Agent) which System Services/Applications should i select?

Best regards - Joël

[Joel Lallemand]

Hi

We could solve this problem by installing the following patch:
ciscocm.cuc_912_ImapSrv.cop.sgn
It's important to restart the server after the installation.