10-26-2011 11:37 PM
Hi
I recently made a fresh install of LMS 4.1 and added all of our devices (about 400 devices). After configuring all jobs and services everything ran fine.
After a few days a guy from the server team called me and told me that the CPU-usage increased over the days. I made some investigations and found out which process is using CPU ressources. Whenever ANIServer is running and collecting data CPU is running at approx. 100%. First it takes only a few minutes to complete but after some cycles ist takes more and more time to complete, after a week it takes more than 4 hours. After restarting ANIServer process it takes again a few minutes and then the duration increases.
Is this normal behaviour or could it be a bug?
I've attached a screenshot which shows the cpu usage.
Server specs:
Windows Server 2008 R2 64-bit
VMWare ESX
4 CPUs @ 2.93 GHz
6 GB RAM
Kind regards
Dominik
11-07-2011 11:24 PM
Can anyone help me?
11-09-2011 11:28 PM
to find out what is causing the issue it is necessary to get the processes consuming CPU. You can use "Process Explorer" from Sysinternals / MS Technet to get details on each running process:
the sysinternals suite (with a lot more usefull tools):
http://technet.microsoft.com/de-de/sysinternals/bb842062
or Process Explorer alone:
11-10-2011 12:37 AM
Hi Martin
I made two screenshots:
1 Process Explorer
2 Process Management section from LMS
CPU-usage raised after manually starting data collection.
11-10-2011 04:01 AM
DataCollection is under control of ANIServer, and it is in deed a CPU and RAM consuming process. This alone should not be a problem.
The first graph you posted is missing the description of the x- and y-axis. So I do not see how often the CPU spikes appear and what the numbers on the right side mean.
You said, your problem ist that the CPU usage is constantly rising. You should see this if you (in process explorer) right click on the process you mentioned to bring up the "Properties" view or if you expand the general view to show also CPU history for each process.
Let Process explorer open for a while so you see if there is one process which does not free up all the resources it takes in the first instance.
11-10-2011 06:46 AM
Hi Martin
I've got a better graph showing utilization over more than one week.
Data collection is scheduled every eight hours, so every eight hours cpu uses resources. The last time I had to kill ANIServer data collection was running for about 6 hours or something like this.
11-10-2011 07:21 AM
Hmm... I searched in this forum and I found some interesting discussions about DEP. Could this problem be triggered because I haven't disabled DEP for LMS?
11-10-2011 07:55 AM
DEP usually makes problems during installation. Also processes and tasks could fail because of DEP. I wouldn't have brought DEP in relation with your problem but you should disable it anyway - you will see if it helps.
I am still thinking about what could be the reason for the permanent increase of CPU usage. In fact, I saw DataCollection consuming less CPU and RAM for the first run after dmgtd restart. I think it is because subsequent collections look for any changes in regard to previous runs.
What are the hardware specs of this server.
11-10-2011 11:34 PM
HI Martin
Till yesterday I had following specs:
Windows Server 2008 R2 64-bit
VMWare ESX
4 CPUs @ 2.93 GHz
6 GB RAM
Now I have only 2 CPU. I had to give two CPUs back because of this problem. But it doesn't matter, the problem occurs with two or four CPUs.
11-11-2011 01:43 AM
are the CPUs strictly assigned to the server or is there a dynamic option active?
How many devices are managed with this server?
Check if the pagefile is statically assigned? It should have a size of 12 GB (min).
If you have to change the pagefile settings also, you'll have to reboot the server. Always stop dmgtd first (net stop crmdmgtd), wait until this is finished, now reboot the server.
Also check if antivirus software is running on the server? If yes, make sure that CSCOpx directory and the location to where the database backups are written are excluded from on-access scanning.
Can you provide the ani.log?
11-11-2011 02:15 AM
11-11-2011 03:43 AM
It seems that you have port-channels configured. There is a bug first found in LMS 4.0.1 by Marvin concerning data collection and port channels (BugId CSCto06189). The log entries looks very similar, though there are other errors which shouldn't be there...
see this thread:
https://supportforums.cisco.com/message/3313091#3313091
I cannot find the bug mentioned in the release notes for LMS 4.1 not as still active, nor as resolved. A patchis available on CCO only for LMS 4.0.1:
You could open a TAC case along with the ani.log to confirm if this is the bug (what I assume) and to get an official patch.
...perhaps Joe could confirm it also...
11-11-2011 04:20 AM
Hi Martin
Thanks for your help. I'll check this with our account manager.
Have a nice weekend!
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide