cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
11870
Views
20
Helpful
1
Comments
Joel Monge
Cisco Employee
Cisco Employee

This is a basic guide on the initial workflow one can follow to troubleshoot a process down situation focused mostly on Windows.

 

 

Basic Workflow

 

1. Check your Windows startup services:

 

CiscoWorks ANI database engine: Manual
CiscoWorks Daemon Manager: Automatic
CiscoWorks RME NG database engine: Manual
CiscoWorks Tomcat Servlet Engine: Manual
CiscoWorks Web Server: Manual
CWCS Cmf database engine: Manual
CWCS rsh/rcp service: Automatic
CWCS syslog service: Automatic
CWCS tftp service: Automatic
DFM dfmEpm database engine: Manual
DFM dfmFh database engine: Manual
DFM dfmInv database engine: Manual
IPM NG database engine - Manual

 

These should always remain this way and it is best to avoid stopping/starting any of these directly from the Windows GUI.  To restart the LMS daemons always use:

 


 

net stop crmdmgtd

net start crmdmgtd

 

2. Make sure DEP is set to Windows Essential Programs only (right click My Computer > Advanced > Performance Settings > Data Execution Prevention) and your anti-virus has an exclusion for the LMS install directory and all its subfolders.

 

3. Make sure you have enough SWAP space assigned as per the LMS documentation.  For most LMS Windows installations, 4GB is enough, but it can go higher.  This is specially important in Windows 2008 since it will default to automatic SWAP managing.

 

4. Use the DOS pdshow command to determine exactly where the problem is.  To understand the processes and what they do, you can download the following LMS Daemon Cheat Sheets.

 

 

Understanding the LMS processes

 

The first thing to check is the process dependencies.  For example, if Tomcat is down, a multitude of other processes will be down since they are dependent on Tomcat.  If this is the case, you want to focus on troubleshooting why Tomcat is down.  Next step is to check the log files for the particular process and ports.  To check the port availability one can use:

 

netstat -anob

 

This will produce such an output:

 


 

Active Connections

 

  Proto  Local Address          Foreign Address        State           PID

  TCP    127.0.0.1:9007         0.0.0.0:0              LISTENING       6060
  [tomcat.exe]

 

  TCP    127.0.0.1:9009         127.0.0.1:1768         ESTABLISHED     6060
  [tomcat.exe]

 

If a different application other than Tomcat is listening to ports 9007 and 9009, this would be an indication of a problem.  Port conflicts are usually caused due to a third party application occupying ports LMS needs or security software.  One common case is CSCte49301, where a Windows patch causes a port conflict between RM and EssentialsDM.

 

Next step is to enable debugging if the process has a debug option and restart the daemons to collect the log files.  It might be a good idea to clean the logs for easier viewing using:

 

logrot_trunc C:\Progra~1\CSCOpx\MDC\tomcat\logs\stdout.log

 

One can also restart only the process for troubleshooting purposes to get output on the logs:

 


 

pdterm tomcat

pdexec tomcat

 

Some processes also have a configuration file which one can check to see if there is any corruption or any changes that might be impacting.

 

 

Common Issues

 

  • Port being used.  As already mentioned, use the netstat -anob to verify this and suspect third party apps or security software.  Windows Event Viewer and process logs can also be helpful.  In the syslog.log one could find such errors:

 

System Error: The Daemon Manger cannot bind to port:42340

 

          This could be an indication of CSCtd18854 which affects Windows 2008 installations.

 

  • Hostname changed.  To verify this, one can check the hostname using the DOS command: hostname.  Then check the NMSROOT\MDC\etc\regdaemon.xml file to make sure they match.  Look under the <Name>HostName</Name> tag.  This is not case sensitive.  To properly change the hostname in LMS one should use the below steps:

 

    • Stop daemons: net stop crmdmgtd.
    • Go to NMSROOT\bin in and issue: perl hostnamechange.pl -ohost <oldhostname> -nhost <newhostname>.
    • Start daemons: net start crmdmgtd.

 

 

  • Security settings changing casuser permissions.  Try resetting the casuser:
    • Stop daemons: net stop crmdmgtd.
    • Now go to NMSROOT\setup\support\, and type: resetCasuser.exe.
    • Choose a password that applies to your server password policies.
    • Start daemons: net start crmdmgtd.

 

     If this is required for the daemons to start, it means something or someone is changing the casuser password in the SAM database or the security policies.  This should be checked with your Windows admin.

 

  • Certificate expired or bad.  Re-create with these steps:

 

    1. Stop daemons: net stop crmdmgtd.
    2. Delete all the server.* files in NMSROOT\MDC\Apache\conf\SSL.
    3. Go to NMSROOT\MDC\Apache and issue: perl configssl.pl -disable.  Then: perl configssl.pl -enable.
    4. Fill out form and restart daemons: net start crmdmgtd.

     

    NOTE: NMSROOT stands for your installation path.  By default this would be C:\Program Files\CSCOpx.

    Comments
    Marvin Rhoads
    Hall of Fame
    Hall of Fame

    Adding to Joels' guide for those needing to do the same from a soft appliance installation:

    For Soft Appliance version you can start by checking the status from the basic cli. If you need to restart a specific process and don't want or need to restart the whole application ("application start LMS"), you need to drop into the OS shell and issue the pdexec <process name> command.

    Here is an example (using pipe to include to abbreviate output and the pdexec command to show verbose process status at the OS cli):

    SecLab-LMS/admin# show application status LMS | i Apache

      Apache                Running normally                          5674

    SecLab-LMS/admin# shell

    Enter shell password :

    Starting shell...

    [SecLab-LMS/root-ade ~]# pdshow Apache

            Process= Apache

            State  = Running normally

            Pid    = 5674

            RC     = 0

            Signo  = 0

            Start  = 04/23/13 08:03:20

            Stop   = Not applicable

            Core   = Not applicable

            Info   = Web Server initialized ok.

    [SecLab-LMS/root-ade ~]#

    Getting Started

    Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: