This Document is intended to share a brief detail on Ciscoworks LMS Syslog Architechture and how it works. The Image explains the how Syslog works in LMS and some basic information required to troubleshoot the syslog issues :
For any issues with Syslog process, we need to consider some of the following points:
> First of all please check if the Syslog message is being written to Syslog.log (windows) or Syslog_info (Solaris).
> If the syslog is not coming to the log file check the network & security policies and make sure the port of Ciscoworks LMS IP is not blocked for traffic.
> Check if Syslog process is running :
# crmlog in Windows. Check if process is started in Services (CWCS Syslog Service) and available in Task manager as a process (crmlog.exe).
#syslogd in Solaris. Make sure that syslogd is running by typing in ps -ef | grep syslogd, you should see the syslogd process returned.
> Check Proper permissions for casuser and casusers on syslog.log | syslog_info.
> Check if SyslogAnalyzer and SyslogCollector are up and running and bind to their default port. Use pdshow <process name> to see the detals of the process. Example : pdshow SyslogAnalyzer.
> In case if any other process/software is using the port, we can change the default port for SyslogAnalyzer (3333/tcp) and SyslogCollector (4444/tcp) to bind them to another available port number using the NMSROOT/bin/SyslogConf.pl script.
> Sometimes a excessively huge Syslog*.db may have issues, we can drop the Db Space and Data spaces can be dropped using the NMSROOT/MDC/tomcat/webapps/rme/WEB-INF/debugtools/dbcleanup/DBSpaceReclaimer.pl.
#NOTE: Dropping Syslog*.db Data Space will remove all the previous Syslog information from DB. It can be considered equivalent as re-init of Syslog Db.
@@ DEBUGGING OPTIONS @@
We may have to debug various processes depending on where we identify the issue. Following is the procedure to debug Syslog:
# Debugging Crmlog :
We have to turn the Debug level to 1 in registry setting in windows (Start>Run>regedit) atHKLM\SYSTEM\CurrentControlSet\Services\crmlog\Parameters. Output is written to Syslog_debug.log.
#Debugging SyslogCollector:
We have to modify Collector.Properties and edit DEBUG_LEVEL to DEBUG. Output is written to SyslogCollector.log.
#Debugging SyslogAnalyzer:
This can done via GUI Debug Settings for SyslogAnalyzer module. In RME admin setting in LMS 3.x or earlier andadmin>system>debug settings for LMS 4.x onwards. Output is written to SyslogAnalyzer.log.
#Debugging Syslogd :
svccfg -s svc:/system/system-log setprop config/log_from_remote = true (Solaris 10)
OR we can also start syslogd in debug mode, by using the following procedure:
- Stop syslogd by using /etc/init.d/syslog stop
- Start syslogd in debug mode by using:
/usr/sbin/syslogd -d > /tmp/syslogd_debug.txt 2>&1
- Trigger syslogs from a device and also using the following command:
Logger -p local7.info "test"
- Use Ctrl-C to stop syslogd in debug mode and collect the
/tmp/syslogd_debug.txt file
- Start syslog again normally by using /etc/init.d/syslog start
Hope this will be helpful while troubleshooting Ciscoworks LMS Syslog issues.
