Error with communicating with DCR Server

[mjaffal01]

Dears,

Am having an error with LMS v 4.2.5, this error is always appearing : 

" Error in communicating with DCR Server. For more details click here.
DCR Server may be down. Please start the DCR Server and then refresh the page."

And some other pages are appearing but with error dash board error that No data is available.

I have recently changed my DNS Server but all other monitoring tools are working fine with the new DNS except for LMS

i have double checked the server NIC config and it is a static IP with no DHCP

the result of the pdshow command is as follow:

Process= UPMDbEngine
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= UPMDbMonitor
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= Tomcat
State = Program started - No mgt msgs received
Pid = 33720
RC = 0
Signo = 0
Start = 6/23/2016 8:35:12 AM
Stop = Not applicable
Core = Not applicable
Info = Application started by administrator request.

Process= TomcatMonitor
State = Running normally
Pid = 27988
RC = 0
Signo = 0
Start = 6/23/2016 8:35:16 AM
Stop = Not applicable
Core = Not applicable
Info = Tomcat Server up

Process= Apache
State = Program started - No mgt msgs received
Pid = 12336
RC = 0
Signo = 0
Start = 6/23/2016 8:37:21 AM
Stop = Not applicable
Core = Not applicable
Info = Application started by administrator request.

Process= TopoServer
State = Program started - No mgt msgs received
Pid = 6784
RC = 0
Signo = 0
Start = 6/23/2016 8:37:25 AM
Stop = Not applicable
Core = Not applicable
Info = Server started by admin request

Process= RMEDbEngine
State = Program started - No mgt msgs received
Pid = 30632
RC = 0
Signo = 0
Start = 6/23/2016 8:37:27 AM
Stop = Not applicable
Core = Not applicable
Info = Application started by administrator request.

Process= Proxy
State = Program started - No mgt msgs received
Pid = 9976
RC = 0
Signo = 0
Start = 6/23/2016 8:37:31 AM
Stop = Not applicable
Core = Not applicable
Info = Server started by admin request

Process= NameServer
State = Program started - No mgt msgs received
Pid = 6728
RC = 0
Signo = 0
Start = 6/23/2016 8:37:31 AM
Stop = Not applicable
Core = Not applicable
Info = Server started by admin request

Process= NameServiceMonitor
State = Program started - No mgt msgs received
Pid = 12504
RC = 0
Signo = 0
Start = 6/23/2016 8:37:35 AM
Stop = Not applicable
Core = Not applicable
Info = Server started by admin request

Process= EDS
State = Running normally
Pid = 17624
RC = 0
Signo = 0
Start = 6/23/2016 8:37:39 AM
Stop = Not applicable
Core = Not applicable
Info = Initialization complete

Process= LicenseServer
State = Program started - No mgt msgs received
Pid = 29892
RC = 0
Signo = 0
Start = 6/23/2016 8:38:59 AM
Stop = Not applicable
Core = Not applicable
Info = Server started by admin request

Process= IpmDbEngine
State = Program started - No mgt msgs received
Pid = 19120
RC = 0
Signo = 0
Start = 6/23/2016 8:39:01 AM
Stop = Not applicable
Core = Not applicable
Info = Application started by administrator request.

Process= INVDbEngine
State = Program started - No mgt msgs received
Pid = 26184
RC = 0
Signo = 0
Start = 6/23/2016 8:39:07 AM
Stop = Not applicable
Core = Not applicable
Info = Application started by administrator request.

Process= FHPurgeTask
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= FHDbEngine
State = Program started - No mgt msgs received
Pid = 12088
RC = 0
Signo = 0
Start = 6/23/2016 8:39:13 AM
Stop = Not applicable
Core = Not applicable
Info = Application started by administrator request.

Process= ESS
State = Program started - No mgt msgs received
Pid = 4844
RC = 0
Signo = 0
Start = 6/23/2016 8:39:17 AM
Stop = Not applicable
Core = Not applicable
Info = Server started by admin request

Process= EssMonitor
State = Running normally
Pid = 5648
RC = 0
Signo = 0
Start = 6/23/2016 8:39:21 AM
Stop = Not applicable
Core = Not applicable
Info = ESS is fully up now !

Process= EventFramework
State = Program started - No mgt msgs received
Pid = 24708
RC = 0
Signo = 0
Start = 6/23/2016 8:39:27 AM
Stop = Not applicable
Core = Not applicable
Info = Server started by admin request

Process= PMServer
State = Running normally
Pid = 25720
RC = 0
Signo = 0
Start = 6/23/2016 8:39:27 AM
Stop = Not applicable
Core = Not applicable
Info = PMServer is up and running

Process= SyslogCollector
State = Program started - No mgt msgs received
Pid = 23564
RC = 0
Signo = 0
Start = 6/23/2016 8:39:31 AM
Stop = Not applicable
Core = Not applicable
Info = Server started by admin request

Process= EPMDbEngine
State = Program started - No mgt msgs received
Pid = 28728
RC = 0
Signo = 0
Start = 6/23/2016 8:39:34 AM
Stop = Not applicable
Core = Not applicable
Info = Application started by administrator request.

Process= EPMServer
State = Program started - No mgt msgs received
Pid = 6864
RC = 0
Signo = 0
Start = 6/23/2016 8:39:38 AM
Stop = Not applicable
Core = Not applicable
Info = Server started by admin request

Process= FHServer
State = Running normally
Pid = 19320
RC = 0
Signo = 0
Start = 6/23/2016 8:39:42 AM
Stop = Not applicable
Core = Not applicable
Info = FHServer started.

Process= diskWatcher
State = Running normally
Pid = 33556
RC = 0
Signo = 0
Start = 6/23/2016 8:39:42 AM
Stop = Not applicable
Core = Not applicable
Info = diskWatcher initialized OK.

Process= DFMMultiProcLogger
State = Program started - No mgt msgs received
Pid = 30500
RC = 0
Signo = 0
Start = 6/23/2016 8:39:42 AM
Stop = Not applicable
Core = Not applicable
Info = Server started by admin request

Process= DFMLogServer
State = Program started - No mgt msgs received
Pid = 5272
RC = 0
Signo = 0
Start = 6/23/2016 8:39:42 AM
Stop = Not applicable
Core = Not applicable
Info = Server started by admin request

Process= DFMCTMStartup
State = Administrator has shut down this server
Pid = 0
RC = 0
Signo = 0
Start = 6/23/2016 8:39:42 AM
Stop = 6/23/2016 8:49:53 AM
Core = Not applicable
Info = Server started by admin request

Process= DfmBroker
State = Running normally
Pid = 6468
RC = 0
Signo = 0
Start = 6/23/2016 8:39:42 AM
Stop = Not applicable
Core = Not applicable
Info = DfmBroker is running normally

Process= DfmServer
State = Running normally
Pid = 19960
RC = 0
Signo = 0
Start = 6/23/2016 8:39:46 AM
Stop = Not applicable
Core = Not applicable
Info = DfmServer initialized OK.

Process= DfmServer1
State = Running normally
Pid = 3352
RC = 0
Signo = 0
Start = 6/23/2016 8:39:46 AM
Stop = Not applicable
Core = Not applicable
Info = DfmServer1 initialized OK.

Process= DCRDevicePoll
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= CSSCPServer
State = Program started - No mgt msgs received
Pid = 12888
RC = 0
Signo = 0
Start = 6/23/2016 8:39:46 AM
Stop = Not applicable
Core = Not applicable
Info = Server started by admin request

Process= CSRegistryServer
State = Running normally
Pid = 20116
RC = 0
Signo = 0
Start = 6/23/2016 8:39:46 AM
Stop = Not applicable
Core = Not applicable
Info = CSRegistryServer is running in other process

Process= CSDiscovery
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= CmfDbEngine
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= CmfDbMonitor
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= DCRServer
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= CMFOGSServer
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= EssentialsDM
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= CAAMServer
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= ConfigMgmtServer
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= ConfigUtilityService
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= ICServer
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= EnergyWise
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= PMCOGSServer
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= IPMOGSServer
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= TISServer
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= DFMOGSServer
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= InventoryCollector
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= Interactor
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= InventoryCollector1
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= Interactor1
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= NOSServer
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= PTMServer
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= UPMProcess
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= EDS-GCF
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= jrm
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= 3871
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= 4605
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= 4606
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= 5561
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= 5588
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= 5589
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= 5592
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= 5593
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= 5595
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= 5605
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= 5606
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= 5615
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= CTMJrmServer
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= ChangeAudit
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= SyslogAnalyzer
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= DataPurge
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= IPMProcess
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= ANIDbEngine
State = Program started - No mgt msgs received
Pid = 15132
RC = 0
Signo = 0
Start = 6/23/2016 8:43:08 AM
Stop = Not applicable
Core = Not applicable
Info = Application started by administrator request.

Process= ANIServer
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= UTMajorAcquisition
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= MACUHIC
State = Running normally
Pid = 17488
RC = 0
Signo = 0
Start = 6/23/2016 8:43:12 AM
Stop = Not applicable
Core = Not applicable
Info = Started.

Process= UTLITE
State = Running normally
Pid = 10884
RC = 0
Signo = 0
Start = 6/23/2016 8:43:12 AM
Stop = Not applicable
Core = Not applicable
Info = Started.

Process= UTManager
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= VNMServer
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= WlseUHIC
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

Process= AdapterServer1
State = Program started - No mgt msgs received
Pid = 8420
RC = 0
Signo = 0
Start = 6/23/2016 8:43:12 AM
Stop = Not applicable
Core = Not applicable
Info = Server started by admin request

Process= AdapterServer
State = Program started - No mgt msgs received
Pid = 26388
RC = 0
Signo = 0
Start = 6/23/2016 8:43:12 AM
Stop = Not applicable
Core = Not applicable
Info = Server started by admin request

Please HELP !

Thanks,

[jossabor]

Hi

While checking at the pdshow attached we can see the following:

Process= DCRServer
State = Never started
Pid = 0
RC = 0
Signo = 0
Start = N/A
Stop = Not applicable
Core = Not applicable
Info = Not applicable,

So DCRServer is down as the error mentioned, this could be caused by several problems, most of them end up on a wrong server restart, just to take into account remember that every time you restart your server you first need to stop the LMS services otherwises this will happen, sometimes it will end in worst scenarios. For now to have a better look at the services you could use pdshow -brief, if you do that you will find several processes down, that could look scary but those are because of DCRServer down.

If this is like most of the times a bad server restart happens there are some files that will get corrupted, those are the ctmregistry and ctmregistry.backup, those files are located under

NMSROOT/MDC/tomcat/webapps/cwhp/WEB-INF/lib

Please go to that location and delete those 2 files, once that is done please restart the services:

/etc/init.d/dmgtd stop

/etc/init.d/dmgtd start

If after doing this and waiting about 30 minutes the issue still persist please regenerate the certificates as they have also might got corrupted, you can regenerate them by doing the following steps:

1)       Stop the daemons manager from CLI using:

/etc/init.d/dmgtd stop

2)       Backup the following files found in /opt/CSCOpx/MDC/Apache/conf/ssl and

then delete them:

server.crt

server.csr

server.key

server.pk8

3) Under CLI go to "/opt/CSCOpx/MDC/Apache/bin" and then run ConfigSSL.pl by typed the following

command on CLI.

/opt/CSCOpx/MDC/Apache/perl ConfigSSl.p -disable

/opt/CSCOpx/MDC/Apache/perl ConfigSSl.pl -enable

Then fill the certificate accordingly and restart daemons from CLI using net

start the daemon manager:

/etc/init.d/dmgtd stop

If after this the problem still persist please open a TAC case for deeper troubleshooting.