LMS 4.2.3 - DCRServer stuck in state: Waiting to initialize

[mbilgrav]

Hey Forum,

One of my installation has gone stuck ... OVA install with SP3, has run great until now ...

I had problems with DCR, and restarted the daemons via Shell-CLI.

Did not start up again ...

Then reload via nomal-CLI

Did not start up again ...

Then I noticed this when issuing "show application status LMS"

..

DCRServer             Waiting to initialize.

..

and it is stuck in this status

in daemons.log i see this and similar:

03/Sep/2013 16:11:45:964 ERROR  ? ?  - URN_NOT_FOUND : urn "ogs_server_urn" : Not found !!
03/Sep/2013 16:11:45:964 ERROR  ? ?  - URN_NOT_FOUND : urn "ogs_server_urn" : Not found !!
03/Sep/2013 16:11:45:964 ERROR  ? ?  - URN_NOT_FOUND : urn "ogs_server_urn" : Not found !!
In EPMOgsAdapter:since isOGSup is not true
03/Sep/2013 16:11:55:968 ERROR  ? ?  - URN_NOT_FOUND : urn "ogs_server_urn" : Not found !!
ms.ctm.common.CTMException: URN_NOT_FOUND : urn "ogs_server_urn" : Not found !!
03/Sep/2013 16:11:55:968 ERROR  ? ?  - URN_NOT_FOUND : urn "ogs_server_urn" : Not found !!
03/Sep/2013 16:11:55:968 ERROR  ? ?  - URN_NOT_FOUND : urn "ogs_server_urn" : Not found !!
03/Sep/2013 16:12:01:737 ERROR  ? ?  - URN_NOT_FOUND : urn "TIS-1.0" : Not found !!
com.cisco.nm.trx.tis.service.TISServiceException: URN_NOT_FOUND : urn "TIS-1.0" : Not found !!
03/Sep/2013 16:12:01:737 ERROR  ? ?  - URN_NOT_FOUND : urn "TIS-1.0" : Not found !!
at com.cisco.nm.trx.tis.service.TISRemoteServiceFactory.getService(TISRemoteServiceFactory.java:51)
at com.cisco.nm.trx.tis.service.TISServiceFactory.getService(TISServiceFactory.java:32)
at com.cisco.nm.trx.tis.service.TISService.getService(TISService.java:169)
at com.cisco.nm.trx.tis.service.TISService.<init>(TISService.java:192)
at com.cisco.nm.trx.tis.service.TISService.<init>(TISService.java:180)
at com.cisco.nm.trx.tis.service.TISService.getInstance(TISService.java:130)
at com.cisco.nm.trx.epm.providers.inventory.EpmTISInventory$InnerFactory.getInstance(Unknown Source)
at com.cisco.nm.trx.epm.providers.inventory.EpmTISInventory.tis(Unknown Source)
at com.cisco.nm.trx.epm.providers.inventory.EpmTISInventory.getMOId(Unknown Source)
at com.cisco.nm.cesar.kernel.binding.EventBinder.processNormalizedEvent(Unknown Source)
at com.cisco.nm.cesar.kernel.binding.EventBinder.run(Unknown Source)
at java.lang.Thread.run(Thread.java:662)
03/Sep/2013 16:12:01:737 ERROR  ? ?  - URN_NOT_FOUND : urn "TIS-1.0" : Not found !!
03/Sep/2013 16:12:02:770 ERROR  ? ?  - URN_NOT_FOUND : urn "TIS-1.0" : Not found !!
com.cisco.nm.trx.tis.service.TISServiceException: URN_NOT_FOUND : urn "TIS-1.0" : Not found !!
at com.cisco.nm.trx.tis.service.TISRemoteServiceFactory.getService(TISRemoteServiceFactory.java:51)
at com.cisco.nm.trx.tis.service.TISServiceFactory.getService(TISServiceFactory.java:32)
at com.cisco.nm.trx.tis.service.TISService.getService(TISService.java:169)
at com.cisco.nm.trx.tis.service.TISService.<init>(TISService.java:192)
at com.cisco.nm.trx.tis.service.TISService.<init>(TISService.java:180)
at com.cisco.nm.trx.tis.service.TISService.getInstance(TISService.java:130)
at com.cisco.nm.trx.epm.providers.inventory.EpmTISInventory$InnerFactory.getInstance(Unknown Source)
at com.cisco.nm.trx.epm.providers.inventory.EpmTISInventory.tis(Unknown Source)
at com.cisco.nm.trx.epm.providers.inventory.EpmTISInventory.getMOId(Unknown Source)
at com.cisco.nm.cesar.kernel.binding.EventBinder.processNormalizedEvent(Unknown Source)03/Sep/2013 16:12:02:770 ERROR  ? ?  - URN_NOT_FOUND : urn "TIS-1.0" : Not found !!
In EPMOgsAdapter:since isOGSup is not true
In EPMOgsAdapter:EPMOgsadapter is sleeping. Waiting for OGSServer to come up com.cisco.nm.xms.ctm.common.CTMException: URN_NOT_FOUND : urn "ogs_server_urn" : Not found !!
03/Sep/2013 16:12:05:973 ERROR  ? ?  - URN_NOT_FOUND : urn "ogs_server_urn" : Not found !!
03/Sep/2013 16:12:05:974 ERROR  ? ?  - URN_NOT_FOUND : urn "ogs_server_urn" : Not found !!
03/Sep/2013 16:12:05:974 ERROR  ? ?  - URN_NOT_FOUND : urn "ogs_server_urn" : Not found !!
com.cisco.nm.trx.tis.service.TISServiceException: URN_NOT_FOUND : urn "TIS-1.0" : Not found !!
at com.cisco.nm.trx.tis.service.TISRemoteServiceFactory.getService(TISRemoteServiceFactory.java:51)
at com.cisco.nm.trx.tis.service.TISServiceFactory.getService(TISServiceFactory.java:32)
03/Sep/2013 16:12:06:649 ERROR  ? ?  - URN_NOT_FOUND : urn "TIS-1.0" : Not found !!
03/Sep/2013 16:12:06:649 ERROR  ? ?  - URN_NOT_FOUND : urn "TIS-1.0" : Not found !!
.tis.service.TISService.<init>(TISService.java:180)
at com.cisco.nm.trx.tis.service.TISService.getInstance(TISService.java:130)
at com.cisco.nm.trx.epm.providers.inventory.EpmTISInventory$InnerFactory.getInstance(Unknown Source)
at com.cisco.nm.trx.epm.providers.inventory.EpmTISInventory.tis(Unknown Source)
at com.cisco.nm.trx.epm.providers.inventory.EpmTISInventory.getMOId(Unknown Source)
at com.cisco.nm.cesar.kernel.binding.EventBinder.processNormalizedEvent(Unknown Source)
at com.cisco.nm.cesar.kernel.binding.EventBinder.run(Unknown Source)
at java.lang.Thread.run(Thread.java:662)

In the release note i find this:
http://www.cisco.com/en/US/docs/net_mgmt/ciscoworks_lan_management_solution/4.2.3/release/notes/lms4_2_3_release_notes.html
Table 2 Open Bugs in Cisco Prime LMS 4.2.3 

CSCuc82281
  .. DCR Server takes long time to initialize.
  ... This happens when there are lot of files in < NMSROOT>/log directory.

So i have removed all files i  /var/adm/CSCOpx/log/ ( i only had 252 files, which is not "a lot")

started again:

/etc/init.d/dmgtd start

But still same issue ...

I am think of upgrading to SP4, but the bug is still open in SP4 aswell, so i am not sure this will fix it

Please help

regards

Martin

[AFROJ AHMAD · Accepted solution]

Hi Martin,

Check the below things:

Make sure files 

/opt/CSCOpx/MDC/tomcat/shared/lib/ctm_config.txt and

/opt/CSCOpx/MDC/tomcat/webapps/cwhp/WEB-INF/lib/ctm_config.txt has proper content.

/opt/CSCOpx/MDC/tomcat/lib/ctm_config.txt has proper content.

File  should not be empty. Check any working server (If any available or just  send me these files)  for the content, copy and replace

Also send me the pdshow output ,if above does not help..

Thanks

Afroz

[mbilgrav]

Hallo Afroz,

Thanks for your reply.

In the meantime I was tempted to upgrade to SP4 - so i did and the server can up ...

Got som other problems with utm database and poller being disabled ... but i think its ok now ..

Anyways - the files are now almost identical now, so I think we need not to concern around this issue anylonger

[STEFFEN NEUSER]

Hi Afroz,

Thx for your nice post. I think I was running into the same problem after running the rmeptt.sh script at 4.2.5 Linux VA.

I think the last file from your explanation was destroyed by ptt

[lms4/root-ade bin]# ls -l /opt/CSCOpx/MDC/tomcat/lib/ctm_config.txt

-rwxr-xr-x 1 casuser casusers 52 Jul 28 16:00 /opt/CSCOpx/MDC/tomcat/lib/ctm_config.txt

[lms4/root-ade bin]# cat /opt/CSCOpx/MDC/tomcat/lib/ctm_config.txt

#CSTM Configurations

#Mon Jul 28 16:00:20 CEST 2014

So I want to ask if you can provide an /opt/CSCOpx/MDC/tomcat/lib/ctm_config.txt for 4.2.5?

thx, Steffen

[AFROJ AHMAD]

Hi Steffen,

I have attached the file.

stop the daemon manager

replace the file , make sure the new file have all the required permission.

start the daemon manager

Let me know how it works..

Thanks-

Afroz

[STEFFEN NEUSER]

Hi afroz,

thx, problem has been resolved with this action

Steffen

[Mehdi Talei]

Hello Afroj,

Can the attached file be used for LMS 4.2.3 as well?

The content of my /opt/CSCOpx/MDC/tomcat/lib/ctm_config.txt  is the following:

[ZVS1361U/root-ade lib]# vi ctm_config.txt
#CSTM Configurations
#Thu Sep 03 21:29:05 EDT 2015
~                                

I assume it's corrupted, right?

Thanks

[AFROJ AHMAD]

Hi Mehdi,

Yes ,it is corrupt..

and you can use that file.. should take care of the issue...

Thanks-

Afroz