LMS 4.2 - ANI Server failed to run

[mssnider]

LMS 4.2, Windows VM server - ANI Server failed to run.

We recently copied the application from one Windows VM server farm to a new one. Everything appeared to be working fine, but I recently discovered that devices were not being auto updated in the system. I ran an update inventory job and it failed to run. It mentioned that maybe the ANI server was down. I checked the processes and found that the ANI Server reported "failed to run."

I've stopped and re-started the daemon mgr and also rebooted the VM server, but the ANI Server still reports failed to run.

Here is an excerpt from the ANIServer.log file;

Informational:  ANIServer starting Cisco Process Manager communication thread
messages will be logged to file: D:/CSCOpx/log\ani.log
Loading Lib.... Loading Lib.... Sucess MESSAGE AniMain: Properties will be read from D:\CSCOpx\campus\etc\cwsi\ANIServer.properties
Informational:  ANIServer starting Cisco Process Manager communication thread
messages will be logged to file: D:/CSCOpx/log\ani.log
Loading Lib.... Loading Lib.... Sucess

Please help, Thanks.

Mike S.

[mssnider]

Here is an excerpt from the ani.log file for your reference;

2016/09/02 08:48:48 main ani POStore: Caught exception:

java.lang.NullPointerException

            at com.cisco.nm.ani.server.vlad.VladDeviceDomain.postReadObject(VladDeviceDomain.java:238)

            at com.cisco.nm.ani.server.framework.POStore.resolveObjects(POStore.java:1335)

            at com.cisco.nm.ani.server.framework.POStore.internalPopulateDataModel(POStore.java:1450)

            at com.cisco.nm.ani.server.framework.POStore.populateDataModel(POStore.java:1503)

            at com.cisco.nm.ani.server.frontend.AniMain.loadFromDatabase(AniMain.java:418)

            at com.cisco.nm.ani.server.frontend.AniMain.dynamicInitialize(AniMain.java:717)

            at com.cisco.nm.ani.server.frontend.AniMain.main(AniMain.java:1192)

2016/09/02 08:48:48 main ani ERROR AniMain: java.lang.NullPointerException

ANI Server failed to load from the database.

 Please stop and restart the system. You may need to re-initialize the database.

 If this does not work, please contact technical support.

[AFROJ AHMAD · Accepted solution]

Hi ,

looks like ANI database has been corrupted.

Follow the below steps:

1. stop the daemon manager

2. Re-initialize the database of the ANI database

for this , you can check the below post:

https://supportforums.cisco.com/document/33056/lms-database-troubleshooting

3. Start the daemon manger.

Thanks-
Afroz

***Ratings Encourages Contributors ***

[mssnider]

I followed the directions on the link you provided. The first part of the instructions said to run the DB validation. I did so and it's reporting that the ANI DB was validated, see below. I was expecting this to say that the DB had issues. I just want to verify that I need to reinit the ANI DB before I do so. I am aware that performing this action will remove everything in the DB and I will have load back in the entire inventory (over 4000 devices).

Here is the output from the commands;

D:\CSCOpx\bin>

D:\CSCOpx\bin>net stop crmdmgtd

The Cisco Prime Daemon Manager service is stopping..............................

....................................................

The Cisco Prime Daemon Manager service was stopped successfully.

D:\CSCOpx\bin>perl.exe D:\CSCOpx\objects\db\conf\configureDb.pl action=validate

dsn=ani

INFO: D:\CSCOpx/objects/db/conf/ConfigureDB.LOCK locked for the operation...

INFO: File exists.SQL Anywhere Command File Hiding Utility Version 10.0.1.4295

SQL Anywhere Validation Utility Version 10.0.1.4310

VALIDATE TABLE "cwsiSA"."devicestopoll"

VALIDATE TABLE "cwsiSA"."DbVersion"

VALIDATE TABLE "cwsiSA"."DbVersionHistory"

VALIDATE TABLE "cwsiSA"."DiscrepancySummary"

VALIDATE TABLE "cwsiSA"."DuplicateMACUTData"

VALIDATE TABLE "cwsiSA"."embuIdentityInfo"

VALIDATE TABLE "cwsiSA"."embuPhoneTracking"

VALIDATE TABLE "cwsiSA"."EmbuClass"

VALIDATE TABLE "cwsiSA"."EmbuUserTracking"

VALIDATE TABLE "cwsiSA"."EndPointLocationDetails"

VALIDATE TABLE "cwsiSA"."EndPointType"

VALIDATE TABLE "cwsiSA"."ExpiryMaintainer"

VALIDATE TABLE "cwsiSA"."HistoryHostTracking"

VALIDATE TABLE "cwsiSA"."HistoryUserTracking"

VALIDATE TABLE "cwsiSA"."ManualDevices"

VALIDATE TABLE "cwsiSA"."NewMAC"

VALIDATE TABLE "cwsiSA"."RogueMAC"

VALIDATE TABLE "cwsiSA"."ScheduledReportTableDetails"

VALIDATE TABLE "cwsiSA"."SystemStatus"

VALIDATE TABLE "cwsiSA"."tmpBPDTable"

VALIDATE TABLE "cwsiSA"."tmpDCRPTable"

VALIDATE TABLE "cwsiSA"."VNMDevice"

VALIDATE TABLE "cwsiSA"."VNMDeviceVLAN"

VALIDATE TABLE "cwsiSA"."VNMDeviceVRF"

VALIDATE TABLE "cwsiSA"."VNMInterface"

VALIDATE TABLE "cwsiSA"."VNMLink"

VALIDATE TABLE "cwsiSA"."VNMUnSupportedDevice"

VALIDATE TABLE "cwsiSA"."VNMVirtualInterface"

VALIDATE TABLE "cwsiSA"."VNMVirtualPhysicalInterfaceMap"

VALIDATE TABLE "cwsiSA"."VNMVRF"

VALIDATE TABLE "cwsiSA"."wirelessUTTable"

VALIDATE TABLE "cwsiSA"."WbuAbstractMap"

VALIDATE TABLE "cwsiSA"."WbuAbstractStpInstance"

VALIDATE TABLE "cwsiSA"."WbuAccessPointPort"

VALIDATE TABLE "cwsiSA"."WbuAggregateLink"

VALIDATE TABLE "cwsiSA"."WbuAltAtmAddr"

VALIDATE TABLE "cwsiSA"."WbuAniAggregateAlias"

VALIDATE TABLE "cwsiSA"."WbuAniTimeLineInfo"

VALIDATE TABLE "cwsiSA"."WbuC2900XL_12_0_5_XPPort"

VALIDATE TABLE "cwsiSA"."WbuC2900XLModule"

VALIDATE TABLE "cwsiSA"."WbuC2900XLPort"

VALIDATE TABLE "cwsiSA"."WbuC2900XLPortGroup"

VALIDATE TABLE "cwsiSA"."WbuC2950Port"

VALIDATE TABLE "cwsiSA"."WbuC3550_12_1_11_EA1Port"

VALIDATE TABLE "cwsiSA"."WbuC3750Module"

VALIDATE TABLE "cwsiSA"."WbuC3750Port"

VALIDATE TABLE "cwsiSA"."WbuC3900Module"

VALIDATE TABLE "cwsiSA"."WbuC3900Port"

VALIDATE TABLE "cwsiSA"."WbuC3KModule"

VALIDATE TABLE "cwsiSA"."WbuC3KPort"

VALIDATE TABLE "cwsiSA"."WbuC4KIOSPort"

VALIDATE TABLE "cwsiSA"."WbuC5500Module"

VALIDATE TABLE "cwsiSA"."WbuC5KModule"

VALIDATE TABLE "cwsiSA"."WbuC5KPort"

VALIDATE TABLE "cwsiSA"."WbuC5KVlanPort"

VALIDATE TABLE "cwsiSA"."WbuC6KIOSPort"

VALIDATE TABLE "cwsiSA"."WbuCallManager"

VALIDATE TABLE "cwsiSA"."WbuCallManagerToCcmGroup"

VALIDATE TABLE "cwsiSA"."WbuCcmComponent"

VALIDATE TABLE "cwsiSA"."WbuCcmDevicePool"

VALIDATE TABLE "cwsiSA"."WbuCcmGroup"

VALIDATE TABLE "cwsiSA"."WbuCcmTimeZone"

VALIDATE TABLE "cwsiSA"."WbuCiscoStpDevice"

VALIDATE TABLE "cwsiSA"."WbuCiscoStpPort"

VALIDATE TABLE "cwsiSA"."WbuCluster"

VALIDATE TABLE "cwsiSA"."WbuDcrpTimeStamp"

VALIDATE TABLE "cwsiSA"."WbuDevice"

VALIDATE TABLE "cwsiSA"."WbuDeviceDomain"

VALIDATE TABLE "cwsiSA"."WbuDeviceToModuleIP"

VALIDATE TABLE "cwsiSA"."WbuDomain"

VALIDATE TABLE "cwsiSA"."WbuEntityModule"

VALIDATE TABLE "cwsiSA"."WbuEthernetChannel"

VALIDATE TABLE "cwsiSA"."WbuEtherSwitchModule"

VALIDATE TABLE "cwsiSA"."WbuEtherSwitchPort"

VALIDATE TABLE "cwsiSA"."WbuExcludedDevice"

VALIDATE TABLE "cwsiSA"."WbuIgxBpxModule"

VALIDATE TABLE "cwsiSA"."WbuIgxBpxPort"

VALIDATE TABLE "cwsiSA"."WbuIgxBpxSubinterface"

VALIDATE TABLE "cwsiSA"."WbuIlmiTopoGroup"

VALIDATE TABLE "cwsiSA"."WbuIpTableEntry"

VALIDATE TABLE "cwsiSA"."WbuIpv6TableEntry"

VALIDATE TABLE "cwsiSA"."WbuLS1010Module"

VALIDATE TABLE "cwsiSA"."WbuLS1010Port"

VALIDATE TABLE "cwsiSA"."WbuLS1010SubInterface"

VALIDATE TABLE "cwsiSA"."WbuMib2Interface"

VALIDATE TABLE "cwsiSA"."WbuMLSRouter"

VALIDATE TABLE "cwsiSA"."WbuModule"

VALIDATE TABLE "cwsiSA"."WbuMSTDevice"

VALIDATE TABLE "cwsiSA"."WbuMSTInstPort"

VALIDATE TABLE "cwsiSA"."WbuMSTPort"

VALIDATE TABLE "cwsiSA"."WbuMultiStpInstance"

VALIDATE TABLE "cwsiSA"."WbuObjectToAggr"

VALIDATE TABLE "cwsiSA"."WbuPagpPortChannel"

VALIDATE TABLE "cwsiSA"."WbuPort"

VALIDATE TABLE "cwsiSA"."WbuPortProtocolFilter"

VALIDATE TABLE "cwsiSA"."WbuRouterPort"

VALIDATE TABLE "cwsiSA"."WbuSatelliteChassis"

VALIDATE TABLE "cwsiSA"."WbuSatellitePort"

VALIDATE TABLE "cwsiSA"."WbuSatelliteRSLConnection"

VALIDATE TABLE "cwsiSA"."WbuSMFContainerFlag"

VALIDATE TABLE "cwsiSA"."WbuSMFContainers"

VALIDATE TABLE "cwsiSA"."WbuStpDevice"

VALIDATE TABLE "cwsiSA"."WbuStpInstDevice"

VALIDATE TABLE "cwsiSA"."WbuStpInstPort"

VALIDATE TABLE "cwsiSA"."WbuStpPort"

VALIDATE TABLE "cwsiSA"."WbuStpTrunkPort"

VALIDATE TABLE "cwsiSA"."WbuSubinterface"

VALIDATE TABLE "cwsiSA"."WbuSystemApp"

VALIDATE TABLE "cwsiSA"."WbuSystemAppRunState"

VALIDATE TABLE "cwsiSA"."WbuTopoEndpoint"

VALIDATE TABLE "cwsiSA"."WbuTopoGroup"

VALIDATE TABLE "cwsiSA"."WbuTransparentDomain"

VALIDATE TABLE "cwsiSA"."WbuTRBrfVlan"

VALIDATE TABLE "cwsiSA"."WbuTRCrfVlan"

VALIDATE TABLE "cwsiSA"."WbuV3Domain"

VALIDATE TABLE "cwsiSA"."WbuVlan"

VALIDATE TABLE "cwsiSA"."WbuVlanPort"

VALIDATE TABLE "cwsiSA"."WbuVlanStpInstance"

VALIDATE TABLE "dbo"."EXCLUDEOBJECT"

VALIDATE TABLE "dbo"."ix_consultant_affected_columns"

VALIDATE TABLE "dbo"."ix_consultant_index"

VALIDATE TABLE "dbo"."ix_consultant_ixcol"

VALIDATE TABLE "dbo"."ix_consultant_log"

VALIDATE TABLE "dbo"."ix_consultant_master"

VALIDATE TABLE "dbo"."ix_consultant_query_index"

VALIDATE TABLE "dbo"."ix_consultant_query_phase"

VALIDATE TABLE "dbo"."ix_consultant_query_text"

VALIDATE TABLE "dbo"."jdbc_function_escapes"

VALIDATE TABLE "dbo"."migrate_remote_fks_list"

VALIDATE TABLE "dbo"."migrate_remote_table_list"

VALIDATE TABLE "dbo"."migrate_sql_defn"

VALIDATE TABLE "dbo"."RowGenerator"

VALIDATE TABLE "dbo"."sa_diagnostic_auxiliary_catalog"

VALIDATE TABLE "dbo"."sa_diagnostic_blocking"

VALIDATE TABLE "dbo"."sa_diagnostic_cachecontents"

VALIDATE TABLE "dbo"."sa_diagnostic_connection"

VALIDATE TABLE "dbo"."sa_diagnostic_cursor"

VALIDATE TABLE "dbo"."sa_diagnostic_deadlock"

VALIDATE TABLE "dbo"."sa_diagnostic_hostvariable"

VALIDATE TABLE "dbo"."sa_diagnostic_internalvariable"

VALIDATE TABLE "dbo"."sa_diagnostic_optblock"

VALIDATE TABLE "dbo"."sa_diagnostic_optjoinstrategy"

VALIDATE TABLE "dbo"."sa_diagnostic_optorder"

VALIDATE TABLE "dbo"."sa_diagnostic_optquantifier"

VALIDATE TABLE "dbo"."sa_diagnostic_optrewrite"

VALIDATE TABLE "dbo"."sa_diagnostic_query"

VALIDATE TABLE "dbo"."sa_diagnostic_request"

VALIDATE TABLE "dbo"."sa_diagnostic_statement"

VALIDATE TABLE "dbo"."sa_diagnostic_statistics"

VALIDATE TABLE "dbo"."sa_diagnostic_tracing_level"

VALIDATE TABLE "dbo"."spt_collation_map"

VALIDATE TABLE "dbo"."spt_jdatatype_info"

VALIDATE TABLE "dbo"."spt_jdbc_conversion"

VALIDATE TABLE "dbo"."spt_jtext"

VALIDATE TABLE "dbo"."spt_mda"

VALIDATE TABLE "rs_systabgroup"."rs_lastcommit"

VALIDATE TABLE "rs_systabgroup"."rs_threads"

VALIDATE TABLE "SYS"."DUMMY"

VALIDATE TABLE "SYS"."ISYSARTICLE"

VALIDATE TABLE "SYS"."ISYSARTICLECOL"

VALIDATE TABLE "SYS"."ISYSATTRIBUTE"

VALIDATE TABLE "SYS"."ISYSATTRIBUTENAME"

VALIDATE TABLE "SYS"."ISYSCAPABILITY"

VALIDATE TABLE "SYS"."ISYSCAPABILITYNAME"

VALIDATE TABLE "SYS"."ISYSCHECK"

VALIDATE TABLE "SYS"."ISYSCOLPERM"

VALIDATE TABLE "SYS"."ISYSCOLSTAT"

VALIDATE TABLE "SYS"."ISYSCONSTRAINT"

VALIDATE TABLE "SYS"."ISYSDEPENDENCY"

VALIDATE TABLE "SYS"."ISYSDOMAIN"

VALIDATE TABLE "SYS"."ISYSEVENT"

VALIDATE TABLE "SYS"."ISYSEVENTTYPE"

VALIDATE TABLE "SYS"."ISYSEXTERNLOGIN"

VALIDATE TABLE "SYS"."ISYSFILE"

VALIDATE TABLE "SYS"."ISYSFKEY"

VALIDATE TABLE "SYS"."ISYSGROUP"

VALIDATE TABLE "SYS"."ISYSHISTORY"

VALIDATE TABLE "SYS"."ISYSIDX"

VALIDATE TABLE "SYS"."ISYSIDXCOL"

VALIDATE TABLE "SYS"."ISYSJAR"

VALIDATE TABLE "SYS"."ISYSJARCOMPONENT"

VALIDATE TABLE "SYS"."ISYSJAVACLASS"

VALIDATE TABLE "SYS"."ISYSLOGINMAP"

VALIDATE TABLE "SYS"."ISYSMVOPTION"

VALIDATE TABLE "SYS"."ISYSMVOPTIONNAME"

VALIDATE TABLE "SYS"."ISYSOBJECT"

VALIDATE TABLE "SYS"."ISYSOPTION"

VALIDATE TABLE "SYS"."ISYSOPTSTAT"

VALIDATE TABLE "SYS"."ISYSPHYSIDX"

VALIDATE TABLE "SYS"."ISYSPROCEDURE"

VALIDATE TABLE "SYS"."ISYSPROCPARM"

VALIDATE TABLE "SYS"."ISYSPROCPERM"

VALIDATE TABLE "SYS"."ISYSPROXYTAB"

VALIDATE TABLE "SYS"."ISYSPUBLICATION"

VALIDATE TABLE "SYS"."ISYSREMARK"

VALIDATE TABLE "SYS"."ISYSREMOTEOPTION"

VALIDATE TABLE "SYS"."ISYSREMOTEOPTIONTYPE"

VALIDATE TABLE "SYS"."ISYSREMOTETYPE"

VALIDATE TABLE "SYS"."ISYSREMOTEUSER"

VALIDATE TABLE "SYS"."ISYSSCHEDULE"

VALIDATE TABLE "SYS"."ISYSSERVER"

VALIDATE TABLE "SYS"."ISYSSOURCE"

VALIDATE TABLE "SYS"."ISYSSQLSERVERTYPE"

VALIDATE TABLE "SYS"."ISYSSUBSCRIPTION"

VALIDATE TABLE "SYS"."ISYSSYNC"

VALIDATE TABLE "SYS"."ISYSSYNCSCRIPT"

VALIDATE TABLE "SYS"."ISYSTAB"

VALIDATE TABLE "SYS"."ISYSTABCOL"

VALIDATE TABLE "SYS"."ISYSTABLEPERM"

VALIDATE TABLE "SYS"."ISYSTRIGGER"

VALIDATE TABLE "SYS"."ISYSTYPEMAP"

VALIDATE TABLE "SYS"."ISYSUSER"

VALIDATE TABLE "SYS"."ISYSUSERAUTHORITY"

VALIDATE TABLE "SYS"."ISYSUSERMESSAGE"

VALIDATE TABLE "SYS"."ISYSUSERTYPE"

VALIDATE TABLE "SYS"."ISYSVIEW"

VALIDATE TABLE "SYS"."ISYSWEBSERVICE"

No errors reported

INFO:   D:\CSCOpx\databases\ani\ani.db validation complete.

INFO: D:\CSCOpx/objects/db/conf/ConfigureDB.LOCK released for the future operations...

D:\CSCOpx\bin>

[AFROJ AHMAD]

Hi,

Yes , sometime validation may report it fine. I would still advise to go further with reintialize.

Thanks-

Afroz

[mssnider]

I will be doing the re-init of the ANI DB tomorrow.

I am getting things lined up for the change. I am of the understanding that I will lose the entire inventory when I do the re-init. I have a bulk import ready to go for the current inventory so I should be good there.

I did a complete System Backup this morning.

Are there any other things you can think of I need to prepare for prior to the re-init of the ANI DB?

[mssnider]

I ran the command to re-init the DB for our Window server = NMSROOT\bin\perl.exe NMSROOT\bin\dbRestoreOrig.pl dsn=ani dmprefix=ANI.

It worked great. The ANIServer is now running.

I was under the impression that I would have to load the inventory back into the system, but I was wrong. All of the devices were still in the inventory. Everything appears to be running fine. I don't think I have any follow-up tasks from performing the re-init.

Thanks for the help!

[AFROJ AHMAD]

Hi, yes , when you re-init ANI DB ,it will not removed teh devices from the DCR. I wanted to update you but I was liitle busy and could't get time to update the thread. Anyways good to know , it is working :) Thanks- Afroz