topic Re: Devices in UNKNOWN state in Controllers

Devices in UNKNOWN state

John Capobianco — Fri, 01 Mar 2019 12:40:31 GMT

Hello -

I currently have 6 devices in an UNKNOWN state; they discovery fine with CLI / SSH / SNMP access all passing; I can SSH to the devices directly from APIC-EM. I have tried multiple discoveries; deleted the devices; re-added the devices. However I can not get them into a fully Managed state.

One of the devices is my 6500 Core so things like Pathtrace across the Core fail

1 - 6513E - 15.2(1)SY3

3 - 4500-X - 03.06.06.E

2 - Nexus 5k - 7.1(4)N1(1)

The 6500 is in a VS40 (VSS with 4 supervisors) and my 4500-X are all VSS

This is my lab environment we are hoping to move to a production instance soon however until I can fully manage all of my devices in my lab environment I am holding off on my production deployment. I have an open TAC / have been working with TAC but have not found the root cause.

Do I need to hold off for version 2.0? Is there a release for 2.0 scheduled?

Any assistance would be greatly appreciated

Thanks

Re: Devices in UNKNOWN state

aradford — Wed, 23 Aug 2017 03:59:15 GMT

What version of APIC-EM are you running?

Re: Devices in UNKNOWN state

Parthiv Shah — Wed, 23 Aug 2017 05:05:58 GMT

IF all devices showing unknown then there could be one of the following mismatch between device and Alice-em config

- username / paasword

- snmp community

ccould you make sure that it is correct and confirm whether you changed any of the above after initial discovery?

Re: Devices in UNKNOWN state

John Capobianco — Wed, 23 Aug 2017 12:25:27 GMT

I am running 1.5.0.1368

Username / password and SNMP is all fine during Discovery everything passes (CLI / SNMP / SSH)

Re: Devices in UNKNOWN state

ymeloch — Wed, 23 Aug 2017 12:53:30 GMT

1. Can you share the discovery log?

ssh grapevine@<cluster-ip>

grape instance status | grep inventory (the output would list the <ip-address> of the inventory manager-service

ssh grapevine@<ip-address>

less /var/log/grapevine/services/apic-em-inventory-manager-service/[version-number]/apic-em-inventory-manager-service.log

2. Can you share the output of "snmpwalk -v2c -c <community> <device-ip> 1.3.6.1.2.1"

3. Any https/netconf validations?

Re: Devices in UNKNOWN state

John Capobianco — Wed, 23 Aug 2017 14:44:08 GMT

Attached are the requested outputs

I am not sure about https/netconf validations what do you require?

Thanks

Re: Devices in UNKNOWN state

ymeloch — Wed, 23 Aug 2017 17:24:48 GMT

Thanks for attaching the logs.

Can you please check if the "apic-em-inventory-manager-service" service is running?

Command:

grape instance status | grep apic-em-inventory-manager-service

Example for a proper output:

apic-em-inventory-manager-service 5.1.31.1368 running 3d4390a6-c6d3-4a23-a9d6-e7d605a00aa0 169.254.0.77 14 days, 13:15:10

Re: Devices in UNKNOWN state

John Capobianco — Wed, 23 Aug 2017 17:30:50 GMT

apic-em-inventory-manager-service 5.1.31.1368 running 1c070547-699f-4256-a468-f7f5712ca7a7 169.254.0.209 29 days, 2:52:31

Re: Devices in UNKNOWN state

Parthiv Shah — Wed, 23 Aug 2017 17:55:39 GMT

Hi,

How about enable password?

Could you send the screenshot of discovery and error?

Thanks

Parthiv

Re: Devices in UNKNOWN state

John Capobianco — Wed, 23 Aug 2017 18:04:07 GMT

Everything passes / is successful on my discovery jobs

See attachments / screenshots for the UNKNOWN

The Help icon suggests to re-run discovery (which always passes) or contact TAC (which I have)

TAC has tried to use the database editor to manually discovery / delete / add the device 172.24.128.1 but we can never get it out of the UNKNOWN state

Also as noted this is the 6500 Core - if I cannot manage the Core things like Pathtrace result in errors

Thanks for your time

Re: Devices in UNKNOWN state

ymeloch — Wed, 23 Aug 2017 18:43:32 GMT

Can you please send the output of "grape instance status"?

Also, i do see strange printouts at the end of the "apic-em-inventory-manager-service" log file

(e.g. Full thread dump, [1]+ Stopped less /var/log/grapevine/services/apic-em-inventory-manager-service/5.1.31.1368/apic-em-inventory-manager-service.log)

I'll try to check that..

Thanks!

Re: Devices in UNKNOWN state

John Capobianco — Wed, 23 Aug 2017 18:45:43 GMT

SERVICE VERSION STATE CLIENT IP UPTIME

-----------------------------------------------------------------------------------------------------------------------------

access-policy-programmer-service 5.1.31.1368 running 5b7c0ebe-a304-4dc3-89c3-602166d32a81 169.254.0.104 29 days, 4:13:25

apic-em-event-service 5.1.31.1368 running d02c168d-5d8a-4e77-babd-e1ef6ae8e198 169.254.0.218 29 days, 4:11:56

apic-em-inventory-manager-service 5.1.31.1368 running 1c070547-699f-4256-a468-f7f5712ca7a7 169.254.0.209 29 days, 4:07:27

apic-em-jboss-ejbca 5.1.31.1368 running 5821ae79-ee4e-465a-8688-aa80cae88bfb 169.254.0.22 29 days, 4:13:50

apic-em-network-programmer-service 5.1.31.1368 running 161a5d65-c96e-482f-a983-81b1c4b35bf9 169.254.0.200 29 days, 4:07:31

apic-em-network-programmer-service 5.1.31.1368 running b55aca33-4fc6-4b3e-a5a9-0940395c2d59 169.254.0.14 29 days, 3:40:56

apic-em-pki-broker-service 1.5.0.1368 running 4ff82566-aa30-4a7b-b9ca-7d004c56b1aa 169.254.0.95 29 days, 4:14:33

app-vis-policy-programmer-service 1.4.1.504 running 5b7c0ebe-a304-4dc3-89c3-602166d32a81 169.254.0.104 29 days, 4:13:20

cas-service 5.0.31.4021 running d42d138c-45e8-4dc4-838c-240008325f33 169.254.0.196 29 days, 4:25:18

cassandra 1.0.0 running b0430126-678b-4404-a52c-50de06d6a742 169.254.0.1 41 days, 2:15:13

election-service 5.0.31.4021 running 161a5d65-c96e-482f-a983-81b1c4b35bf9 169.254.0.200 29 days, 4:23:58

file-service 5.1.31.1368 running 161a5d65-c96e-482f-a983-81b1c4b35bf9 169.254.0.200 29 days, 4:16:38

grapevine 1.0.0 running b0430126-678b-4404-a52c-50de06d6a742 169.254.0.1 41 days, 2:15:09

grapevine-coordinator-service 1.0.0 running b0430126-678b-4404-a52c-50de06d6a742 169.254.0.1 41 days, 2:15:00

grapevine-log-collector 1.0.0 running b0430126-678b-4404-a52c-50de06d6a742 169.254.0.1 41 days, 2:14:56

grouping-service 5.1.31.1368 running 161a5d65-c96e-482f-a983-81b1c4b35bf9 169.254.0.200 29 days, 4:09:52

identity-manager-pxgrid-service 5.1.31.1368 running bce49a8b-8555-4691-826c-b3c24e91c773 169.254.0.224 29 days, 4:11:27

integrity-verification-service 5.1.31.203 running ed2c819f-9ddf-4d4c-a10f-13fe53db44a4 169.254.0.250 29 days, 4:17:54

ip-pool-manager-service 1.4.1.504 running d42d138c-45e8-4dc4-838c-240008325f33 169.254.0.196 29 days, 4:17:41

ipgeo-service 1.4.1.504 running 161a5d65-c96e-482f-a983-81b1c4b35bf9 169.254.0.200 29 days, 4:22:23

iwan-analytics-service 1.4 harvested None None

nbar-policy-programmer-service 5.1.31.1368 running e999cab8-fc00-4d6e-ac24-baf59e86a570 169.254.0.241 29 days, 4:13:35

network-poller-service 5.1.31.1368 running 6a2c1648-be91-4ad3-9e2f-0e75b2f0bea8 169.254.0.205 29 days, 4:08:55

node-ui 1.0.0 running b0430126-678b-4404-a52c-50de06d6a742 169.254.0.1 41 days, 2:15:05

pfr-policy-programmer-service 1.4.1.504 running 97ec6075-61dc-491c-b0ea-023fbe34c3e8 169.254.0.49 29 days, 4:16:21

pnp-service 5.17.31.211 running 4ff82566-aa30-4a7b-b9ca-7d004c56b1aa 169.254.0.95 29 days, 4:14:27

policy-analysis-service 5.1.31.1368 running bce49a8b-8555-4691-826c-b3c24e91c773 169.254.0.224 29 days, 4:11:27

policy-manager-service 5.1.31.1368 running bce49a8b-8555-4691-826c-b3c24e91c773 169.254.0.224 29 days, 4:12:02

postgres 5.1.31.1368 running 5902e42e-f79a-4af8-9b03-ca22cca9e898 169.254.0.111 29 days, 4:21:47

qos-lan-policy-programmer-service 5.1.31.1368 running ed2c819f-9ddf-4d4c-a10f-13fe53db44a4 169.254.0.250 29 days, 4:12:26

qos-monitoring-service 5.1.31.1368 running d42d138c-45e8-4dc4-838c-240008325f33 169.254.0.196 29 days, 4:16:33

qos-policy-programmer-service 5.1.31.1368 running e999cab8-fc00-4d6e-ac24-baf59e86a570 169.254.0.241 29 days, 4:13:30

rabbitmq 1.0.0 running b0430126-678b-4404-a52c-50de06d6a742 169.254.0.1 41 days, 2:15:18

rbac-service 5.0.31.4021 running d42d138c-45e8-4dc4-838c-240008325f33 169.254.0.196 29 days, 4:25:23

reverse-proxy 5.0.31.4021 running da9a8a53-2c15-45d5-b9b0-f2d06e625b34 169.254.0.1 29 days, 4:26:58

router 5.0.31.4021 running da9a8a53-2c15-45d5-b9b0-f2d06e625b34 169.254.0.1 29 days, 4:27:04

scheduler-service 5.1.31.1368 running 942ae612-5e21-4f03-b3a6-9a114df9e4cd 169.254.0.228 29 days, 4:16:17

task-service 5.1.31.1368 running 942ae612-5e21-4f03-b3a6-9a114df9e4cd 169.254.0.228 29 days, 4:16:26

telemetry-service 5.1.31.1368 running 942ae612-5e21-4f03-b3a6-9a114df9e4cd 169.254.0.228 29 days, 4:16:26

topology-service 1.5.0.1368 running bce49a8b-8555-4691-826c-b3c24e91c773 169.254.0.224 29 days, 4:11:23

visibility-service 1.4.1.504 running 97ec6075-61dc-491c-b0ea-023fbe34c3e8 169.254.0.49 29 days, 4:14:07

Re: Devices in UNKNOWN state

John Capobianco — Mon, 11 Sep 2017 17:15:07 GMT

When we (TAC) check the inventory logs we found the following - hope this helps

inventory.log:[2017-09-11 13:54:22,888] [XDE ThreadPool 18] [inventory] [ERROR] - Exception executing function [ethernetextra]interfaceCLI runId 10838221 in time 814 hook time 128 deviceId: 824835; CredentialId: 824835; ManagementIP: 172.24.128.1

inventory.log-[2017-09-11 13:54:22,888] [XDE ThreadPool 18] [inventory] [ERROR] - Exception Stack: com.cisco.nm.expression.function.FunctionException: java.lang.reflect.InvocationTargetException

inventory.log:[2017-09-11 13:54:22,888] [ICE Service - Network 11] [inventory] [ERROR] - Feature = feature_config_interface and Procedure = ethernetextra.interfaceCLI with the following error and continuing with other features: com.cisco.nm.expression.function.FunctionException: java.lang.reflect.InvocationTargetException deviceId: 824835; CredentialId: 824835; ManagementIP: 172.24.128.1

inventory.log-[2017-09-11 13:54:22,888] [ICE Service - Network 11] [inventory] [ERROR] - Exception Stack: com.cisco.nm.expression.function.FunctionException: java.lang.reflect.InvocationTargetException

inventory.log-[2017-09-11 13:54:33,221] [XDE ThreadPool 10] [AssociationBuilderUtil] [ERROR] - Cannot locate the ICE metadata file: /ice-metadata/ifm_networkservices_technologies_trustsec/ifm_networkservices_technologies_trustsec.xml.

inventory.log-[2017-09-11 14:19:39,335] [XDE ThreadPool 6] [inventory] [ERROR] - Exception executing function [com.cisco.ef.config.xde.lisp7k-inventory]procedure runId 10842227 in time 3 hook time 0 deviceId: 824834; CredentialId: 824834; ManagementIP: 172.24.170.16

inventory.log-[2017-09-11 14:19:39,335] [XDE ThreadPool 6] [inventory] [ERROR] - Exception Stack: com.cisco.nm.pal.client.PALException: <palError><deviceId>824834</deviceId><code>NO_PAL_RULE</code><message>No rule test for function "[com.cisco.ef.config.xde.lisp7k-inventory]lisp7k-inventory.xpa/xmp-im-lisp-module/xmp-im-lisp-module.par" matched for device "824834". For a rule test to succeed, every test variable/value in the rule must be set on the device. Either this device is not recognized by your inventory provider or the set of attributes for the device does not match any rule test in the action. </message><credential name="software">NX-OS</credential><credential name="DEVICE_NAME">824834</credential><credential name="DEVICE_ID">824834</credential></palError>

inventory.log:[2017-09-11 14:19:50,725] [XDE ThreadPool 26] [inventory] [ERROR] - Exception occured while converting the output of workflow runId 10842279 deviceId: 824835; CredentialId: 824835; ManagementIP: 172.24.128.1

inventory.log-[2017-09-11 14:19:50,726] [XDE ThreadPool 26] [inventory] [ERROR] - Exception Stack: com.cisco.xmp.deviceaccess.businessProcessor.BusinessProcessHookException: java.lang.reflect.InvocationTargetException::sun.reflect.GeneratedMethodAccessor342.invoke(Unknown Source)::sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)::java.lang.reflect.Method.invoke(Method.java:606)::com.cisco.xmp.deviceaccess.businessProcessor.BusinessProcessManager._postProcessModelObject(BusinessProcessManager.java:119)::com.cisco.xmp.deviceaccess.businessProcessor.BusinessProcessManager.postProcessModelObjectWithResponse(BusinessProcessManager.java:146)::com.cisco.xmp.inventory.pojogen.PalXMLToPOJOConvertor.getClassInfo(PalXMLToPOJOConvertor.java:296)::com.cisco.xmp.inventory.pojogen.PalXMLToPOJOConvertor.getClassInfo(PalXMLToPOJOConvertor.java:306)::com.cisco.xmp.inventory.pojogen.PalXMLToPOJOConvertor.parseObjectsFromXML(PalXMLToPOJOConvertor.java:190)::com.cisco.xmp.inventory.ice.collectortask.CollectorFeatureHookTask.featureHook(CollectorFeatureHookTask.java:53)::com.cisco.xmp.inventory.ice.collectortask.CollectorFeatureHookTask.externalFunctionCall(CollectorFeatureHookTask.java:113)::com.cisco.nm.expression.function.system.system.AbstractExternalFunction.callAsync(AbstractExternalFunction.java:86)::com.cisco.nm.expression.function.system.system.HookFunction.callAsync(HookFunction.java:53)::com.cisco.nm.expression.function.FunctionRunnerContext.runFunction(FunctionRunnerContext.java:97)::com.cisco.nm.expression.function.FunctionRunner.runFunctionAsync(FunctionRunner.java:65)::com.cisco.nm.expression.js.packageimport.JavaScriptXdeFunctionObject$JavaScriptXdeFunctionObjectContext.call(JavaScriptXdeFunctionObject.java:56)::com.cisco.nm.expression.js.packageimport.JavaScriptXdeFunctionObject$JavaScriptXdeFunctionObjectContext.access$300(JavaScriptXdeFunctionObject.java:40)::com.cisco.nm.expression.js.packageimport.JavaScriptXdeFunctionObject.call(JavaScriptXdeFunctionObject.java:193)::org.mozilla.javascript.Interpreter.interpretLoop(Interpreter.java:1482)::org.mozilla.javascript.Interpreter.interpret(Interpreter.java:815)::org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:109)::org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:394)::org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3096)::org.mozilla.javascript.Context.callFunctionWithContinuations(Context.java:1172)::org.mozilla.javascript.Context.executeScriptWithContinuations(Context.java:1139)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.execute(JavaScriptCompiler.java:213)::com.cisco.nm.expression.js.JavaScriptCompiler.executeFunction(JavaScriptCompiler.java:155)::com.cisco.nm.expression.js.JavaScriptCompiler.executeExpression(JavaScriptCompiler.java:143)::com.cisco.nm.workflow.steps.StepExpressionJavaScript.executeExpression(StepExpressionJavaScript.java:118)::com.cisco.nm.workflow.steps.JavaScriptStep.run(JavaScriptStep.java:46)::com.cisco.nm.workflow.run.FlowStepContainerRunner.processLoop(FlowStepContainerRunner.java:106)::com.cisco.nm.util.async.AsyncWhileLoop.processLoopInternal(AsyncWhileLoop.java:35)::com.cisco.nm.util.async.AsyncWhileLoop.continueLoop(AsyncWhileLoop.java:53)::com.cisco.nm.workflow.run.FlowStepContainerRunner.runStepResult(FlowStepContainerRunner.java:132)::com.cisco.nm.workflow.steps.JavaScriptStep$1.executeExpressionResult(JavaScriptStep.java:53)::com.cisco.nm.workflow.steps.StepExpressionJavaScript$2.executeExpressionResult(StepExpressionJavaScript.java:121)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.callbackSuccess(JavaScriptCompiler.java:315)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.processResult(JavaScriptCompiler.java:305)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.futureCallback(JavaScriptCompiler.java:254)::com.cisco.nm.expression.js.packageimport.JavaScriptXdeFunctionObject$JavaScriptXdeFunctionObjectContext.functionResult(JavaScriptXdeFunctionObject.java:89)::com.cisco.nm.expression.function.FunctionRunnerContext.functionResult(FunctionRunnerContext.java:170)::com.cisco.nm.expression.function.xep.XepBindingCallingContextRunAsRules.functionResult(XepBindingCallingContextRunAsRules.java:75)::com.cisco.nm.expression.function.FunctionRunnerContext.functionResult(FunctionRunnerContext.java:170)::com.cisco.nm.pal.function.par.ParFileFunctionContext.workflowFinished(ParFileFunctionContext.java:109)::com.cisco.nm.workflow.run.WorkflowRunner.runFlowResult(WorkflowRunner.java:72)::com.cisco.nm.workflow.run.FlowStepContainerRunner.finishedLoopWithSuccess(FlowStepContainerRunner.java:196)::com.cisco.nm.util.async.AsyncWhileLoop.haltLoopWithSuccess(AsyncWhileLoop.java:68)::com.cisco.nm.workflow.run.FlowStepContainerRunner.flowFinishedWithResult(FlowStepContainerRunner.java:208)::com.cisco.nm.workflow.run.FlowStepContainerRunner.runStepResult(FlowStepContainerRunner.java:142)::com.cisco.nm.workflow.steps.ReturnStep$1.executeExpressionResult(ReturnStep.java:56)::com.cisco.nm.workflow.steps.StepExpressionJavaScript$2.executeExpressionResult(StepExpressionJavaScript.java:121)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.callbackSuccess(JavaScriptCompiler.java:315)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.processResult(JavaScriptCompiler.java:305)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.execute(JavaScriptCompiler.java:224)::com.cisco.nm.expression.js.JavaScriptCompiler.executeFunction(JavaScriptCompiler.java:155)::com.cisco.nm.expression.js.JavaScriptCompiler.executeExpression(JavaScriptCompiler.java:143)::com.cisco.nm.workflow.steps.StepExpressionJavaScript.executeExpression(StepExpressionJavaScript.java:118)::com.cisco.nm.workflow.steps.ReturnStep.run(ReturnStep.java:49)::com.cisco.nm.workflow.run.FlowStepContainerRunner.processLoop(FlowStepContainerRunner.java:106)::com.cisco.nm.util.async.AsyncWhileLoop.processLoopInternal(AsyncWhileLoop.java:35)::com.cisco.nm.util.async.AsyncWhileLoop.continueLoop(AsyncWhileLoop.java:53)::com.cisco.nm.workflow.run.FlowStepContainerRunner.runStepResult(FlowStepContainerRunner.java:132)::com.cisco.nm.workflow.steps.HandlerStep$HandlerStepImpl.handlerResult(HandlerStep.java:157)::com.cisco.nm.pal.handler.HandlerCallbackWrapped.run(HandlerCallbackWrapped.java:102)::com.cisco.nm.pal.handler.HandlerCallbackWrapped.makeCallback(HandlerCallbackWrapped.java:90)::com.cisco.nm.pal.handler.HandlerCallbackWrapped.handlerResult(HandlerCallbackWrapped.java:62)::com.cisco.nm.pal.customhandler.cli.CLIHandlerImplBaseShowDocument.finishedIterationWithSuccess(CLIHandlerImplBaseShowDocument.java:74)::com.cisco.nm.util.async.AsyncIterator.haltIterationWithSuccess(AsyncIterator.java:87)::com.cisco.nm.util.async.AsyncIterator.processLoopInternal(AsyncIterator.java:39)::com.cisco.nm.util.async.AsyncIterator.continueIteration(AsyncIterator.java:72)::com.cisco.nm.pal.customhandler.cli.CLIHandlerImplBaseShowDocument.sessionRunnerCallbackResult(CLIHandlerImplBaseShowDocument.java:151)::com.cisco.xmp.utilities.sessionrunner.XMPCustomSessionRunner$XMPCacheCLIHandlerCallback.sessionRunnerCallbackResult(XMPCustomSessionRunner.java:79)::com.cisco.nm.pal.cli.runner.SessionRunnerWorker.continueTidyUp(SessionRunnerWorker.java:305)::com.cisco.nm.pal.cli.runner.SessionRunnerWorker.sessionStopSuccess(SessionRunnerWorker.java:236)::com.cisco.nm.pal.cli.session.CliSessionContextBase.stopSuccess(CliSessionContextBase.java:96)::com.cisco.nm.pal.cli.transport.CliTransportCommunicator.stopSuccess(CliTransportCommunicator.java:227)::com.cisco.nm.pal.cli.transport.ssh2.CliTransportProviderAsyncSSH2.stop(CliTransportProviderAsyncSSH2.java:258)::com.cisco.nm.pal.cli.transport.CliTransportCommunicator.stopAsync(CliTransportCommunicator.java:112)::com.cisco.nm.pal.cli.session.CliSessionContextBase.stopAsync(CliSessionContextBase.java:91)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveSession.stopAsync(CliInteractiveSession.java:35)::com.cisco.nm.pal.cli.runner.SessionRunnerWorker.continueTidyUp(SessionRunnerWorker.java:275)::com.cisco.nm.pal.cli.runner.SessionRunnerWorker.sessionRunCommandSuccess(SessionRunnerWorker.java:199)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveShowCommandRunner.runInteractiveCommandsAsyncSuccess(CliInteractiveShowCommandRunner.java:80)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveCommandRunner.succeeded(CliInteractiveCommandRunner.java:281)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveCommandRunner.responseReceived(CliInteractiveCommandRunner.java:219)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveCommandRunner.access$000(CliInteractiveCommandRunner.java:23)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveCommandRunner$MainCommandCallback.expectResult(CliInteractiveCommandRunner.java:30)::com.cisco.nm.pal.cli.transport.CliTransportCommunicatorExpectEngine$1.run(CliTransportCommunicatorExpectEngine.java:151)::java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)::java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)::com.cisco.nm.run.threadpool.ThreadPool$PooledThread.run(ThreadPool.java:60)

inventory.log:[2017-09-11 14:19:50,726] [XDE ThreadPool 26] [inventory] [ERROR] - Exception executing function [ethernetextra]interfaceCLI runId 10842279 in time 1922 hook time 133 deviceId: 824835; CredentialId: 824835; ManagementIP: 172.24.128.1

inventory.log-[2017-09-11 14:19:50,727] [XDE ThreadPool 26] [inventory] [ERROR] - Exception Stack: com.cisco.nm.expression.function.FunctionException: java.lang.reflect.InvocationTargetException

inventory.log:[2017-09-11 14:19:50,727] [ICE Service - Network 11] [inventory] [ERROR] - Feature = feature_config_interface and Procedure = ethernetextra.interfaceCLI with the following error and continuing with other features: com.cisco.nm.expression.function.FunctionException: java.lang.reflect.InvocationTargetException deviceId: 824835; CredentialId: 824835; ManagementIP: 172.24.128.1

inventory.log-[2017-09-11 14:19:50,727] [ICE Service - Network 11] [inventory] [ERROR] - Exception Stack: com.cisco.nm.expression.function.FunctionException: java.lang.reflect.InvocationTargetException

inventory.log-[2017-09-11 14:19:57,937] [XDE ThreadPool 15] [AssociationBuilderUtil] [ERROR] - Cannot locate the ICE metadata file: /ice-metadata/ifm_networkservices_technologies_trustsec/ifm_networkservices_technologies_trustsec.xml.

inventory.log-[2017-09-11 14:45:12,542] [XDE ThreadPool 19] [AssociationBuilderUtil] [ERROR] - Cannot locate the ICE metadata file: /ice-metadata/ifm_networkservices_technologies_trustsec/ifm_networkservices_technologies_trustsec.xml.

inventory.log-[2017-09-11 14:45:15,179] [XDE ThreadPool 21] [inventory] [ERROR] - Exception Stack: java.lang.reflect.InvocationTargetException

inventory.log:[2017-09-11 14:45:15,179] [XDE ThreadPool 21] [inventory] [ERROR] - Exception occured while converting the output of workflow runId 10846309 deviceId: 824835; CredentialId: 824835; ManagementIP: 172.24.128.1

inventory.log-[2017-09-11 14:45:15,180] [XDE ThreadPool 21] [inventory] [ERROR] - Exception Stack: com.cisco.xmp.deviceaccess.businessProcessor.BusinessProcessHookException: java.lang.reflect.InvocationTargetException::sun.reflect.GeneratedMethodAccessor342.invoke(Unknown Source)::sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)::java.lang.reflect.Method.invoke(Method.java:606)::com.cisco.xmp.deviceaccess.businessProcessor.BusinessProcessManager._postProcessModelObject(BusinessProcessManager.java:119)::com.cisco.xmp.deviceaccess.businessProcessor.BusinessProcessManager.postProcessModelObjectWithResponse(BusinessProcessManager.java:146)::com.cisco.xmp.inventory.pojogen.PalXMLToPOJOConvertor.getClassInfo(PalXMLToPOJOConvertor.java:296)::com.cisco.xmp.inventory.pojogen.PalXMLToPOJOConvertor.getClassInfo(PalXMLToPOJOConvertor.java:306)::com.cisco.xmp.inventory.pojogen.PalXMLToPOJOConvertor.parseObjectsFromXML(PalXMLToPOJOConvertor.java:190)::com.cisco.xmp.inventory.ice.collectortask.CollectorFeatureHookTask.featureHook(CollectorFeatureHookTask.java:53)::com.cisco.xmp.inventory.ice.collectortask.CollectorFeatureHookTask.externalFunctionCall(CollectorFeatureHookTask.java:113)::com.cisco.nm.expression.function.system.system.AbstractExternalFunction.callAsync(AbstractExternalFunction.java:86)::com.cisco.nm.expression.function.system.system.HookFunction.callAsync(HookFunction.java:53)::com.cisco.nm.expression.function.FunctionRunnerContext.runFunction(FunctionRunnerContext.java:97)::com.cisco.nm.expression.function.FunctionRunner.runFunctionAsync(FunctionRunner.java:65)::com.cisco.nm.expression.js.packageimport.JavaScriptXdeFunctionObject$JavaScriptXdeFunctionObjectContext.call(JavaScriptXdeFunctionObject.java:56)::com.cisco.nm.expression.js.packageimport.JavaScriptXdeFunctionObject$JavaScriptXdeFunctionObjectContext.access$300(JavaScriptXdeFunctionObject.java:40)::com.cisco.nm.expression.js.packageimport.JavaScriptXdeFunctionObject.call(JavaScriptXdeFunctionObject.java:193)::org.mozilla.javascript.Interpreter.interpretLoop(Interpreter.java:1482)::org.mozilla.javascript.Interpreter.interpret(Interpreter.java:815)::org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:109)::org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:394)::org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3096)::org.mozilla.javascript.Context.callFunctionWithContinuations(Context.java:1172)::org.mozilla.javascript.Context.executeScriptWithContinuations(Context.java:1139)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.execute(JavaScriptCompiler.java:213)::com.cisco.nm.expression.js.JavaScriptCompiler.executeFunction(JavaScriptCompiler.java:155)::com.cisco.nm.expression.js.JavaScriptCompiler.executeExpression(JavaScriptCompiler.java:143)::com.cisco.nm.workflow.steps.StepExpressionJavaScript.executeExpression(StepExpressionJavaScript.java:118)::com.cisco.nm.workflow.steps.JavaScriptStep.run(JavaScriptStep.java:46)::com.cisco.nm.workflow.run.FlowStepContainerRunner.processLoop(FlowStepContainerRunner.java:106)::com.cisco.nm.util.async.AsyncWhileLoop.processLoopInternal(AsyncWhileLoop.java:35)::com.cisco.nm.util.async.AsyncWhileLoop.continueLoop(AsyncWhileLoop.java:53)::com.cisco.nm.workflow.run.FlowStepContainerRunner.runStepResult(FlowStepContainerRunner.java:132)::com.cisco.nm.workflow.steps.JavaScriptStep$1.executeExpressionResult(JavaScriptStep.java:53)::com.cisco.nm.workflow.steps.StepExpressionJavaScript$2.executeExpressionResult(StepExpressionJavaScript.java:121)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.callbackSuccess(JavaScriptCompiler.java:315)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.processResult(JavaScriptCompiler.java:305)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.futureCallback(JavaScriptCompiler.java:254)::com.cisco.nm.expression.js.packageimport.JavaScriptXdeFunctionObject$JavaScriptXdeFunctionObjectContext.functionResult(JavaScriptXdeFunctionObject.java:89)::com.cisco.nm.expression.function.FunctionRunnerContext.functionResult(FunctionRunnerContext.java:170)::com.cisco.nm.expression.function.xep.XepBindingCallingContextRunAsRules.functionResult(XepBindingCallingContextRunAsRules.java:75)::com.cisco.nm.expression.function.FunctionRunnerContext.functionResult(FunctionRunnerContext.java:170)::com.cisco.nm.pal.function.par.ParFileFunctionContext.workflowFinished(ParFileFunctionContext.java:109)::com.cisco.nm.workflow.run.WorkflowRunner.runFlowResult(WorkflowRunner.java:72)::com.cisco.nm.workflow.run.FlowStepContainerRunner.finishedLoopWithSuccess(FlowStepContainerRunner.java:196)::com.cisco.nm.util.async.AsyncWhileLoop.haltLoopWithSuccess(AsyncWhileLoop.java:68)::com.cisco.nm.workflow.run.FlowStepContainerRunner.flowFinishedWithResult(FlowStepContainerRunner.java:208)::com.cisco.nm.workflow.run.FlowStepContainerRunner.runStepResult(FlowStepContainerRunner.java:142)::com.cisco.nm.workflow.steps.ReturnStep$1.executeExpressionResult(ReturnStep.java:56)::com.cisco.nm.workflow.steps.StepExpressionJavaScript$2.executeExpressionResult(StepExpressionJavaScript.java:121)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.callbackSuccess(JavaScriptCompiler.java:315)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.processResult(JavaScriptCompiler.java:305)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.execute(JavaScriptCompiler.java:224)::com.cisco.nm.expression.js.JavaScriptCompiler.executeFunction(JavaScriptCompiler.java:155)::com.cisco.nm.expression.js.JavaScriptCompiler.executeExpression(JavaScriptCompiler.java:143)::com.cisco.nm.workflow.steps.StepExpressionJavaScript.executeExpression(StepExpressionJavaScript.java:118)::com.cisco.nm.workflow.steps.ReturnStep.run(ReturnStep.java:49)::com.cisco.nm.workflow.run.FlowStepContainerRunner.processLoop(FlowStepContainerRunner.java:106)::com.cisco.nm.util.async.AsyncWhileLoop.processLoopInternal(AsyncWhileLoop.java:35)::com.cisco.nm.util.async.AsyncWhileLoop.continueLoop(AsyncWhileLoop.java:53)::com.cisco.nm.workflow.run.FlowStepContainerRunner.runStepResult(FlowStepContainerRunner.java:132)::com.cisco.nm.workflow.steps.HandlerStep$HandlerStepImpl.handlerResult(HandlerStep.java:157)::com.cisco.nm.pal.handler.HandlerCallbackWrapped.run(HandlerCallbackWrapped.java:102)::com.cisco.nm.pal.handler.HandlerCallbackWrapped.makeCallback(HandlerCallbackWrapped.java:90)::com.cisco.nm.pal.handler.HandlerCallbackWrapped.handlerResult(HandlerCallbackWrapped.java:62)::com.cisco.nm.pal.customhandler.cli.CLIHandlerImplBaseShowDocument.finishedIterationWithSuccess(CLIHandlerImplBaseShowDocument.java:74)::com.cisco.nm.util.async.AsyncIterator.haltIterationWithSuccess(AsyncIterator.java:87)::com.cisco.nm.util.async.AsyncIterator.processLoopInternal(AsyncIterator.java:39)::com.cisco.nm.util.async.AsyncIterator.continueIteration(AsyncIterator.java:72)::com.cisco.nm.pal.customhandler.cli.CLIHandlerImplBaseShowDocument.sessionRunnerCallbackResult(CLIHandlerImplBaseShowDocument.java:151)::com.cisco.xmp.utilities.sessionrunner.XMPCustomSessionRunner$XMPCacheCLIHandlerCallback.sessionRunnerCallbackResult(XMPCustomSessionRunner.java:79)::com.cisco.nm.pal.cli.runner.SessionRunnerWorker.continueTidyUp(SessionRunnerWorker.java:305)::com.cisco.nm.pal.cli.runner.SessionRunnerWorker.sessionStopSuccess(SessionRunnerWorker.java:236)::com.cisco.nm.pal.cli.session.CliSessionContextBase.stopSuccess(CliSessionContextBase.java:96)::com.cisco.nm.pal.cli.transport.CliTransportCommunicator.stopSuccess(CliTransportCommunicator.java:227)::com.cisco.nm.pal.cli.transport.ssh2.CliTransportProviderAsyncSSH2.stop(CliTransportProviderAsyncSSH2.java:258)::com.cisco.nm.pal.cli.transport.CliTransportCommunicator.stopAsync(CliTransportCommunicator.java:112)::com.cisco.nm.pal.cli.session.CliSessionContextBase.stopAsync(CliSessionContextBase.java:91)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveSession.stopAsync(CliInteractiveSession.java:35)::com.cisco.nm.pal.cli.runner.SessionRunnerWorker.continueTidyUp(SessionRunnerWorker.java:275)::com.cisco.nm.pal.cli.runner.SessionRunnerWorker.sessionRunCommandSuccess(SessionRunnerWorker.java:199)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveShowCommandRunner.runInteractiveCommandsAsyncSuccess(CliInteractiveShowCommandRunner.java:80)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveCommandRunner.succeeded(CliInteractiveCommandRunner.java:281)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveCommandRunner.responseReceived(CliInteractiveCommandRunner.java:219)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveCommandRunner.access$000(CliInteractiveCommandRunner.java:23)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveCommandRunner$MainCommandCallback.expectResult(CliInteractiveCommandRunner.java:30)::com.cisco.nm.pal.cli.transport.CliTransportCommunicatorExpectEngine$1.run(CliTransportCommunicatorExpectEngine.java:151)::java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)::java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)::com.cisco.nm.run.threadpool.ThreadPool$PooledThread.run(ThreadPool.java:60)

inventory.log:[2017-09-11 14:45:15,181] [XDE ThreadPool 21] [inventory] [ERROR] - Exception executing function [ethernetextra]interfaceCLI runId 10846309 in time 831 hook time 131 deviceId: 824835; CredentialId: 824835; ManagementIP: 172.24.128.1

inventory.log-[2017-09-11 14:45:15,181] [XDE ThreadPool 21] [inventory] [ERROR] - Exception Stack: com.cisco.nm.expression.function.FunctionException: java.lang.reflect.InvocationTargetException

inventory.log:[2017-09-11 14:45:15,181] [ICE Service - Network 9] [inventory] [ERROR] - Feature = feature_config_interface and Procedure = ethernetextra.interfaceCLI with the following error and continuing with other features: com.cisco.nm.expression.function.FunctionException: java.lang.reflect.InvocationTargetException deviceId: 824835; CredentialId: 824835; ManagementIP: 172.24.128.1

inventory.log-[2017-09-11 14:45:15,182] [ICE Service - Network 9] [inventory] [ERROR] - Exception Stack: com.cisco.nm.expression.function.FunctionException: java.lang.reflect.InvocationTargetException

inventory.log-[2017-09-11 14:45:21,982] [XDE ThreadPool 7] [AssociationBuilderUtil] [ERROR] - Cannot locate the ICE metadata file: /ice-metadata/ifm_networkservices_technologies_trustsec/ifm_networkservices_technologies_trustsec.xml.

inventory.log-[2017-09-11 14:46:11,741] [XDE ThreadPool 13] [inventory] [ERROR] - Exception executing function [com.cisco.ef.config.xde.lisp7k-inventory]procedure runId 10846387 in time 2 hook time 0 deviceId: 824834; CredentialId: 824834; ManagementIP: 172.24.170.16

inventory.log-[2017-09-11 14:46:11,742] [XDE ThreadPool 13] [inventory] [ERROR] - Exception Stack: com.cisco.nm.pal.client.PALException: <palError><deviceId>824834</deviceId><code>NO_PAL_RULE</code><message>No rule test for function "[com.cisco.ef.config.xde.lisp7k-inventory]lisp7k-inventory.xpa/xmp-im-lisp-module/xmp-im-lisp-module.par" matched for device "824834". For a rule test to succeed, every test variable/value in the rule must be set on the device. Either this device is not recognized by your inventory provider or the set of attributes for the device does not match any rule test in the action. </message><credential name="software">NX-OS</credential><credential name="DEVICE_NAME">824834</credential><credential name="DEVICE_ID">824834</credential></palError>

inventory.log-[2017-09-11 15:10:41,273] [XDE ThreadPool 21] [inventory] [ERROR] - Exception Stack: java.lang.reflect.InvocationTargetException

inventory.log:[2017-09-11 15:10:41,273] [XDE ThreadPool 21] [inventory] [ERROR] - Exception occured while converting the output of workflow runId 10850105 deviceId: 824835; CredentialId: 824835; ManagementIP: 172.24.128.1

inventory.log-[2017-09-11 15:10:41,275] [XDE ThreadPool 21] [inventory] [ERROR] - Exception Stack: com.cisco.xmp.deviceaccess.businessProcessor.BusinessProcessHookException: java.lang.reflect.InvocationTargetException::sun.reflect.GeneratedMethodAccessor342.invoke(Unknown Source)::sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)::java.lang.reflect.Method.invoke(Method.java:606)::com.cisco.xmp.deviceaccess.businessProcessor.BusinessProcessManager._postProcessModelObject(BusinessProcessManager.java:119)::com.cisco.xmp.deviceaccess.businessProcessor.BusinessProcessManager.postProcessModelObjectWithResponse(BusinessProcessManager.java:146)::com.cisco.xmp.inventory.pojogen.PalXMLToPOJOConvertor.getClassInfo(PalXMLToPOJOConvertor.java:296)::com.cisco.xmp.inventory.pojogen.PalXMLToPOJOConvertor.getClassInfo(PalXMLToPOJOConvertor.java:306)::com.cisco.xmp.inventory.pojogen.PalXMLToPOJOConvertor.parseObjectsFromXML(PalXMLToPOJOConvertor.java:190)::com.cisco.xmp.inventory.ice.collectortask.CollectorFeatureHookTask.featureHook(CollectorFeatureHookTask.java:53)::com.cisco.xmp.inventory.ice.collectortask.CollectorFeatureHookTask.externalFunctionCall(CollectorFeatureHookTask.java:113)::com.cisco.nm.expression.function.system.system.AbstractExternalFunction.callAsync(AbstractExternalFunction.java:86)::com.cisco.nm.expression.function.system.system.HookFunction.callAsync(HookFunction.java:53)::com.cisco.nm.expression.function.FunctionRunnerContext.runFunction(FunctionRunnerContext.java:97)::com.cisco.nm.expression.function.FunctionRunner.runFunctionAsync(FunctionRunner.java:65)::com.cisco.nm.expression.js.packageimport.JavaScriptXdeFunctionObject$JavaScriptXdeFunctionObjectContext.call(JavaScriptXdeFunctionObject.java:56)::com.cisco.nm.expression.js.packageimport.JavaScriptXdeFunctionObject$JavaScriptXdeFunctionObjectContext.access$300(JavaScriptXdeFunctionObject.java:40)::com.cisco.nm.expression.js.packageimport.JavaScriptXdeFunctionObject.call(JavaScriptXdeFunctionObject.java:193)::org.mozilla.javascript.Interpreter.interpretLoop(Interpreter.java:1482)::org.mozilla.javascript.Interpreter.interpret(Interpreter.java:815)::org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:109)::org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:394)::org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3096)::org.mozilla.javascript.Context.callFunctionWithContinuations(Context.java:1172)::org.mozilla.javascript.Context.executeScriptWithContinuations(Context.java:1139)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.execute(JavaScriptCompiler.java:213)::com.cisco.nm.expression.js.JavaScriptCompiler.executeFunction(JavaScriptCompiler.java:155)::com.cisco.nm.expression.js.JavaScriptCompiler.executeExpression(JavaScriptCompiler.java:143)::com.cisco.nm.workflow.steps.StepExpressionJavaScript.executeExpression(StepExpressionJavaScript.java:118)::com.cisco.nm.workflow.steps.JavaScriptStep.run(JavaScriptStep.java:46)::com.cisco.nm.workflow.run.FlowStepContainerRunner.processLoop(FlowStepContainerRunner.java:106)::com.cisco.nm.util.async.AsyncWhileLoop.processLoopInternal(AsyncWhileLoop.java:35)::com.cisco.nm.util.async.AsyncWhileLoop.continueLoop(AsyncWhileLoop.java:53)::com.cisco.nm.workflow.run.FlowStepContainerRunner.runStepResult(FlowStepContainerRunner.java:132)::com.cisco.nm.workflow.steps.JavaScriptStep$1.executeExpressionResult(JavaScriptStep.java:53)::com.cisco.nm.workflow.steps.StepExpressionJavaScript$2.executeExpressionResult(StepExpressionJavaScript.java:121)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.callbackSuccess(JavaScriptCompiler.java:315)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.processResult(JavaScriptCompiler.java:305)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.futureCallback(JavaScriptCompiler.java:254)::com.cisco.nm.expression.js.packageimport.JavaScriptXdeFunctionObject$JavaScriptXdeFunctionObjectContext.functionResult(JavaScriptXdeFunctionObject.java:89)::com.cisco.nm.expression.function.FunctionRunnerContext.functionResult(FunctionRunnerContext.java:170)::com.cisco.nm.expression.function.xep.XepBindingCallingContextRunAsRules.functionResult(XepBindingCallingContextRunAsRules.java:75)::com.cisco.nm.expression.function.FunctionRunnerContext.functionResult(FunctionRunnerContext.java:170)::com.cisco.nm.pal.function.par.ParFileFunctionContext.workflowFinished(ParFileFunctionContext.java:109)::com.cisco.nm.workflow.run.WorkflowRunner.runFlowResult(WorkflowRunner.java:72)::com.cisco.nm.workflow.run.FlowStepContainerRunner.finishedLoopWithSuccess(FlowStepContainerRunner.java:196)::com.cisco.nm.util.async.AsyncWhileLoop.haltLoopWithSuccess(AsyncWhileLoop.java:68)::com.cisco.nm.workflow.run.FlowStepContainerRunner.flowFinishedWithResult(FlowStepContainerRunner.java:208)::com.cisco.nm.workflow.run.FlowStepContainerRunner.runStepResult(FlowStepContainerRunner.java:142)::com.cisco.nm.workflow.steps.ReturnStep$1.executeExpressionResult(ReturnStep.java:56)::com.cisco.nm.workflow.steps.StepExpressionJavaScript$2.executeExpressionResult(StepExpressionJavaScript.java:121)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.callbackSuccess(JavaScriptCompiler.java:315)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.processResult(JavaScriptCompiler.java:305)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.execute(JavaScriptCompiler.java:224)::com.cisco.nm.expression.js.JavaScriptCompiler.executeFunction(JavaScriptCompiler.java:155)::com.cisco.nm.expression.js.JavaScriptCompiler.executeExpression(JavaScriptCompiler.java:143)::com.cisco.nm.workflow.steps.StepExpressionJavaScript.executeExpression(StepExpressionJavaScript.java:118)::com.cisco.nm.workflow.steps.ReturnStep.run(ReturnStep.java:49)::com.cisco.nm.workflow.run.FlowStepContainerRunner.processLoop(FlowStepContainerRunner.java:106)::com.cisco.nm.util.async.AsyncWhileLoop.processLoopInternal(AsyncWhileLoop.java:35)::com.cisco.nm.util.async.AsyncWhileLoop.continueLoop(AsyncWhileLoop.java:53)::com.cisco.nm.workflow.run.FlowStepContainerRunner.runStepResult(FlowStepContainerRunner.java:132)::com.cisco.nm.workflow.steps.HandlerStep$HandlerStepImpl.handlerResult(HandlerStep.java:157)::com.cisco.nm.pal.handler.HandlerCallbackWrapped.run(HandlerCallbackWrapped.java:102)::com.cisco.nm.pal.handler.HandlerCallbackWrapped.makeCallback(HandlerCallbackWrapped.java:90)::com.cisco.nm.pal.handler.HandlerCallbackWrapped.handlerResult(HandlerCallbackWrapped.java:62)::com.cisco.nm.pal.customhandler.cli.CLIHandlerImplBaseShowDocument.finishedIterationWithSuccess(CLIHandlerImplBaseShowDocument.java:74)::com.cisco.nm.util.async.AsyncIterator.haltIterationWithSuccess(AsyncIterator.java:87)::com.cisco.nm.util.async.AsyncIterator.processLoopInternal(AsyncIterator.java:39)::com.cisco.nm.util.async.AsyncIterator.continueIteration(AsyncIterator.java:72)::com.cisco.nm.pal.customhandler.cli.CLIHandlerImplBaseShowDocument.sessionRunnerCallbackResult(CLIHandlerImplBaseShowDocument.java:151)::com.cisco.xmp.utilities.sessionrunner.XMPCustomSessionRunner$XMPCacheCLIHandlerCallback.sessionRunnerCallbackResult(XMPCustomSessionRunner.java:79)::com.cisco.nm.pal.cli.runner.SessionRunnerWorker.continueTidyUp(SessionRunnerWorker.java:305)::com.cisco.nm.pal.cli.runner.SessionRunnerWorker.sessionStopSuccess(SessionRunnerWorker.java:236)::com.cisco.nm.pal.cli.session.CliSessionContextBase.stopSuccess(CliSessionContextBase.java:96)::com.cisco.nm.pal.cli.transport.CliTransportCommunicator.stopSuccess(CliTransportCommunicator.java:227)::com.cisco.nm.pal.cli.transport.ssh2.CliTransportProviderAsyncSSH2.stop(CliTransportProviderAsyncSSH2.java:258)::com.cisco.nm.pal.cli.transport.CliTransportCommunicator.stopAsync(CliTransportCommunicator.java:112)::com.cisco.nm.pal.cli.session.CliSessionContextBase.stopAsync(CliSessionContextBase.java:91)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveSession.stopAsync(CliInteractiveSession.java:35)::com.cisco.nm.pal.cli.runner.SessionRunnerWorker.continueTidyUp(SessionRunnerWorker.java:275)::com.cisco.nm.pal.cli.runner.SessionRunnerWorker.sessionRunCommandSuccess(SessionRunnerWorker.java:199)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveShowCommandRunner.runInteractiveCommandsAsyncSuccess(CliInteractiveShowCommandRunner.java:80)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveCommandRunner.succeeded(CliInteractiveCommandRunner.java:281)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveCommandRunner.responseReceived(CliInteractiveCommandRunner.java:219)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveCommandRunner.access$000(CliInteractiveCommandRunner.java:23)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveCommandRunner$MainCommandCallback.expectResult(CliInteractiveCommandRunner.java:30)::com.cisco.nm.pal.cli.transport.CliTransportCommunicatorExpectEngine$1.run(CliTransportCommunicatorExpectEngine.java:151)::java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)::java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)::com.cisco.nm.run.threadpool.ThreadPool$PooledThread.run(ThreadPool.java:60)

inventory.log:[2017-09-11 15:10:41,275] [XDE ThreadPool 21] [inventory] [ERROR] - Exception executing function [ethernetextra]interfaceCLI runId 10850105 in time 888 hook time 137 deviceId: 824835; CredentialId: 824835; ManagementIP: 172.24.128.1

inventory.log-[2017-09-11 15:10:41,276] [XDE ThreadPool 21] [inventory] [ERROR] - Exception Stack: com.cisco.nm.expression.function.FunctionException: java.lang.reflect.InvocationTargetException

inventory.log:[2017-09-11 15:10:41,276] [ICE Service - Network 8] [inventory] [ERROR] - Feature = feature_config_interface and Procedure = ethernetextra.interfaceCLI with the following error and continuing with other features: com.cisco.nm.expression.function.FunctionException: java.lang.reflect.InvocationTargetException deviceId: 824835; CredentialId: 824835; ManagementIP: 172.24.128.1

inventory.log-[2017-09-11 15:10:41,276] [ICE Service - Network 8] [inventory] [ERROR] - Exception Stack: com.cisco.nm.expression.function.FunctionException: java.lang.reflect.InvocationTargetException

inventory.log-[2017-09-11 15:10:43,815] [XDE ThreadPool 20] [AssociationBuilderUtil] [ERROR] - Cannot locate the ICE metadata file: /ice-metadata/ifm_networkservices_technologies_trustsec/ifm_networkservices_technologies_trustsec.xml.

inventory.log-[2017-09-11 15:10:47,966] [XDE ThreadPool 26] [AssociationBuilderUtil] [ERROR] - Cannot locate the ICE metadata file: /ice-metadata/ifm_networkservices_technologies_trustsec/ifm_networkservices_technologies_trustsec.xml.

inventory.log-[2017-09-11 15:10:52,539] [XDE ThreadPool 9] [AssociationBuilderUtil] [ERROR] - Cannot locate the ICE metadata file: /ice-metadata/ifm_networkservices_technologies_trustsec/ifm_networkservices_technologies_trustsec.xml.

inventory.log-[2017-09-11 15:10:53,413] [XDE ThreadPool 21] [AssociationBuilderUtil] [ERROR] - Cannot locate the ICE metadata file: /ice-metadata/ifm_networkservices_technologies_trustsec/ifm_networkservices_technologies_trustsec.xml.

inventory.log:[2017-09-11 15:36:05,784] [XDE ThreadPool 5] [inventory] [ERROR] - Exception occured while converting the output of workflow runId 10854081 deviceId: 824835; CredentialId: 824835; ManagementIP: 172.24.128.1

inventory.log-[2017-09-11 15:36:05,785] [XDE ThreadPool 5] [inventory] [ERROR] - Exception Stack: com.cisco.xmp.deviceaccess.businessProcessor.BusinessProcessHookException: java.lang.reflect.InvocationTargetException::sun.reflect.GeneratedMethodAccessor342.invoke(Unknown Source)::sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)::java.lang.reflect.Method.invoke(Method.java:606)::com.cisco.xmp.deviceaccess.businessProcessor.BusinessProcessManager._postProcessModelObject(BusinessProcessManager.java:119)::com.cisco.xmp.deviceaccess.businessProcessor.BusinessProcessManager.postProcessModelObjectWithResponse(BusinessProcessManager.java:146)::com.cisco.xmp.inventory.pojogen.PalXMLToPOJOConvertor.getClassInfo(PalXMLToPOJOConvertor.java:296)::com.cisco.xmp.inventory.pojogen.PalXMLToPOJOConvertor.getClassInfo(PalXMLToPOJOConvertor.java:306)::com.cisco.xmp.inventory.pojogen.PalXMLToPOJOConvertor.parseObjectsFromXML(PalXMLToPOJOConvertor.java:190)::com.cisco.xmp.inventory.ice.collectortask.CollectorFeatureHookTask.featureHook(CollectorFeatureHookTask.java:53)::com.cisco.xmp.inventory.ice.collectortask.CollectorFeatureHookTask.externalFunctionCall(CollectorFeatureHookTask.java:113)::com.cisco.nm.expression.function.system.system.AbstractExternalFunction.callAsync(AbstractExternalFunction.java:86)::com.cisco.nm.expression.function.system.system.HookFunction.callAsync(HookFunction.java:53)::com.cisco.nm.expression.function.FunctionRunnerContext.runFunction(FunctionRunnerContext.java:97)::com.cisco.nm.expression.function.FunctionRunner.runFunctionAsync(FunctionRunner.java:65)::com.cisco.nm.expression.js.packageimport.JavaScriptXdeFunctionObject$JavaScriptXdeFunctionObjectContext.call(JavaScriptXdeFunctionObject.java:56)::com.cisco.nm.expression.js.packageimport.JavaScriptXdeFunctionObject$JavaScriptXdeFunctionObjectContext.access$300(JavaScriptXdeFunctionObject.java:40)::com.cisco.nm.expression.js.packageimport.JavaScriptXdeFunctionObject.call(JavaScriptXdeFunctionObject.java:193)::org.mozilla.javascript.Interpreter.interpretLoop(Interpreter.java:1482)::org.mozilla.javascript.Interpreter.interpret(Interpreter.java:815)::org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:109)::org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:394)::org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3096)::org.mozilla.javascript.Context.callFunctionWithContinuations(Context.java:1172)::org.mozilla.javascript.Context.executeScriptWithContinuations(Context.java:1139)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.execute(JavaScriptCompiler.java:213)::com.cisco.nm.expression.js.JavaScriptCompiler.executeFunction(JavaScriptCompiler.java:155)::com.cisco.nm.expression.js.JavaScriptCompiler.executeExpression(JavaScriptCompiler.java:143)::com.cisco.nm.workflow.steps.StepExpressionJavaScript.executeExpression(StepExpressionJavaScript.java:118)::com.cisco.nm.workflow.steps.JavaScriptStep.run(JavaScriptStep.java:46)::com.cisco.nm.workflow.run.FlowStepContainerRunner.processLoop(FlowStepContainerRunner.java:106)::com.cisco.nm.util.async.AsyncWhileLoop.processLoopInternal(AsyncWhileLoop.java:35)::com.cisco.nm.util.async.AsyncWhileLoop.continueLoop(AsyncWhileLoop.java:53)::com.cisco.nm.workflow.run.FlowStepContainerRunner.runStepResult(FlowStepContainerRunner.java:132)::com.cisco.nm.workflow.steps.JavaScriptStep$1.executeExpressionResult(JavaScriptStep.java:53)::com.cisco.nm.workflow.steps.StepExpressionJavaScript$2.executeExpressionResult(StepExpressionJavaScript.java:121)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.callbackSuccess(JavaScriptCompiler.java:315)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.processResult(JavaScriptCompiler.java:305)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.futureCallback(JavaScriptCompiler.java:254)::com.cisco.nm.expression.js.packageimport.JavaScriptXdeFunctionObject$JavaScriptXdeFunctionObjectContext.functionResult(JavaScriptXdeFunctionObject.java:89)::com.cisco.nm.expression.function.FunctionRunnerContext.functionResult(FunctionRunnerContext.java:170)::com.cisco.nm.expression.function.xep.XepBindingCallingContextRunAsRules.functionResult(XepBindingCallingContextRunAsRules.java:75)::com.cisco.nm.expression.function.FunctionRunnerContext.functionResult(FunctionRunnerContext.java:170)::com.cisco.nm.pal.function.par.ParFileFunctionContext.workflowFinished(ParFileFunctionContext.java:109)::com.cisco.nm.workflow.run.WorkflowRunner.runFlowResult(WorkflowRunner.java:72)::com.cisco.nm.workflow.run.FlowStepContainerRunner.finishedLoopWithSuccess(FlowStepContainerRunner.java:196)::com.cisco.nm.util.async.AsyncWhileLoop.haltLoopWithSuccess(AsyncWhileLoop.java:68)::com.cisco.nm.workflow.run.FlowStepContainerRunner.flowFinishedWithResult(FlowStepContainerRunner.java:208)::com.cisco.nm.workflow.run.FlowStepContainerRunner.runStepResult(FlowStepContainerRunner.java:142)::com.cisco.nm.workflow.steps.ReturnStep$1.executeExpressionResult(ReturnStep.java:56)::com.cisco.nm.workflow.steps.StepExpressionJavaScript$2.executeExpressionResult(StepExpressionJavaScript.java:121)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.callbackSuccess(JavaScriptCompiler.java:315)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.processResult(JavaScriptCompiler.java:305)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.execute(JavaScriptCompiler.java:224)::com.cisco.nm.expression.js.JavaScriptCompiler.executeFunction(JavaScriptCompiler.java:155)::com.cisco.nm.expression.js.JavaScriptCompiler.executeExpression(JavaScriptCompiler.java:143)::com.cisco.nm.workflow.steps.StepExpressionJavaScript.executeExpression(StepExpressionJavaScript.java:118)::com.cisco.nm.workflow.steps.ReturnStep.run(ReturnStep.java:49)::com.cisco.nm.workflow.run.FlowStepContainerRunner.processLoop(FlowStepContainerRunner.java:106)::com.cisco.nm.util.async.AsyncWhileLoop.processLoopInternal(AsyncWhileLoop.java:35)::com.cisco.nm.util.async.AsyncWhileLoop.continueLoop(AsyncWhileLoop.java:53)::com.cisco.nm.workflow.run.FlowStepContainerRunner.runStepResult(FlowStepContainerRunner.java:132)::com.cisco.nm.workflow.steps.HandlerStep$HandlerStepImpl.handlerResult(HandlerStep.java:157)::com.cisco.nm.pal.handler.HandlerCallbackWrapped.run(HandlerCallbackWrapped.java:102)::com.cisco.nm.pal.handler.HandlerCallbackWrapped.makeCallback(HandlerCallbackWrapped.java:90)::com.cisco.nm.pal.handler.HandlerCallbackWrapped.handlerResult(HandlerCallbackWrapped.java:62)::com.cisco.nm.pal.customhandler.cli.CLIHandlerImplBaseShowDocument.finishedIterationWithSuccess(CLIHandlerImplBaseShowDocument.java:74)::com.cisco.nm.util.async.AsyncIterator.haltIterationWithSuccess(AsyncIterator.java:87)::com.cisco.nm.util.async.AsyncIterator.processLoopInternal(AsyncIterator.java:39)::com.cisco.nm.util.async.AsyncIterator.continueIteration(AsyncIterator.java:72)::com.cisco.nm.pal.customhandler.cli.CLIHandlerImplBaseShowDocument.sessionRunnerCallbackResult(CLIHandlerImplBaseShowDocument.java:151)::com.cisco.xmp.utilities.sessionrunner.XMPCustomSessionRunner$XMPCacheCLIHandlerCallback.sessionRunnerCallbackResult(XMPCustomSessionRunner.java:79)::com.cisco.nm.pal.cli.runner.SessionRunnerWorker.continueTidyUp(SessionRunnerWorker.java:305)::com.cisco.nm.pal.cli.runner.SessionRunnerWorker.sessionStopSuccess(SessionRunnerWorker.java:236)::com.cisco.nm.pal.cli.session.CliSessionContextBase.stopSuccess(CliSessionContextBase.java:96)::com.cisco.nm.pal.cli.transport.CliTransportCommunicator.stopSuccess(CliTransportCommunicator.java:227)::com.cisco.nm.pal.cli.transport.ssh2.CliTransportProviderAsyncSSH2.stop(CliTransportProviderAsyncSSH2.java:258)::com.cisco.nm.pal.cli.transport.CliTransportCommunicator.stopAsync(CliTransportCommunicator.java:112)::com.cisco.nm.pal.cli.session.CliSessionContextBase.stopAsync(CliSessionContextBase.java:91)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveSession.stopAsync(CliInteractiveSession.java:35)::com.cisco.nm.pal.cli.runner.SessionRunnerWorker.continueTidyUp(SessionRunnerWorker.java:275)::com.cisco.nm.pal.cli.runner.SessionRunnerWorker.sessionRunCommandSuccess(SessionRunnerWorker.java:199)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveShowCommandRunner.runInteractiveCommandsAsyncSuccess(CliInteractiveShowCommandRunner.java:80)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveCommandRunner.succeeded(CliInteractiveCommandRunner.java:281)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveCommandRunner.responseReceived(CliInteractiveCommandRunner.java:219)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveCommandRunner.access$000(CliInteractiveCommandRunner.java:23)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveCommandRunner$MainCommandCallback.expectResult(CliInteractiveCommandRunner.java:30)::com.cisco.nm.pal.cli.transport.CliTransportCommunicatorExpectEngine$1.run(CliTransportCommunicatorExpectEngine.java:151)::java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)::java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)::com.cisco.nm.run.threadpool.ThreadPool$PooledThread.run(ThreadPool.java:60)

inventory.log:[2017-09-11 15:36:05,786] [XDE ThreadPool 5] [inventory] [ERROR] - Exception executing function [ethernetextra]interfaceCLI runId 10854081 in time 822 hook time 163 deviceId: 824835; CredentialId: 824835; ManagementIP: 172.24.128.1

inventory.log-[2017-09-11 15:36:05,786] [XDE ThreadPool 5] [inventory] [ERROR] - Exception Stack: com.cisco.nm.expression.function.FunctionException: java.lang.reflect.InvocationTargetException

inventory.log:[2017-09-11 15:36:05,786] [ICE Service - Network 0] [inventory] [ERROR] - Feature = feature_config_interface and Procedure = ethernetextra.interfaceCLI with the following error and continuing with other features: com.cisco.nm.expression.function.FunctionException: java.lang.reflect.InvocationTargetException deviceId: 824835; CredentialId: 824835; ManagementIP: 172.24.128.1

inventory.log-[2017-09-11 15:36:05,787] [ICE Service - Network 0] [inventory] [ERROR] - Exception Stack: com.cisco.nm.expression.function.FunctionException: java.lang.reflect.InvocationTargetException

inventory.log-[2017-09-11 15:36:12,962] [XDE ThreadPool 1] [AssociationBuilderUtil] [ERROR] - Cannot locate the ICE metadata file: /ice-metadata/ifm_networkservices_technologies_trustsec/ifm_networkservices_technologies_trustsec.xml.

inventory.log-[2017-09-11 16:01:30,286] [XDE ThreadPool 10] [inventory] [ERROR] - Exception Stack: java.lang.reflect.InvocationTargetException

inventory.log:[2017-09-11 16:01:30,286] [XDE ThreadPool 10] [inventory] [ERROR] - Exception occured while converting the output of workflow runId 10858579 deviceId: 824835; CredentialId: 824835; ManagementIP: 172.24.128.1

inventory.log-[2017-09-11 16:01:30,287] [XDE ThreadPool 10] [inventory] [ERROR] - Exception Stack: com.cisco.xmp.deviceaccess.businessProcessor.BusinessProcessHookException: java.lang.reflect.InvocationTargetException::sun.reflect.GeneratedMethodAccessor342.invoke(Unknown Source)::sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)::java.lang.reflect.Method.invoke(Method.java:606)::com.cisco.xmp.deviceaccess.businessProcessor.BusinessProcessManager._postProcessModelObject(BusinessProcessManager.java:119)::com.cisco.xmp.deviceaccess.businessProcessor.BusinessProcessManager.postProcessModelObjectWithResponse(BusinessProcessManager.java:146)::com.cisco.xmp.inventory.pojogen.PalXMLToPOJOConvertor.getClassInfo(PalXMLToPOJOConvertor.java:296)::com.cisco.xmp.inventory.pojogen.PalXMLToPOJOConvertor.getClassInfo(PalXMLToPOJOConvertor.java:306)::com.cisco.xmp.inventory.pojogen.PalXMLToPOJOConvertor.parseObjectsFromXML(PalXMLToPOJOConvertor.java:190)::com.cisco.xmp.inventory.ice.collectortask.CollectorFeatureHookTask.featureHook(CollectorFeatureHookTask.java:53)::com.cisco.xmp.inventory.ice.collectortask.CollectorFeatureHookTask.externalFunctionCall(CollectorFeatureHookTask.java:113)::com.cisco.nm.expression.function.system.system.AbstractExternalFunction.callAsync(AbstractExternalFunction.java:86)::com.cisco.nm.expression.function.system.system.HookFunction.callAsync(HookFunction.java:53)::com.cisco.nm.expression.function.FunctionRunnerContext.runFunction(FunctionRunnerContext.java:97)::com.cisco.nm.expression.function.FunctionRunner.runFunctionAsync(FunctionRunner.java:65)::com.cisco.nm.expression.js.packageimport.JavaScriptXdeFunctionObject$JavaScriptXdeFunctionObjectContext.call(JavaScriptXdeFunctionObject.java:56)::com.cisco.nm.expression.js.packageimport.JavaScriptXdeFunctionObject$JavaScriptXdeFunctionObjectContext.access$300(JavaScriptXdeFunctionObject.java:40)::com.cisco.nm.expression.js.packageimport.JavaScriptXdeFunctionObject.call(JavaScriptXdeFunctionObject.java:193)::org.mozilla.javascript.Interpreter.interpretLoop(Interpreter.java:1482)::org.mozilla.javascript.Interpreter.interpret(Interpreter.java:815)::org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:109)::org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:394)::org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3096)::org.mozilla.javascript.Context.callFunctionWithContinuations(Context.java:1172)::org.mozilla.javascript.Context.executeScriptWithContinuations(Context.java:1139)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.execute(JavaScriptCompiler.java:213)::com.cisco.nm.expression.js.JavaScriptCompiler.executeFunction(JavaScriptCompiler.java:155)::com.cisco.nm.expression.js.JavaScriptCompiler.executeExpression(JavaScriptCompiler.java:143)::com.cisco.nm.workflow.steps.StepExpressionJavaScript.executeExpression(StepExpressionJavaScript.java:118)::com.cisco.nm.workflow.steps.JavaScriptStep.run(JavaScriptStep.java:46)::com.cisco.nm.workflow.run.FlowStepContainerRunner.processLoop(FlowStepContainerRunner.java:106)::com.cisco.nm.util.async.AsyncWhileLoop.processLoopInternal(AsyncWhileLoop.java:35)::com.cisco.nm.util.async.AsyncWhileLoop.continueLoop(AsyncWhileLoop.java:53)::com.cisco.nm.workflow.run.FlowStepContainerRunner.runStepResult(FlowStepContainerRunner.java:132)::com.cisco.nm.workflow.steps.JavaScriptStep$1.executeExpressionResult(JavaScriptStep.java:53)::com.cisco.nm.workflow.steps.StepExpressionJavaScript$2.executeExpressionResult(StepExpressionJavaScript.java:121)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.callbackSuccess(JavaScriptCompiler.java:315)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.processResult(JavaScriptCompiler.java:305)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.futureCallback(JavaScriptCompiler.java:254)::com.cisco.nm.expression.js.packageimport.JavaScriptXdeFunctionObject$JavaScriptXdeFunctionObjectContext.functionResult(JavaScriptXdeFunctionObject.java:89)::com.cisco.nm.expression.function.FunctionRunnerContext.functionResult(FunctionRunnerContext.java:170)::com.cisco.nm.expression.function.xep.XepBindingCallingContextRunAsRules.functionResult(XepBindingCallingContextRunAsRules.java:75)::com.cisco.nm.expression.function.FunctionRunnerContext.functionResult(FunctionRunnerContext.java:170)::com.cisco.nm.pal.function.par.ParFileFunctionContext.workflowFinished(ParFileFunctionContext.java:109)::com.cisco.nm.workflow.run.WorkflowRunner.runFlowResult(WorkflowRunner.java:72)::com.cisco.nm.workflow.run.FlowStepContainerRunner.finishedLoopWithSuccess(FlowStepContainerRunner.java:196)::com.cisco.nm.util.async.AsyncWhileLoop.haltLoopWithSuccess(AsyncWhileLoop.java:68)::com.cisco.nm.workflow.run.FlowStepContainerRunner.flowFinishedWithResult(FlowStepContainerRunner.java:208)::com.cisco.nm.workflow.run.FlowStepContainerRunner.runStepResult(FlowStepContainerRunner.java:142)::com.cisco.nm.workflow.steps.ReturnStep$1.executeExpressionResult(ReturnStep.java:56)::com.cisco.nm.workflow.steps.StepExpressionJavaScript$2.executeExpressionResult(StepExpressionJavaScript.java:121)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.callbackSuccess(JavaScriptCompiler.java:315)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.processResult(JavaScriptCompiler.java:305)::com.cisco.nm.expression.js.JavaScriptCompiler$JavaScriptExecutor.execute(JavaScriptCompiler.java:224)::com.cisco.nm.expression.js.JavaScriptCompiler.executeFunction(JavaScriptCompiler.java:155)::com.cisco.nm.expression.js.JavaScriptCompiler.executeExpression(JavaScriptCompiler.java:143)::com.cisco.nm.workflow.steps.StepExpressionJavaScript.executeExpression(StepExpressionJavaScript.java:118)::com.cisco.nm.workflow.steps.ReturnStep.run(ReturnStep.java:49)::com.cisco.nm.workflow.run.FlowStepContainerRunner.processLoop(FlowStepContainerRunner.java:106)::com.cisco.nm.util.async.AsyncWhileLoop.processLoopInternal(AsyncWhileLoop.java:35)::com.cisco.nm.util.async.AsyncWhileLoop.continueLoop(AsyncWhileLoop.java:53)::com.cisco.nm.workflow.run.FlowStepContainerRunner.runStepResult(FlowStepContainerRunner.java:132)::com.cisco.nm.workflow.steps.HandlerStep$HandlerStepImpl.handlerResult(HandlerStep.java:157)::com.cisco.nm.pal.handler.HandlerCallbackWrapped.run(HandlerCallbackWrapped.java:102)::com.cisco.nm.pal.handler.HandlerCallbackWrapped.makeCallback(HandlerCallbackWrapped.java:90)::com.cisco.nm.pal.handler.HandlerCallbackWrapped.handlerResult(HandlerCallbackWrapped.java:62)::com.cisco.nm.pal.customhandler.cli.CLIHandlerImplBaseShowDocument.finishedIterationWithSuccess(CLIHandlerImplBaseShowDocument.java:74)::com.cisco.nm.util.async.AsyncIterator.haltIterationWithSuccess(AsyncIterator.java:87)::com.cisco.nm.util.async.AsyncIterator.processLoopInternal(AsyncIterator.java:39)::com.cisco.nm.util.async.AsyncIterator.continueIteration(AsyncIterator.java:72)::com.cisco.nm.pal.customhandler.cli.CLIHandlerImplBaseShowDocument.sessionRunnerCallbackResult(CLIHandlerImplBaseShowDocument.java:151)::com.cisco.xmp.utilities.sessionrunner.XMPCustomSessionRunner$XMPCacheCLIHandlerCallback.sessionRunnerCallbackResult(XMPCustomSessionRunner.java:79)::com.cisco.nm.pal.cli.runner.SessionRunnerWorker.continueTidyUp(SessionRunnerWorker.java:305)::com.cisco.nm.pal.cli.runner.SessionRunnerWorker.sessionStopSuccess(SessionRunnerWorker.java:236)::com.cisco.nm.pal.cli.session.CliSessionContextBase.stopSuccess(CliSessionContextBase.java:96)::com.cisco.nm.pal.cli.transport.CliTransportCommunicator.stopSuccess(CliTransportCommunicator.java:227)::com.cisco.nm.pal.cli.transport.ssh2.CliTransportProviderAsyncSSH2.stop(CliTransportProviderAsyncSSH2.java:258)::com.cisco.nm.pal.cli.transport.CliTransportCommunicator.stopAsync(CliTransportCommunicator.java:112)::com.cisco.nm.pal.cli.session.CliSessionContextBase.stopAsync(CliSessionContextBase.java:91)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveSession.stopAsync(CliInteractiveSession.java:35)::com.cisco.nm.pal.cli.runner.SessionRunnerWorker.continueTidyUp(SessionRunnerWorker.java:275)::com.cisco.nm.pal.cli.runner.SessionRunnerWorker.sessionRunCommandSuccess(SessionRunnerWorker.java:199)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveShowCommandRunner.runInteractiveCommandsAsyncSuccess(CliInteractiveShowCommandRunner.java:80)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveCommandRunner.succeeded(CliInteractiveCommandRunner.java:281)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveCommandRunner.responseReceived(CliInteractiveCommandRunner.java:219)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveCommandRunner.access$000(CliInteractiveCommandRunner.java:23)::com.cisco.nm.pal.cli.session.interactive.CliInteractiveCommandRunner$MainCommandCallback.expectResult(CliInteractiveCommandRunner.java:30)::com.cisco.nm.pal.cli.transport.CliTransportCommunicatorExpectEngine$1.run(CliTransportCommunicatorExpectEngine.java:151)::java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)::java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)::com.cisco.nm.run.threadpool.ThreadPool$PooledThread.run(ThreadPool.java:60)

inventory.log:[2017-09-11 16:01:30,288] [XDE ThreadPool 10] [inventory] [ERROR] - Exception executing function [ethernetextra]interfaceCLI runId 10858579 in time 1799 hook time 136 deviceId: 824835; CredentialId: 824835; ManagementIP: 172.24.128.1

inventory.log-[2017-09-11 16:01:30,288] [XDE ThreadPool 10] [inventory] [ERROR] - Exception Stack: com.cisco.nm.expression.function.FunctionException: java.lang.reflect.InvocationTargetException

inventory.log:[2017-09-11 16:01:30,288] [ICE Service - Network 1] [inventory] [ERROR] - Feature = feature_config_interface and Procedure = ethernetextra.interfaceCLI with the following error and continuing with other features: com.cisco.nm.expression.function.FunctionException: java.lang.reflect.InvocationTargetException deviceId: 824835; CredentialId: 824835; ManagementIP: 172.24.128.1

inventory.log-[2017-09-11 16:01:30,289] [ICE Service - Network 1] [inventory] [ERROR] - Exception Stack: com.cisco.nm.expression.function.FunctionException: java.lang.reflect.InvocationTargetException

inventory.log-[2017-09-11 16:01:37,769] [XDE ThreadPool 17] [AssociationBuilderUtil] [ERROR] - Cannot locate the ICE metadata file: /ice-metadata/ifm_networkservices_technologies_trustsec/ifm_networkservices_technologies_trustsec.xml.

noAppender.log-Caused by: com.cisco.nm.pal.client.PALException: <palError><deviceId>67070</deviceId><code>HANDLER_ERROR</code><itemLocation>[com.cisco.xmp.config.xde.system-bootstrap]mib2-getSysOID.xpa/getSysOID/getSysOID.par@Rule0/HandlerChain/Handler2</itemLocation><message>Get Scalar Operation[Op ID: 5271271] Received no response from 172.24.146.9/161</message><handlerCode>ERROR_NO_RESPONSE</handlerCode></palError>

noAppender.log-2017-09-08 12:37:56,392 [pooledTaskScheduler-26] ERROR TaskRunnable - Could not invoke method public final com.cisco.xmp.inventory.ice.IceStatus com.sun.proxy.$Proxy85.collectDeviceInventoryInCurrentThread(java.lang.String,java.util.List) throws com.cisco.xmp.inventory.ice.InventoryException,com.cisco.xmp.inventory.ice.PostCollectionHookException on target com.cisco.xmp.inventory.ice.InventoryCollectorEngineServiceImpl$$EnhancerByCGLIB$$cd851df1@5b5d68a with parameters [824835, null]. Exception java.lang.reflect.InvocationTargetException

noAppender.log-2017-09-08 12:37:56,392 [pooledTaskScheduler-26] ERROR TaskUtils$LoggingErrorHandler - Unexpected error occurred in scheduled task.

Re: Devices in UNKNOWN state

Diana Karolina Rojas — Fri, 10 Aug 2018 17:40:31 GMT

Hello!

Can you please share the troubleshooting for this? I am having the same issue.

Best Regards,