Re: Inventory Collection Issue

joemurray · ‎06-22-2007

I am running LMS 2.6 and RME 4.0.5 and am having a Device sensed, but collection failed message on just one of my devices. I have read through all the other listings on this issue and all say to increase the SNMP timeout...I have done this...to accually 120 but get the same result. Has anyone seen this isse after the upping of the SNMP timeout?

Thanks in advance.

Joe Clarke · ‎06-22-2007

This may be any number of known bugs. What device? What version of software is it running? What errors are in the IC_Server.log after reproducing this problem?

joemurray · ‎06-22-2007

I have multiple devices that are exactly the same (3750's) but this is the only one that is having the issue. To me it does point to a device issue...but I thought I would check and see if anyone has seen this issue. See below the out put from the IC_Server.log

[ Fri Jun 22 09:35:58 EDT 2007 ],INFO ,[Thread-17],com.cisco.nm.rmeng.inventory.ics.core.CollectionController,194,Started processing device Name: 172.16.37.2

[ Fri Jun 22 09:36:29 EDT 2007 ],INFO ,[Thread-17],com.cisco.nm.rmeng.inventory.ics.core.CollectionController,239,DP time is 31 seconds for 172.16.37.2

[ Fri Jun 22 09:36:29 EDT 2007 ],ERROR,[Thread-17],com.cisco.nm.rmeng.inventory.ics.core.CollectionController,329,Exception for device 172.16.37.2 is

[ Fri Jun 22 09:36:29 EDT 2007 ],INFO ,[Thread-17],com.cisco.nm.rmeng.inventory.ics.core.CollectionController,377,Device sensed but collection failed for 172.16.37.2

[ Fri Jun 22 10:18:58 EDT 2007 ],INFO ,[Thread-21],com.cisco.nm.rmeng.inventory.ics.core.CollectionController,194,Started processing device Name: 172.16.37.2

[ Fri Jun 22 10:19:30 EDT 2007 ],INFO ,[Thread-21],com.cisco.nm.rmeng.inventory.ics.core.CollectionController,239,DP time is 31 seconds for 172.16.37.2

[ Fri Jun 22 10:19:30 EDT 2007 ],ERROR,[Thread-21],com.cisco.nm.rmeng.inventory.ics.core.CollectionController,329,Exception for device 172.16.37.2 is

[ Fri Jun 22 10:19:30 EDT 2007 ],INFO ,[Thread-21],com.cisco.nm.rmeng.inventory.ics.core.CollectionController,377,Device sensed but collection failed for 172.16.37.2

[ Fri Jun 22 10:42:02 EDT 2007 ],INFO ,[Thread-20],com.cisco.nm.rmeng.inventory.ics.core.CollectionController,194,Started processing device Name: 172.16.37.2

[ Fri Jun 22 10:42:34 EDT 2007 ],INFO ,[Thread-20],com.cisco.nm.rmeng.inventory.ics.core.CollectionController,239,DP time is 31 seconds for 172.16.37.2

[ Fri Jun 22 10:42:34 EDT 2007 ],ERROR,[Thread-20],com.cisco.nm.rmeng.inventory.ics.core.CollectionController,329,Exception for device 172.16.37.2 is

[ Fri Jun 22 10:42:34 EDT 2007 ],INFO ,[Thread-20],com.cisco.nm.rmeng.inventory.ics.core.CollectionController,377,Device sensed but collection failed for 172.16.37.2

[ Fri Jun 22 10:49:33 EDT 2007 ],INFO ,[Thread-17],com.cisco.nm.rmeng.inventory.ics.core.CollectionController,194,Started processing device Name: 172.16.37.2

[ Fri Jun 22 10:50:04 EDT 2007 ],INFO ,[Thread-17],com.cisco.nm.rmeng.inventory.ics.core.CollectionController,239,DP time is 31 seconds for 172.16.37.2

[ Fri Jun 22 10:50:04 EDT 2007 ],ERROR,[Thread-17],com.cisco.nm.rmeng.inventory.ics.core.CollectionController,329,Exception for device 172.16.37.2 is

[ Fri Jun 22 10:50:04 EDT 2007 ],INFO ,[Thread-17],com.cisco.nm.rmeng.inventory.ics.core.CollectionController,377,Device sensed but collection failed for 172.16.37.2

[ Fri Jun 22 12:43:36 EDT 2007 ],INFO ,[Thread-21],com.cisco.nm.rmeng.inventory.ics.core.CollectionController,194,Started processing device Name: 172.16.37.2

[ Fri Jun 22 12:44:08 EDT 2007 ],INFO ,[Thread-21],com.cisco.nm.rmeng.inventory.ics.core.CollectionController,239,DP time is 32 seconds for 172.16.37.2

[ Fri Jun 22 12:44:08 EDT 2007 ],ERROR,[Thread-21],com.cisco.nm.rmeng.inventory.ics.core.CollectionController,329,Exception for device 172.16.37.2 is

[ Fri Jun 22 12:44:08 EDT 2007 ],INFO ,[Thread-21],com.cisco.nm.rmeng.inventory.ics.core.CollectionController,377,Device sensed but collection failed for 172.16.37.2

Joe Clarke · ‎06-22-2007

Please post the entire IC_Server.log as wel as a show ver from this switch? Is this 3750 actually a multi-node stack, or is it just a single standalone switch?

joemurray · ‎06-22-2007

The IC_Server.log is 9meg and also contains info that I can not release...but here is the show ver for the switch. Oh it is a stand alone 3750.

#show ver

Cisco IOS Software, C3750 Software (C3750-ADVIPSERVICESK9-M), Version 12.2(35)SE1, RELEASE SOFTWARE (fc1)

Compiled Tue 19-Dec-06 10:18 by antonino

Image text-base: 0x00003000, data-base: 0x013CF978

ROM: Bootstrap program is C3750 boot loader

BOOTLDR: C3750 Boot Loader (C3750-HBOOT-M) Version 12.2(25r)SEE4, RELEASE SOFTWARE (fc1)

uptime is 16 weeks, 22 hours, 13 minutes

System returned to ROM by power-on

System restarted at 19:53:19 UTC Thu Mar 1 2007

System image file is "flash:/c3750-advipservicesk9-mz.122-35.SE1.bin"

This product contains cryptographic features and is subject to United

States and local country laws governing import, export, transfer and

use. Delivery of Cisco cryptographic products does not imply

third-party authority to import, export, distribute or use encryption.

Importers, exporters, distributors and users are responsible for

compliance with U.S. and local country laws. By using this product you

agree to comply with applicable laws and regulations. If you are unable

to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:

http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to

export@cisco.com.

cisco WS-C3750G-48PS (PowerPC405) processor (revision F0) with 118784K/12280K bytes of memory.

Processor board ID FOC

Last reset from power-on

3 Virtual Ethernet interfaces

104 Gigabit Ethernet interfaces

The password-recovery mechanism is enabled.

512K bytes of flash-simulated non-volatile configuration memory.

Base ethernet MAC Address : 00:

Motherboard assembly number : 73-10216-08

Power supply part number : 341-0108-03

Motherboard serial number : FOC

Power supply serial number : DCA

Model revision number : F0

Motherboard revision number : B0

Model number : WS-C3750G-48PS-E

System serial number : FO

Top Assembly Part Number : 800-27481-01

Top Assembly Revision Number : A0

Version ID : V05

CLEI Code Number : CNM

Hardware Board Revision Number : 0x09

Switch Ports Model SW Version SW Image

------ ----- ----- ---------- ----------

* 1 52 WS-C3750G-48PS 12.2(35)SE1 C3750-ADVIPSERVICESK

Configuration register

Joe Clarke · ‎06-22-2007

There should be Java stack traces in the IC_Server.log that I need to see. I have not heard of an inventory collection failure on a single 3750 running this code, but we do have a few reports of a stack running this code seeing some ENTITY-MIB related NullPointerExceptions during collection.

If that turns out to be the case here, a reload of the switch may workaround the problem. But getting a sniffer trace to TAC would help determine if this a new bug.

joemurray · ‎06-22-2007

See attached

Joe Clarke · ‎06-22-2007

This is exactly the same trace we've seen in other multi-node 3750 stacks. As I said, a reload may workaround this problem, and allow you to fetch the inventory, but it would be great to get a sniffer trace of the inventory collection to see what the problem is with the ENTITY-MIB objects. Having a single switch exhibiting this problem is much easier to troubleshoot.

joemurray · ‎06-22-2007

I would love to give you that info...but like all other things here...I could not release the info out. I pushed it here with this transfer of info.

Joe Clarke · ‎06-22-2007

That's why I suggested you open another TAC service request and provide the sniffer trace there. All of our attempts to reproduce this in the lab have failed, so we suspect there is a state problem that occurs that triggers some ENTITY-MIB corruption.

joemurray · ‎06-22-2007

I will see if my managment will allow the TAC case.

Thanks for all your help!!!