3750x - snmpwalk shows faulty power supply which isn't

[CSCO10382367]

Hy there,


we have some stacked catalyst 3750x and our monitoring system (PRTG) shows some of the power supplies with a faulty

status. When checking the power supplies via cmd line and physically, they are without any issues. Does anybody know

what could be the issue. I do not think that the problem is with the monitoring system, because we do have the same

setup with other (exactly the same) devices and there are no issues at all.

bug search
https://bst.cloudapps.cisco.com/bugsearch/bug/CSCtx16194

BUT: According to the known fixed releases the problem should have been solved with the IOS we have implemented.

Below some details

Does anybody haven an idea what could be our problem?

Regards
Thomas


----------------------------------------------------------------------------------------------------------------

XXXXsw015#sh run | s snmp
snmp-server user public public v1
snmp-server user public public v2c
snmp-server view iso iso included
snmp-server community XXXX RO ACL_MONITORING
snmp-server community XXXX RO ACL_IPAM
snmp-server community XXXX RW ACL_NIMS
snmp-server trap-source Vlan599
snmp-server location DE-XXXX
snmp-server contact xxx@xxx.com
snmp-server chassis-id xxx
snmp-server enable traps snmp authentication linkdown linkup coldstart warmstart
snmp-server enable traps tty
snmp-server enable traps license
snmp-server enable traps config
snmp-server enable traps entity
snmp-server enable traps bridge newroot topologychange
snmp-server enable traps stpx inconsistency root-inconsistency loop-inconsistency
snmp-server enable traps syslog
snmp-server enable traps vlancreate
snmp-server enable traps vlandelete
snmp-server enable traps flash insertion removal
snmp-server enable traps envmon fan shutdown supply temperature status
snmp-server enable traps vlan-membership

----------------------------------------------------------------------------------------------------------------

XXXXsw015#sh vers | i IOS
Cisco IOS Software, C3750E Software (C3750E-UNIVERSALK9-M), Version 15.0(1)SE3, RELEASE SOFTWARE (fc1)
XXXXsw015#
XXXXsw015#
XXXXsw015#dir flash:
Directory of flash:/

    2  -rwx        3833   Nov 1 2016 13:07:12 +00:00  private-config.text
    3  -rwx        4661   Mar 1 1993 00:02:02 +00:00  config.text.backup
    4  -rwx        1918   Mar 1 1993 00:02:02 +00:00  private-config.text.backup
    5  -rwx        3524  Jun 24 2016 12:27:04 +01:00  vlan.dat
    6  -rwx        3096   Nov 1 2016 13:07:12 +00:00  multiple-fs
    7  drwx         512   Mar 1 1993 00:11:08 +00:00  c3750e-universalk9-mz.150-1.SE3
  471  -rwx       47025   Nov 1 2016 13:07:12 +00:00  config.text

57671680 bytes total (34399744 bytes free)
XXXXsw015#

----------------------------------------------------------------------------------------------------------------

XXXXsw015#sh inv
NAME: "1", DESCR: "WS-C3750X-48P"
PID: WS-C3750X-48PF-L  , VID: V03  , SN: FDO1628P0AN

NAME: "Switch 1 - Power Supply 0", DESCR: "FRU Power Supply"
PID: C3KX-PWR-1100WAC  , VID: V02L , SN: LIT16221523

NAME: "Switch 1 - Power Supply 1", DESCR: "FRU Power Supply"
PID: C3KX-PWR-1100WAC  , VID: V02D , SN: DTN1923R0JE

NAME: "Switch 1 - FRULink Slot 1 - FRULink Module", DESCR: "FRULink 10G ST Module"
PID: C3KX-NM-10G       , VID: V01  , SN: FDO16270N5Q

NAME: "GigabitEthernet1/1/1", DESCR: "1000BaseLX SFP"
PID: Unspecified       , VID:      , SN: FNS162724RV

NAME: "2", DESCR: "WS-C3750X-48P"
PID: WS-C3750X-48PF-L  , VID: V03  , SN: FDO1628R0LJ

NAME: "Switch 2 - Power Supply 0", DESCR: "FRU Power Supply"
PID: C3KX-PWR-1100WAC  , VID: V02L , SN: LIT16221535

NAME: "Switch 2 - Power Supply 1", DESCR: "FRU Power Supply"
PID: C3KX-PWR-1100WAC  , VID: V02D , SN: DTN1923R0NQ

NAME: "Switch 2 - FRULink Slot 1 - FRULink Module", DESCR: "FRULink 10G ST Module"
PID: C3KX-NM-10G       , VID: V01  , SN: FDO16330KLE

NAME: "GigabitEthernet2/1/1", DESCR: "1000BaseLX SFP"
PID: Unspecified       , VID:      , SN: FNS1627249U


XXXXsw015#

----------------------------------------------------------------------------------------------------------------

XXXXsw015#sh env all
FAN 1 is OK
FAN 2 is OK
FAN PS-1 is OK
FAN PS-2 is OK
SYSTEM TEMPERATURE is OK
System Temperature Value: 25 Degree Celsius
System Temperature State: GREEN
Yellow Threshold : 46 Degree Celsius
Red Threshold    : 60 Degree Celsius
 SW  PID                 Serial#     Status           Sys Pwr  PoE Pwr  Watts
---  ------------------  ----------  ---------------  -------  -------  -----
1A  C3KX-PWR-1100WAC    LIT16221523 OK              Good     Good     1100/0
1B  C3KX-PWR-1100WAC    DTN1923R0JE OK              Good     Good     1100/0
2A  C3KX-PWR-1100WAC    LIT16221535 OK              Good     Good     1100/0
2B  C3KX-PWR-1100WAC    DTN1923R0NQ OK              Good     Good     1100/0

SW  Status          RPS Name          RPS Serial#  RPS Port#
--  -------------   ----------------  -----------  ---------
1   Not Present     <>
2   Not Present     <>

XXXXsw015#

----------------------------------------------------------------------------------------------------------------

----------------------- New Test -----------------------
Paessler SNMP Tester 5.2.3 Computername: XXXX Interface: (10.10.10.123)
11/1/2016 2:50:39 PM (1 ms) : Device: XXXXsw015
11/1/2016 2:50:39 PM (2 ms) : SNMP V2c
11/1/2016 2:50:39 PM (3 ms) : Walk 1.3.6.1.4.1.9.9.13.1.5.1.2
11/1/2016 2:50:39 PM (24 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.2.1058 = "Sw1, PS2 Normal, RPS NotExist" [ASN_OCTET_STR]
11/1/2016 2:50:39 PM (39 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.2.1069 = "Sw1, PS1 Faulty, RPS NotExist" [ASN_OCTET_STR]
11/1/2016 2:50:39 PM (68 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.2.2059 = "Sw2, PS2 Normal, RPS NotExist" [ASN_OCTET_STR]
11/1/2016 2:50:39 PM (168 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.2.2069 = "Sw2, PS1 Normal, RPS NotExist" [ASN_OCTET_STR]

----------------------------------------------------------------------------------------------------------------

[marce1000]

 - Despite  the referenced bug; it probably, still remains an IOS bug; try the most  recommended release for your platform (gold-starred).

M.

[Palani Mohan]

1.3.6.1.4.1.9.9.13.1.5.1.2 maps to ciscoEnvMonSupplyStatusDescr whereas 1.3.6.1.4.1.9.9.13.1.5.1.3 maps to the actual/functioning state. Looking at the possible values for the "state", what you posted "Faulty" is not one of the defined states for this MIB object. Also, the "Faulty" wording appears in conjunction with the Descr. This is also confusing a little bit.

Would it be possible to do a simple SNMPwalk of .1.3.6.1.4.1.9.9.13.1.5? This is the OID for ciscoEnvMonSupplyStatusTable. Its output would tell us what is seen on the device/switch side and if the SNMP data is consistent with CLI data.

Kind regards .... Palani

[CSCO10382367]

This is all I get (see below)

We did a reload on one of the "faulty" devices and after that the "Faulty" doesn't show anymore.

----------------------- New Test -----------------------
Paessler SNMP Tester 5.2.3 Computername: XXXX Interface: (10.56.65.131)
11/8/2016 10:39:00 AM (2 ms) : Device: xxxxsw015
11/8/2016 10:39:00 AM (3 ms) : SNMP V2c
11/8/2016 10:39:00 AM (3 ms) : Walk .1.3.6.1.4.1.9.9.13.1.5

[Palani Mohan]

Hi there

The response seem to indicate that you see this in more than one switch? Is this assessment accurate?

If yes, that is interesting and makes it a sw problem. Question is, is this a switch side sw issue OR the application side sw issue?

I looked at the documentation for Paessler SNMP Tester. It is here

In page 24 (Section 4.2), it describes howto do a walk. Is this what you did? In your output, I see a . (period) at the beginning of the string. The app I use is from UC Davis net-SNMP project. This app needs a period in front. Based on Paessler SNMP Tester documentation, the leading . should not be there? Would you please consider trying the walk again, this time without the leading period and let me know what you observe? Please do this against the switch that is still reporting faulty.

Thank you ... Palani

[CSCO10382367]

Hi Palani,

sorry that was my mistake. I did put a period up front, which was wrong. Enclosed two snmpwalks (done via Paessler SNMP Tester like you assumed). The first device is the one with the "Faulty" information and the second one is the one that we reloaded, which also had a "Faulty" information in it before the reload.

Hopefully this info is sufficient for you. By the way, we are planning to reload the second device as well.

----------------------- New Test -----------------------
Paessler SNMP Tester 5.2.3 Computername: XXXX Interface: (10.56.65.131)
11/9/2016 11:01:09 AM (1 ms) : Device: xxxsw015
11/9/2016 11:01:09 AM (2 ms) : SNMP V2c
11/9/2016 11:01:09 AM (2 ms) : Walk 1.3.6.1.4.1.9.9.13.1.5
11/9/2016 11:01:09 AM (48 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.2.1058 = "Sw1, PS2 Normal, RPS NotExist" [ASN_OCTET_STR]
11/9/2016 11:01:09 AM (77 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.2.1069 = "Sw1, PS1 Faulty, RPS NotExist" [ASN_OCTET_STR]
11/9/2016 11:01:09 AM (97 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.2.2059 = "Sw2, PS2 Normal, RPS NotExist" [ASN_OCTET_STR]
11/9/2016 11:01:09 AM (124 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.2.2069 = "Sw2, PS1 Normal, RPS NotExist" [ASN_OCTET_STR]
11/9/2016 11:01:09 AM (147 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.3.1058 = "1" [ASN_INTEGER]
11/9/2016 11:01:09 AM (175 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.3.1069 = "6" [ASN_INTEGER]
11/9/2016 11:01:09 AM (212 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.3.2059 = "1" [ASN_INTEGER]
11/9/2016 11:01:09 AM (253 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.3.2069 = "1" [ASN_INTEGER]
11/9/2016 11:01:09 AM (289 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.4.1058 = "2" [ASN_INTEGER]
11/9/2016 11:01:09 AM (311 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.4.1069 = "2" [ASN_INTEGER]
11/9/2016 11:01:09 AM (338 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.4.2059 = "2" [ASN_INTEGER]
11/9/2016 11:01:09 AM (390 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.4.2069 = "2" [ASN_INTEGER]

----------------------- New Test -----------------------
Paessler SNMP Tester 5.2.3 Computername: XXXX Interface: (10.56.65.131)
11/9/2016 11:01:16 AM (62 ms) : Device: xxxsw019
11/9/2016 11:01:16 AM (99 ms) : SNMP V2c
11/9/2016 11:01:16 AM (128 ms) : Walk 1.3.6.1.4.1.9.9.13.1.5
11/9/2016 11:01:16 AM (216 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.2.1058 = "Sw1, PS1 Normal, RPS NotExist" [ASN_OCTET_STR]
11/9/2016 11:01:16 AM (356 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.2.1059 = "Sw1, PS2 Normal, RPS NotExist" [ASN_OCTET_STR]
11/9/2016 11:01:16 AM (450 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.2.2059 = "Sw2, PS1 Normal, RPS NotExist" [ASN_OCTET_STR]
11/9/2016 11:01:16 AM (620 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.2.2060 = "Sw2, PS2 Normal, RPS NotExist" [ASN_OCTET_STR]
11/9/2016 11:01:16 AM (711 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.3.1058 = "1" [ASN_INTEGER]
11/9/2016 11:01:16 AM (748 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.3.1059 = "1" [ASN_INTEGER]
11/9/2016 11:01:16 AM (793 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.3.2059 = "1" [ASN_INTEGER]
11/9/2016 11:01:16 AM (851 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.3.2060 = "1" [ASN_INTEGER]
11/9/2016 11:01:16 AM (875 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.4.1058 = "2" [ASN_INTEGER]
11/9/2016 11:01:16 AM (930 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.4.1059 = "2" [ASN_INTEGER]
11/9/2016 11:01:16 AM (949 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.4.2059 = "2" [ASN_INTEGER]
11/9/2016 11:01:16 AM (967 ms) : 1.3.6.1.4.1.9.9.13.1.5.1.4.2060 = "2" [ASN_INTEGER]

[Palani Mohan]

Ok, thanks. The problem you see is indeed the manifestation of the known bug you cited originally.

The problem is that the "Known Fixed Releases" contains incorrect info. The fix is present in 12.2(55)SE8 and later in this train. Likewise, the fix is present in 15.0(2)SE3 and later. Fix is NOT present in15.0(1)SE3. Given this please consider upgrading the IOS to 15.0(2)SE10a

I am working on fixing the contents of "Known Fixed Releases". Regret the inconvenience caused. Will update on this thread once the incorrect info is corrected.