Cisco 2960x stack (3x) - Inline Power internal error on only one member

darijohorvat · ‎06-30-2015

Hi

We have a strange problem, there is a stack formed of three 2960x switches (same model and IOS), and third member reports strange messages in log regarding inline power:

14181681: 14008908: Jun 30 08:39:07.998 UTC: %ILPOWER-3-ILPOWER_INTERNAL_IF_ERROR: Inline Power internal error, interface Gi3/0/4: failed to get subblock from hwidb when power alloc change (Hostname-2)
-Traceback= 42D924z 2663518z 20E08F0z 21D45B4z 270CED8z 2707118z (Hostname-2)
14181682: 14008909: Jun 30 08:39:07.998 UTC: %ILPOWER-3-ILPOWER_INTERNAL_IF_ERROR: Inline Power internal error, interface Gi3/0/4: failed to get subblock from hwidb when pd change (Hostname-2)
-Traceback= 42D924z 2664FD8z 20E095Cz 21D45B4z 270CED8z 2707118z (Hostname-2)

Some outputs:

Switch Ports Model SW Version SW Image
------ ----- ----- ---------- ----------
* 1 52 WS-C2960XR-48FPD-I 15.0(2)EX3 C2960X-UNIVERSALK9-M
2 52 WS-C2960XR-48FPD-I 15.0(2)EX3 C2960X-UNIVERSALK9-M
3 52 WS-C2960XR-48FPD-I 15.0(2)EX3 C2960X-UNIVERSALK9-M

Hostname#sh platform stack manager all
Switch/Stack Mac Address : 189c.5d27.a900
H/W Current
Switch# Role Mac Address Priority Version State
----------------------------------------------------------
*1 Master 189c.5d27.a900 1 4 Ready
2 Member 189c.5d27.b080 1 4 Ready
3 Member 189c.5d27.d700 1 4 Ready

Stack Port Status Neighbors
Switch# Port 1 Port 2 Port 1 Port 2
--------------------------------------------------------
1 Ok Ok 3 2
2 Ok Ok 3 1
3 Ok Ok 2 1

Stack Discovery Protocol View
==============================================================

Switch Active Role Current Sequence Dirty
Number State Number Bit
--------------------------------------------------------------------
1 TRUE Master Ready 125 FALSE
2 TRUE Member Ready 110 FALSE
3 TRUE Member Ready 150 FALSE

Stack State Machine View
==============================================================

Switch Master/ Mac Address Version Current
Number Member (maj.min) State
-----------------------------------------------------------
1 Master 189c.5d27.a900 4.55 Ready
2 Member 189c.5d27.b080 4.55 Ready
3 Member 189c.5d27.d700 4.55 Ready

Last Conflict Parameters

Switch Master/ Cfgd Default Image H/W # of Mac Address
Number Member Prio Config Type Prio Members
-----------------------------------------------------------------------
1 Master 1 No 4 5 1 189c.5d27.a900
2 Member 1 No 4 5 0 189c.5d27.b080
3 Member 1 Yes 4 5 0 189c.5d27.d700

It looks like a bug to me (was unable to find it in bug toolkit although), and strange is that ports affected are on Switch number3 but log also says Hostname-2 - this should a label for second member??

And if someone could explain this, says Default Config Yes, but I wouldn't say so:

3 Member 1 Yes 4 5 0 189c.5d27.d700

Any suggestion is welcome!

Regards

Leo Laohoo · ‎06-30-2015

Post the complete output to the following commands:

1. sh power in Gi 3/0/4; and

2. sh interface Gi 3/0/4

Unfortunately, the IOS you're using prevents 2960X from supporting 10/100 Mbps TDR test so there's little chance of verifying if the cable is faulty or not.

darijohorvat · ‎06-30-2015

Hi Leo

Thank you for response. Here you go (it hapens on every port and constantly, so this is most recent one):

14190226: 14017340: Jun 30 13:39:58.652 UTC: %ILPOWER-3-ILPOWER_INTERNAL_IF_ERROR: Inline Power internal error, interface Gi3/0/17: failed to get subblock from hwidb when power alloc change (Hostname-2)
-Traceback= 42D924z 2663518z 20E08F0z 21D45B4z 270CED8z 2707118z (Hostname-2)
14190227: 14017341: Jun 30 13:39:58.652 UTC: %ILPOWER-3-ILPOWER_INTERNAL_IF_ERROR: Inline Power internal error, interface Gi3/0/17: failed to get subblock from hwidb when pd change (Hostname-2)

Hostname-01#sh power inline gi3/0/17
Interface Admin Oper Power Device Class Max
(Watts)
--------- ------ ---------- ------- ------------------- ----- ----
Gi3/0/17 auto on 12.8 IP Phone 9971 3 30.0

Interface AdminPowerMax AdminConsumption
(Watts) (Watts)
---------- --------------- --------------------

Gi3/0/17 30.0 30.0

Hostname#sh int gi3/0/17
GigabitEthernet3/0/17 is up, line protocol is up (connected)
Hardware is Gigabit Ethernet, address is 189c.5d27.d711 (bia 189c.5d27.d711)
Description: ### Desktop&Phone ###
MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:14, output 00:00:01, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 15009
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 1000 bits/sec, 2 packets/sec
116487675 packets input, 38632646864 bytes, 0 no buffer
Received 1636589 broadcasts (1326285 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 1326285 multicast, 0 pause input
0 input packets with dribble condition detected
301938236 packets output, 177280856813 bytes, 0 underruns
0 output errors, 0 collisions, 2 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out

Best regards

Leo Laohoo · ‎07-01-2015

I'm suspecting this is an IOS bug but there is no good IOS for the 2960X so I won't make any recommendations where you should go.

I'd say create a TAC Case instead.

darijohorvat · ‎07-01-2015

Leo, thank you for your time on this topic.

Best Regards

rosaho · ‎07-07-2015

This discussion has been reposted to the LAN, Switching and Routing community.