09-21-2020 06:32 AM
this bug is not resolved in 16.12.4 for the 3650 48p sw. i just encountered it today.
the work around was to move the PD to another switch in the stack. which is by no means a fix
further testing revealed that the poe power is not available on the remaining free ports on my second switch in the stack.
i would like more feedback on how to resolve this issue.
09-21-2020 05:02 PM
Tell us what you are seeing: Are you actually seeing the above error message(s) or something else?
Have you also tried cold rebooting (completely remove the power cables)?
11-17-2020 09:33 AM
Seeing the same issue on one module in a 3850 stack on 16.12.4 , the other 4 modules in the stack aren't affected. The option I'm looking at is moving the PDs to another switch in the stack but agree that's not a proper solution
TDR tests on the drops came back clean, tried various config changes at the individual port level:
Bouncing ports
power inline never / power inline auto,
Setting Max power draw to 15.4 W
Setting POE to 4 wires
As it is a state machine time out message I think that would likely suggest a bug, but I'm not seeing a fix anywhere
Any ideas?
11-23-2020 03:04 PM
I am seeing same problem on 16.12.3a code which also listed as "fixed" under bug description. Any more suggestions?
11-23-2020 04:15 PM
@JOeKeenan2659 wrote:
Seeing the same issue on one module in a 3850 stack on 16.12.4 , the other 4 modules in the stack aren't affected.
Wait ... I think I know what this is ...
So you have some PDs, right? And they do not "talk" CDP or LLDP, am I correct?
To check if they talk CDP or LLDP, use the command "sh power inline <PORT> detail".
11-23-2020 05:30 PM
CDP is disabled on this port. LLDP is enabled on switch and on this interface. Device is Axis IP camera.
Switch1# show power inline GigabitEthernet 2/0/9 detail
Interface: Gi2/0/9
Inline Power Mode: auto
Operational status: on
Device Detected: yes
Device Type: Ieee PD
IEEE Class: 2
Discovery mechanism used/configured: Ieee and Cisco
Police: off
Power Allocated
Admin Value: 30.0
Power drawn from the source: 7.0
Power available to the device: 7.0
Actual consumption
Measured at the port: 0.0
Maximum Power drawn by the device since powered on: 0.0
Absent Counter: 0
Over Current Counter: 1
Short Current Counter: 0
Invalid Signature Counter: 0
Power Denied Counter: 0
Power Negotiation Used: None
LLDP Power Negotiation --Sent to PD-- --Rcvd from PD--
Power Type: - -
Power Source: - -
Power Priority: - -
Requested Power(W): - -
Allocated Power(W): - -
Four-Pair PoE Supported: No
Spare Pair Power Enabled: No
Four-Pair PD Architecture: N/A
------------------
Power drawn is flipping from 7 to zero every few minutes.
11-23-2020 05:54 PM
Basically port is detecting POE device every 5 minutes. This is the same device which is constantly connected to port and powered via POE, so device cannot just be in slow reboot loop because in the middle for 4 minutes port is in disconnect state and device has no power. So, this is some sort of timer on interface side which is probing device and provide it some power every 5 minutes.
I know if I reboot switch device will be powered, but I am waiting to see if anyone can suggest something other than reboot.
11-23-2020 07:10 PM - edited 11-23-2020 07:17 PM
@Art Astafiev wrote:
CDP is disabled on this port. LLDP is enabled on switch and on this interface. Device is Axis IP camera.
We have this same issue now and ours is probably very bad because our stacks are running Dot1X and the flapping ports are causing the stack members to crash every 36 hours.
Based on the output to the command "sh power inline <PORT> detail", the cameras DO NOT TALK LLDP.
This is a bug with the entire 16.12.X train (until 16.12.4). It is not known if 16.12.5, slated for 1Q2021 release, will fix this issue.
TAC is talking about "downgrading" to 16.3.11 and "see if this helps" (aka TAC is not even sure if this will fix the issue or not).
Our workaround, so far is to apply SMU to fix CSCvv28324 (cat3k_caa-universalk9.16.12.04.CSCvv28324.SPA.smu.bin).
WARNING: This SMU will reboot the stack.
This bug gets triggered when a PD does not talk CDP nor LLDP gets connected to the port. (Normally, if the same PD gets plugged into a "classic" IOS switch, it will just work -- But IOS-XE is "special".)
The bug emerges after 3 weeks uptime. If the stack members do not crash, the constant flapping of PoE will cause the ports to "throw hands in the air and give up" after >3 weeks -- This means the port will stop to "work", PoE or not. If a non-PoE is connected to the port, NO data traffic will pass. The port will report up/up but the traffic will be reported as 0 input and 0 output traffic.
01-08-2021 03:55 PM - edited 01-08-2021 03:57 PM
Any one upgrade any version and get fixed?
01-08-2021 04:56 PM
@WayneTW wrote:
Any one upgrade any version and get fixed?
What issue(s) are you seeing/getting, @WayneTW?
TAC is saying 16.12.5, scheduled for release on 21 January 2021, "will" contain thie fix for this bug and CSCvv28324. I am, however, skeptical about this claim because this bug and CSCvv28324 has been present since 16.12.1 and previous releases have failed to fix this bug.
01-08-2021 05:20 PM
Hi Leo,
I got the same the issue, here's the message, several ports cannot boot up successfully today with IP Phones.
Here's is one of the port message, this port was working with IP Phone and after I re-plugged the cable, no more function.
*Jan 8 18:18:28.659: %ILPOWER-5-IEEE_DISCONNECT: Interface Gi2/0/28: PD removed *Jan 8 18:18:28.662: %ILPOWER-3-CONTROLLER_PORT_ERR: Controller port error, Interface Gi2/0/28: Power given, but State Machine Power Good wait timer timed out *Jan 8 18:18:38.672: %ILPOWER-5-IEEE_DISCONNECT: Interface Gi2/0/28: PD removed *Jan 8 18:18:49.055: %ILPOWER-5-DETECT: Interface Gi2/0/28: Power Device detected: Cisco PD *Jan 8 18:19:27.591: %LINK-3-UPDOWN: Interface GigabitEthernet2/0/28, changed state to up *Jan 8 18:19:28.591: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet2/0/28, changed state to up *Jan 8 18:19:39.943: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet2/0/28, changed state to down *Jan 8 18:19:41.947: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet2/0/28, changed state to up
ServerRoom-1#show power inline gigabitEthernet 2/0/28 detail Interface: Gi2/0/28 Inline Power Mode: auto Operational status: faulty Device Detected: no Device Type: n/a IEEE Class: n/a Discovery mechanism used/configured: Ieee and Cisco Police: off Power Allocated Admin Value: 30.0 Power drawn from the source: 0.0 Power available to the device: 0.0 Actual consumption Measured at the port: 0.0 Maximum Power drawn by the device since powered on: 0.0 Absent Counter: 0 Over Current Counter: 0 Short Current Counter: 0 Invalid Signature Counter: 0 Power Denied Counter: 0 Power Negotiation Used: None LLDP Power Negotiation --Sent to PD-- --Rcvd from PD-- Power Type: - - Power Source: - - Power Priority: - - Requested Power(W): - - Allocated Power(W): - - Four-Pair PoE Supported: No Spare Pair Power Enabled: No Four-Pair PD Architecture: N/A
Wayne
01-08-2021 06:04 PM - edited 01-08-2021 06:04 PM
Is the stack running 16.12.X?
01-11-2021 09:34 AM
Hi Leo,
Yes! the stacks version are Version 16.12.03a
01-11-2021 04:17 PM
@WayneTW wrote:
Yes! the stacks version are Version 16.12.03a
Wait for 16.12.5 to be released shortly (21 January 2021) and upgrade the firmware.
Hopefully, if the bug is indeed fixed in 16.12.5, then you will NOT see any problems after >8 weeks of uptime.
03-28-2022 06:11 AM
We have 43 stacks of Catalyst 3850's on code 16.12.5b and another 51 standalone 3850 on this code. We had been running on this code for around 10 months now. I would say that it is relatively stable and has maximum amount of security fixes. So, I would recommend everyone to move from 16.9.x code due to security vulnerabilities. I don't remember that PoE issue was reported since we did upgrade.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide