I just received two responds on my TAC case. I will list both one next to another
-------------------------------------------
Respond 1
---------------------------------------
I’ve been researching about this issue, and it was already identified that on 16.12.x trail there are some issues with PoE, this issue has been recreated and fixed. The fix will come with the 16.12.5 code release which is due out tentative on Jan22, 2021.
Please check this out:
https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvv50628
Cat3850 : PoE doesn't work - Power given, but State Machine Power Good wait timer timed out
CSCvv50628
Description
Symptom:
Switches and/or stack running versions Gibraltar 16.12.3 and Gibraltar 16.12.3a stop providing PoE on certain ports. This issue is seen after the following log is seen:
%ILPOWER-3-CONTROLLER_PORT_ERR: Controller port error, Interface Gix/0/y: Power given, but State Machine Power Good wait timer timed out.
The impacted ports could experience the following scenarios:
a) The PoE device will power up for a few seconds (5-45) and then it dies, there have been cases where the device powers up for up to 5 minutes.
c) No power at all is seen.
Disconnecting/reconnecting the cable or a shut/no shut the impacted port does not resolve the issue.
Conditions:
It looks like it is a matter of time (weeks) when the issue is seen.
Workaround:
So far the only workaround found is to reload the impacted stack/switch
Further Problem Description:
Once one single port reaches this status all the ports are likely to experience the same issue, this means: if a single port has this issue in stack, if any other port or ports is/are disconnected/re-connected these other ports will experience the same issue
Alternatively, a software patch (SMU) for 16.12.3a is tentatively planned for Oct 23, 2020
The SMU fix for this PoE issue is cat3k_caa-universalk9.16.12.03a.CSCvv28324.SPA.smu.bin. Please note that this patch is for the 16.12.3a release only:
https://software.cisco.com/download/home/284455380/type/286308587/release/Gibraltar-16.12.3a
More info regarding SMU upgrade procedures in general can be found here:
https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst3850/software/release/16-12/configuration_guide/sys_mgmt/b_1612_sys_mgmt_3850_cg/software_maintenance_upgrade.html
--------------
Respond 2
-----------------
I was looking at the TAC case, per the TAC engineer the fix will be on version 16.12.5 tentative release Jan 22, 2021. Also, there is a software maintenance upgrade (SMU) that was released to fix this issue, below is the link from where we can get the SMU to fix the PoE related issue. In the description of this SMU its mentioned that it is for SNMP, but it has been working fine for PoE issue as well. https://software.cisco.com/download/home/284455429/type/286308587/release/Gibraltar-16.12.4
File name: “cat3k_caa-universalk9.16.12.04.CSCvv28324.SPA.smu.bin”
This SMU will solve the issue with the %ILPOWER-3-CONTROLLER_PORT_ERR: Controller port error, Interface Gix/0/y: Power given, but State Machine Power Good wait timer timed out
Please see below the steps to make the install of the SMU:
Copy the SMU to the switch
And then use #install add file flash:<filename> active commit
Example: #install add file flash: cat3k_caa-universalk9.16.12.04.CSCvv28324.SPA.smu.bin active commit
It will ask you to reload the switch
After the reload it will be install
Use show install summary and in the state need to be "C" = Activated & Committed
