sg300-28p HAL_config_poe-A-POEFailDetected

[breathevalue]

I have a small network with a number of sg300-28p's (8).

Two of them have 'failed' after recently upgrading to the 1.3.0.62 firmware release.  One failed immediately after upgrading (the below symptoms started on the first boot after upgrade) -- another failed in the same fashion a few weeks after upgrading.

The symptom they are reporting is that they log messages like this at least once per second to the console/rsyslog:

Oct  2 12:35:03 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x11c6 dev_num 1 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:04 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x11c0 dev_num 1 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:05 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x12b8 dev_num 1 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:05 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x12ca dev_num 1 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:06 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x11be dev_num 1 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:07 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x11b2 dev_num 1 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:08 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x11ca dev_num 1 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:09 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x12bc dev_num 1 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:10 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x11b6 dev_num 1 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:11 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x11ce dev_num 1 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:13 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x11c0 dev_num 1 val 0x0) was detected: POE is Disabled untill Reset, aggregated (1) 

Oct  2 12:35:13 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x318 dev_num 0 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:13 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(HALP_config_poe_690xx_check_master_reset - Master HW RESET . val 0x0 dev_num 0!!) was detected: POE is Disabled untill Reset 

Oct  2 12:35:14 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x12ea dev_num 0 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:14 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_write_mem failed. addr 0x139c dev_num 0 val 0x1) was detected: POE is Disabled untill Reset 

Oct  2 12:35:15 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x139c dev_num 0 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:16 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x12e6 dev_num 0 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:17 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x12e8 dev_num 0 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:18 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x12c0 dev_num 1 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:19 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x12c8 dev_num 1 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:20 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x11be dev_num 1 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:21 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x11ba dev_num 1 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:22 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x11d2 dev_num 1 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:23 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x12c4 dev_num 1 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:24 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x11be dev_num 1 val 0x0) was detected: POE is Disabled untill Reset, aggregated (1) 

Oct  2 12:35:24 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x11d6 dev_num 1 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:25 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x11c0 dev_num 1 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:27 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x12c8 dev_num 1 val 0x0) was detected: POE is Disabled untill Reset, aggregated (1) 

Oct  2 12:35:27 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(i2c_read_mem failed addr 0x318 dev_num 0 val 0x0) was detected: POE is Disabled untill Reset 

Oct  2 12:35:27 rr-cisco-sg300-mainlab-2 %HAL_config_poe-A-POEFailDetected: POE ERROR(HALP_config_poe_690xx_check_master_reset - Master HW RESET . val 0x0 dev_num 0!!) was detected: POE is Disabled untill Reset

The units take a long time to boot correctly -- if looking at the bootup process while logged intot the serial console, the above messages are shown shortly after the firmware image is unpacked -- after a lengthy amount of time the units seem to come up however the messages never stop ...

I've always had most of the poe ports explicitly disabled with `power inline never` as part of the switchport configuration.  I've tested one of the switches with nothing plugged in except power cable and serial console.  What's going on here?  What can I do to fix this?

Many thanks,

[Tom Watts]

Hi Nathaniel, default reset the switch, if it doesn't clear, I'd call the SBSC and probably request RMA.

-Tom
Please mark answered for helpful posts

[Brandon Svec]

Hi Nathaniel, did you ever resolve this?  I am seeing the same erors on one witch in a 3 switch stack right now.

thanks,

-- please remember to rate helpful posts --

[Nagaraja Thanthry]

Hello Brandon,

Can you please contact Cisco Support and open a Service Request? One of our Engineers will be able to review the error message and find the root cause.

Thanks,

Nagaraja

[Ibrahim Hamid]

Hi Brandon, was your problem on SG300 solved, if yes how, I am facing the same issue. thank you

[Brandon Svec]

For the benefit of anyone who comes across this I saw this message on an SF300-48P today.  I had staged the switch in my lab, updated to 1.3.7.18 and new bootloader.  I put a static IP on it and plugged a couple devices into it.  The switch ran for a few days before the status light flashed amber and console showed this same error.  I tried reloading, rebooting and the switch would not come back.  Unresponsive to factory reset button too.  Console just starting showing garbage and some port lights lit solid.  I am getting it replaced now.