Re: Catalyst 3850 swich stack PoE faulty - Page 2

479130 · ‎02-26-2019

Hello,

I have a stack of 6x 3850 PoE capable switches, recently some ports started to fail to give PoE and/or going up/down where PoE devices are connected to, like Cisco Access Points, Digium desk phones, other PoE devices.

I have:

1) changed cabling

2) perform shut / no shut

3) check PoE module is not overloaded

4) set power inline auto / static

5) connected devices to different switch ports on the same switch

6) connected devices to different switch ports on a different switch

7) reboot switch stack

From time to time some ports seems to fail to negotiate PoE and go to faulty state.

This is failing from switch 1,2 oldest to switch 5,6 newest devices added no more than a year ago.

I am having trouble diagnosing where the issue could be

below configuration:

#show env power all

SW PID Serial# Status Sys Pwr PoE Pwr Watts

-- ------------------ ---------- --------------- ------- ------- -----

1A PWR-C1-350WAC ART2003F757 OK Good Good 350

1B PWR-C1-715WAC DCA1949G04Z OK Good Good 715

2A PWR-C1-350WAC ART2003F78P OK Good Good 350

2B PWR-C1-715WAC DCA1949G052 OK Good Good 715

3A PWR-C1-350WAC ART2003F72S OK Good Good 350

3B PWR-C1-715WAC DCA1949G057 OK Good Good 715

4A PWR-C1-350WAC LIT17461E6H OK Good Good 350

4B PWR-C1-715WAC DCB1910G12V OK Good Good 715

5A PWR-C1-715WAC DCA2149G499 OK Good Good 715

5B PWR-C1-715WAC DCA2149G49K OK Good Good 715

6A PWR-C1-715WAC DCA2149G4DT No Input Power Bad Bad 715

#show switch detail

Switch/Stack Mac Address : 042a.e23f.7700 - Local Mac Address

Mac persistency wait time: Indefinite

H/W Current

Switch# Role Mac Address Priority Version State

------------------------------------------------------------

*1 Active 042a.e23f.7700 15 V07 Ready

2 Standby 042a.e23f.7600 14 V07 Ready

3 Member 042a.e23f.7680 13 V07 Ready

4 Member 8890.8dd8.6300 12 V04 Ready

5 Member 00f8.2c9b.9a00 1 V08 Ready

6 Member 00f8.2c9b.9700 1 V08 Ready

Stack Port Status Neighbors

Switch# Port 1 Port 2 Port 1 Port 2

--------------------------------------------------------

1 OK OK 2 6

2 OK OK 3 1

3 OK OK 4 2

4 OK OK 5 3

5 OK OK 6 4

6 OK OK 1 5

6B PWR-C1-715WAC DCA2149G4D2 OK Good Good 715

#show inventory

NAME: "c38xx Stack", DESCR: "c38xx Stack"

PID: WS-C3850-24P , VID: V07 , SN: FOC2012U0LV

NAME: "Switch 1", DESCR: "WS-C3850-24P-S"

PID: WS-C3850-24P-S , VID: V07 , SN: FOC2012U0LV

NAME: "Switch 1 - Power Supply A", DESCR: "Switch 1 - Power Supply A"

PID: PWR-C1-350WAC , VID: V01 , SN: ART2003F757

NAME: "Switch 1 - Power Supply B", DESCR: "Switch 1 - Power Supply B"

PID: PWR-C1-715WAC , VID: V01 , SN: DCA1949G04Z

NAME: "Switch 1 FRU Uplink Module 1", DESCR: "2x1G 2x10G Uplink Module"

PID: C3850-NM-2-10G , VID: V01 , SN: FOC20010HPK

NAME: "TenGigabitEthernet1/1/4", DESCR: "SFP-10GBase-SR"

PID: SFP-10G-SR , VID: V03 , SN: JUR2004GFHW

NAME: "Switch 2", DESCR: "WS-C3850-24P-S"

PID: WS-C3850-24P-S , VID: V07 , SN: FCW2012C0L6

NAME: "Switch 2 - Power Supply A", DESCR: "Switch 2 - Power Supply A"

PID: PWR-C1-350WAC , VID: V01 , SN: ART2003F78P

NAME: "Switch 2 - Power Supply B", DESCR: "Switch 2 - Power Supply B"

PID: PWR-C1-715WAC , VID: V01 , SN: DCA1949G052

NAME: "Switch 2 FRU Uplink Module 1", DESCR: "2x1G 2x10G Uplink Module"

PID: C3850-NM-2-10G , VID: V01 , SN: FOC20010HMB

NAME: "TenGigabitEthernet2/1/4", DESCR: "SFP-10GBase-SR"

PID: SFP-10G-SR , VID: V03 , SN: JUR2007G1DH

NAME: "Switch 3", DESCR: "WS-C3850-24P-S"

PID: WS-C3850-24P-S , VID: V07 , SN: FOC2012U0LW

NAME: "Switch 3 - Power Supply A", DESCR: "Switch 3 - Power Supply A"

PID: PWR-C1-350WAC , VID: V01 , SN: ART2003F72S

NAME: "Switch 3 - Power Supply B", DESCR: "Switch 3 - Power Supply B"

PID: PWR-C1-715WAC , VID: V01 , SN: DCA1949G057

NAME: "Switch 4", DESCR: "WS-C3850-24T-S"

PID: WS-C3850-24T-S , VID: V04 , SN: FOC1823U10S

NAME: "Switch 4 - Power Supply A", DESCR: "Switch 4 - Power Supply A"

PID: PWR-C1-350WAC , VID: V01 , SN: LIT17461E6H

NAME: "Switch 4 - Power Supply B", DESCR: "Switch 4 - Power Supply B"

PID: PWR-C1-715WAC , VID: V01 , SN: DCB1910G12V

NAME: "Switch 5", DESCR: "WS-C3850-24P-S"

PID: WS-C3850-24P-S , VID: V08 , SN: FCW2204C0VC

NAME: "Switch 5 - Power Supply A", DESCR: "Switch 5 - Power Supply A"

PID: PWR-C1-715WAC , VID: V02 , SN: DCA2149G499

NAME: "Switch 5 - Power Supply B", DESCR: "Switch 5 - Power Supply B"

PID: PWR-C1-715WAC , VID: V02 , SN: DCA2149G49K

NAME: "Switch 6", DESCR: "WS-C3850-24P-S"

PID: WS-C3850-24P-S , VID: V08 , SN: FCW2204C0V2

NAME: "Switch 6 - Power Supply A", DESCR: "Switch 6 - Power Supply A"

PID: PWR-C1-715WAC , VID: V02 , SN: DCA2149G4DT

NAME: "Switch 6 - Power Supply B", DESCR: "Switch 6 - Power Supply B"

PID: PWR-C1-715WAC , VID: V02 , SN: DCA2149G4D2

#show power inline

Module Available Used Remaining

(Watts) (Watts) (Watts)

------ --------- -------- ---------

1 720.0 0.0 720.0

Interface Admin Oper Power Device Class Max

(Watts)

--------- ------ ---------- ------- ------------------- ----- ----

Gi1/0/1 auto off 0.0 n/a n/a 30.0

Gi1/0/2 auto faulty 0.0 n/a n/a 30.0

Gi1/0/3 auto off 0.0 n/a n/a 30.0

Gi1/0/4 off off 0.0 n/a n/a 30.0

Gi1/0/5 auto faulty 0.0 n/a n/a 30.0

Gi1/0/6 auto off 0.0 n/a n/a 30.0

Gi1/0/7 auto faulty 0.0 n/a n/a 30.0

Gi1/0/8 auto off 0.0 n/a n/a 30.0

Gi1/0/9 auto off 0.0 n/a n/a 30.0

Gi1/0/10 auto off 0.0 n/a n/a 30.0

Gi1/0/11 auto off 0.0 n/a n/a 30.0

Gi1/0/12 auto off 0.0 n/a n/a 30.0

Gi1/0/13 auto off 0.0 n/a n/a 30.0

Gi1/0/14 auto off 0.0 n/a n/a 30.0

Gi1/0/15 auto off 0.0 n/a n/a 30.0

Gi1/0/16 auto off 0.0 n/a n/a 30.0

Gi1/0/17 auto off 0.0 n/a n/a 30.0

Gi1/0/18 auto off 0.0 n/a n/a 30.0

Gi1/0/19 auto off 0.0 n/a n/a 30.0

Gi1/0/20 auto off 0.0 n/a n/a 30.0

Gi1/0/21 auto off 0.0 n/a n/a 30.0

Gi1/0/22 auto off 0.0 n/a n/a 30.0

Gi1/0/23 auto off 0.0 n/a n/a 30.0

Gi1/0/24 auto off 0.0 n/a n/a 30.0

Module Available Used Remaining

(Watts) (Watts) (Watts)

------ --------- -------- ---------

2 720.0 79.7 640.3

Interface Admin Oper Power Device Class Max

(Watts)

--------- ------ ---------- ------- ------------------- ----- ----

Gi2/0/1 auto off 0.0 n/a n/a 30.0

Gi2/0/2 auto on 3.9 Ieee PD 2 30.0

Gi2/0/3 auto off 0.0 n/a n/a 30.0

Gi2/0/4 auto off 0.0 n/a n/a 30.0

Gi2/0/5 auto on 3.9 Ieee PD 2 30.0

Gi2/0/6 auto off 0.0 n/a n/a 30.0

Gi2/0/7 auto on 3.9 Ieee PD 2 30.0

Gi2/0/8 auto off 0.0 n/a n/a 30.0

Gi2/0/9 auto off 0.0 n/a n/a 30.0

Gi2/0/10 auto off 0.0 n/a n/a 30.0

Gi2/0/11 auto off 0.0 n/a n/a 30.0

Gi2/0/12 auto on 5.8 Ieee PD 2 30.0

Gi2/0/13 auto off 0.0 n/a n/a 30.0

Interface Admin Oper Power Device Class Max

(Watts)

--------- ------ ---------- ------- ------------------- ----- ----

Gi2/0/14 auto faulty 0.0 n/a n/a 30.0

Gi2/0/15 static faulty 0.0 n/a n/a 30.0

Gi2/0/16 auto off 0.0 n/a n/a 30.0

Gi2/0/17 auto off 0.0 n/a n/a 30.0

Gi2/0/18 auto off 0.0 n/a n/a 30.0

Gi2/0/19 auto off 0.0 n/a n/a 30.0

Gi2/0/20 auto off 0.0 n/a n/a 30.0

Gi2/0/21 auto on 16.8 AIR-CAP3602I-E-K9 4 30.0

Gi2/0/22 auto on 15.4 AIR-CAP3602I-E-K9 4 30.0

Gi2/0/23 auto off 0.0 n/a n/a 30.0

Gi2/0/24 auto off 0.0 n/a n/a 30.0

Module Available Used Remaining

(Watts) (Watts) (Watts)

------ --------- -------- ---------

3 720.0 78.3 641.7

Interface Admin Oper Power Device Class Max

(Watts)

--------- ------ ---------- ------- ------------------- ----- ----

Gi3/0/1 auto off 0.0 n/a n/a 30.0

Gi3/0/2 auto off 0.0 n/a n/a 30.0

Gi3/0/3 auto off 0.0 n/a n/a 30.0

Gi3/0/4 auto off 0.0 n/a n/a 30.0

Gi3/0/5 auto off 0.0 n/a n/a 30.0

Gi3/0/6 auto off 0.0 n/a n/a 30.0

Gi3/0/7 auto on 3.9 Ieee PD 2 30.0

Gi3/0/8 auto off 0.0 n/a n/a 30.0

Gi3/0/9 auto on 3.9 Ieee PD 2 30.0

Gi3/0/10 auto on 5.8 Ieee PD 2 30.0

Gi3/0/11 auto on 3.9 Ieee PD 2 30.0

Gi3/0/12 auto off 0.0 n/a n/a 30.0

Gi3/0/13 auto off 0.0 n/a n/a 30.0

Gi3/0/14 auto off 0.0 n/a n/a 30.0

Gi3/0/15 static off 30.0 n/a n/a 30.0

Gi3/0/16 auto off 0.0 n/a n/a 30.0

Gi3/0/17 auto off 0.0 n/a n/a 30.0

Gi3/0/18 auto off 0.0 n/a n/a 30.0

Gi3/0/19 auto off 0.0 n/a n/a 30.0

Gi3/0/20 auto faulty 0.0 n/a n/a 30.0

Gi3/0/21 auto on 15.4 AIR-CAP3602I-E-K9 4 30.0

Gi3/0/22 auto on 15.4 AIR-CAP3602I-E-K9 4 30.0

Gi3/0/23 auto faulty 0.0 n/a n/a 30.0

Gi3/0/24 auto off 0.0 n/a n/a 30.0

Module Available Used Remaining

(Watts) (Watts) (Watts)

------ --------- -------- ---------

4 n/a n/a n/a

Interface Admin Oper Power Device Class Max

(Watts)

--------- ------ ---------- ------- ------------------- ----- ----

Module Available Used Remaining

Interface Admin Oper Power Device Class Max

(Watts)

--------- ------ ---------- ------- ------------------- ----- ----

(Watts) (Watts) (Watts)

------ --------- -------- ---------

5 720.0 11.7 708.3

Interface Admin Oper Power Device Class Max

(Watts)

--------- ------ ---------- ------- ------------------- ----- ----

Gi5/0/1 auto off 0.0 n/a n/a 30.0

Gi5/0/2 auto off 0.0 n/a n/a 30.0

Gi5/0/3 auto off 0.0 n/a n/a 30.0

Gi5/0/4 auto off 0.0 n/a n/a 30.0

Gi5/0/5 auto off 0.0 n/a n/a 30.0

Gi5/0/6 auto off 0.0 n/a n/a 30.0

Gi5/0/7 auto off 0.0 n/a n/a 30.0

Gi5/0/8 auto off 0.0 n/a n/a 30.0

Gi5/0/9 auto off 0.0 n/a n/a 30.0

Gi5/0/10 auto off 0.0 n/a n/a 30.0

Gi5/0/11 auto off 0.0 n/a n/a 30.0

Gi5/0/12 auto on 3.9 Ieee PD 2 30.0

Gi5/0/13 auto off 0.0 n/a n/a 30.0

Gi5/0/14 auto on 3.9 Ieee PD 2 30.0

Gi5/0/15 auto off 0.0 n/a n/a 30.0

Gi5/0/16 auto off 0.0 n/a n/a 30.0

Gi5/0/17 auto off 0.0 n/a n/a 30.0

Gi5/0/18 auto off 0.0 n/a n/a 30.0

Gi5/0/19 auto on 3.9 Ieee PD 2 30.0

Gi5/0/20 auto off 0.0 n/a n/a 30.0

Gi5/0/21 auto off 0.0 n/a n/a 30.0

Gi5/0/22 auto off 0.0 n/a n/a 30.0

Gi5/0/23 auto off 0.0 n/a n/a 30.0

Gi5/0/24 auto off 0.0 n/a n/a 30.0

Module Available Used Remaining

(Watts) (Watts) (Watts)

------ --------- -------- ---------

6 720.0 16.8 703.2

Interface Admin Oper Power Device Class Max

(Watts)

--------- ------ ---------- ------- ------------------- ----- ----

Gi6/0/1 auto off 0.0 n/a n/a 30.0

Gi6/0/2 auto off 0.0 n/a n/a 30.0

Gi6/0/3 auto off 0.0 n/a n/a 30.0

Gi6/0/4 auto off 0.0 n/a n/a 30.0

Gi6/0/5 auto off 0.0 n/a n/a 30.0

Gi6/0/6 auto off 0.0 n/a n/a 30.0

Gi6/0/7 auto off 0.0 n/a n/a 30.0

Gi6/0/8 auto off 0.0 n/a n/a 30.0

Gi6/0/9 auto off 0.0 n/a n/a 30.0

Gi6/0/10 auto faulty 0.0 n/a n/a 30.0

Gi6/0/11 auto off 0.0 n/a n/a 30.0

Gi6/0/12 auto off 0.0 n/a n/a 30.0

Gi6/0/13 auto off 0.0 n/a n/a 30.0

Gi6/0/14 auto off 0.0 n/a n/a 30.0

Gi6/0/15 auto off 0.0 n/a n/a 30.0

Interface Admin Oper Power Device Class Max

(Watts)

--------- ------ ---------- ------- ------------------- ----- ----

Gi6/0/16 auto off 0.0 n/a n/a 30.0

Gi6/0/17 auto off 0.0 n/a n/a 30.0

Gi6/0/18 auto off 0.0 n/a n/a 30.0

Gi6/0/19 auto off 0.0 n/a n/a 30.0

Gi6/0/20 auto off 0.0 n/a n/a 30.0

Gi6/0/21 auto off 0.0 n/a n/a 30.0

Gi6/0/22 auto on 16.8 AIR-CAP3602I-E-K9 4 30.0

Gi6/0/23 auto off 0.0 n/a n/a 30.0

Gi6/0/24 auto off 0.0 n/a n/a 30.0

#show ver

Cisco IOS Software, IOS-XE Software, Catalyst L3 Switch Software (CAT3K_CAA-UNIVERSALK9-M), Version 03.06.06E RELEASE SOFTWARE (fc1)

Technical Support: http://www.cisco.com/techsupport

Compiled Sat 17-Dec-16 00:22 by prod_rel_team

licensed under the GNU General Public License ("GPL") Version 2.0. The

software code licensed under GPL Version 2.0 is free software that comes

with ABSOLUTELY NO WARRANTY. You can redistribute and/or modify such

GPL code under the terms of GPL Version 2.0.

(http://www.gnu.org/licenses/gpl-2.0.html) For more details, see the

documentation or "License Notice" file accompanying the IOS-XE software,

or the applicable URL provided on the flyer accompanying the IOS-XE

software.

ROM: IOS-XE ROMMON

BOOTLDR: CAT3K_CAA Boot Loader (CAT3K_CAA-HBOOT-M) Version 3.58, RELEASE SOFTWARE (P)

LLSASW01 uptime is 4 weeks, 1 hour, 59 minutes

Uptime for this control processor is 4 weeks, 2 hours, 6 minutes

System returned to ROM by reload at 07:20:26 GMT Tue Jan 29 2019

System restarted at 07:29:13 GMT Tue Jan 29 2019

System image file is "flash:packages.conf"

Last reload reason: Reload command

This product contains cryptographic features and is subject to United

States and local country laws governing import, export, transfer and

use. Delivery of Cisco cryptographic products does not imply

third-party authority to import, export, distribute or use encryption.

Importers, exporters, distributors and users are responsible for

compliance with U.S. and local country laws. By using this product you

agree to comply with applicable laws and regulations. If you are unable

to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:

http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to

export@cisco.com.

License Level: Ipbase

License Type: Permanent

Next reload license Level: Ipbase

cisco WS-C3850-24P (MIPS) processor with 4194304K bytes of physical memory.

Processor board ID FOC2012U0LV

18 Virtual Ethernet interfaces

168 Gigabit Ethernet interfaces

24 Ten Gigabit Ethernet interfaces

2048K bytes of non-volatile configuration memory.

4194304K bytes of physical memory.

250456K bytes of Crash Files at crashinfo:.

250456K bytes of Crash Files at crashinfo-2:.

250456K bytes of Crash Files at crashinfo-3:.

250456K bytes of Crash Files at crashinfo-4:.

252000K bytes of Crash Files at crashinfo-5:.

252000K bytes of Crash Files at crashinfo-6:.

1609272K bytes of Flash at flash:.

1609272K bytes of Flash at flash-2:.

1609272K bytes of Flash at flash-3:.

1609272K bytes of Flash at flash-4:.

1611414K bytes of Flash at flash-5:.

1611414K bytes of Flash at flash-6:.

0K bytes of Dummy USB Flash at usbflash0:.

0K bytes of Dummy USB Flash at usbflash0-2:.

0K bytes of Dummy USB Flash at usbflash0-3:.

0K bytes of Dummy USB Flash at usbflash0-4:.

0K bytes of Dummy USB Flash at usbflash0-5:.

0K bytes of Dummy USB Flash at usbflash0-6:.

0K bytes of at webui:.

Base Ethernet MAC Address : 04:2a:e2:3f:77:00

Motherboard Assembly Number : 73-15805-04

Motherboard Serial Number : FOC20123AG3

Model Revision Number : AA0

Motherboard Revision Number : B0

Model Number : WS-C3850-24P

System Serial Number : FOC2012U0LV

Switch Ports Model SW Version SW Image Mode

------ ----- ----- ---------- ---------- ----

* 1 32 WS-C3850-24P 03.06.06E cat3k_caa-universalk9 INSTALL

2 32 WS-C3850-24P 03.06.06E cat3k_caa-universalk9 INSTALL

3 32 WS-C3850-24P 03.06.06E cat3k_caa-universalk9 INSTALL

4 32 WS-C3850-24T 03.06.06E cat3k_caa-universalk9 INSTALL

5 32 WS-C3850-24P 03.06.06E cat3k_caa-universalk9 INSTALL

6 32 WS-C3850-24P 03.06.06E cat3k_caa-universalk9 INSTALL

Switch 02

---------

Switch uptime : 4 weeks, 2 hours, 5 minutes

Base Ethernet MAC Address : 04:2a:e2:3f:76:00

Motherboard Assembly Number : 73-15805-04

Motherboard Serial Number : FOC20123AHM

Model Revision Number : AA0

Motherboard Revision Number : B0

Model Number : WS-C3850-24P

System Serial Number : FCW2012C0L6

Switch 03

---------

Switch uptime : 4 weeks, 2 hours, 6 minutes

Base Ethernet MAC Address : 04:2a:e2:3f:76:80

Motherboard Assembly Number : 73-15805-04

Motherboard Serial Number : FOC20123AT2

Model Revision Number : AA0

Motherboard Revision Number : B0

Model Number : WS-C3850-24P

System Serial Number : FOC2012U0LW

Switch 04

---------

Switch uptime : 4 weeks, 2 hours, 5 minutes

Base Ethernet MAC Address : 88:90:8d:d8:63:00

Motherboard Assembly Number : 73-14443-08

Motherboard Serial Number : FOC1815006Y

Model Revision Number : S0

Motherboard Revision Number : A0

Model Number : WS-C3850-24T

System Serial Number : FOC1823U10S

Switch 05

---------

Switch uptime : 4 weeks, 2 hours, 6 minutes

Base Ethernet MAC Address : 00:f8:2c:9b:9a:00

Motherboard Assembly Number : 73-15805-05

Motherboard Serial Number : FOC22035QYP

Model Revision Number : AD0

Motherboard Revision Number : B0

Model Number : WS-C3850-24P

System Serial Number : FCW2204C0VC

Switch 06

---------

Switch uptime : 4 weeks, 2 hours, 6 minutes

Base Ethernet MAC Address : 00:f8:2c:9b:97:00

Motherboard Assembly Number : 73-15805-05

Motherboard Serial Number : FOC22035QMH

Model Revision Number : AD0

Motherboard Revision Number : B0

Model Number : WS-C3850-24P

System Serial Number : FCW2204C0V2

Configuration register is 0x102

Note:

we have an unmanaged 4ports PoE switch that is connected to Gi2/0/15 and throws out PoE messages constantly but I don't think this is related to the current issue as it was in this hardware configuration way before any problems came up, here is the term mon just in case:

Feb 26 08:21:24: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/15, changed state to up

Feb 26 08:21:54: %ILPOWER-7-DETECT: Interface Gi2/0/15: Power Device detected: IEEE PD

Feb 26 08:21:57: %ILPOWER-5-IEEE_DISCONNECT: Interface Gi2/0/15: PD removed

Feb 26 08:21:57: %ILPOWER-3-CONTROLLER_PORT_ERR: Controller port error, Interface Gi2/0/15: Power given, but Power Controller does not report Power Good

Feb 26 08:23:31: %ILPOWER-7-DETECT: Interface Gi2/0/15: Power Device detected: IEEE PD

Feb 26 08:23:34: %ILPOWER-5-IEEE_DISCONNECT: Interface Gi2/0/15: PD removed

Feb 26 08:23:34: %ILPOWER-3-CONTROLLER_PORT_ERR: Controller port error, Interface Gi2/0/15: Power given, but Power Controller does not report Power Good

Feb 26 08:23:36: %ILPOWER-7-DETECT: Interface Gi2/0/15: Power Device detected: IEEE PD

Feb 26 08:23:39: %ILPOWER-5-IEEE_DISCONNECT: Interface Gi2/0/15: PD removed

Feb 26 08:23:39: %ILPOWER-3-CONTROLLER_PORT_ERR: Controller port error, Interface Gi2/0/15: Power given, but Power Controller does not report Power Good

thanks for your suggestions in advance

Leo Laohoo · ‎03-12-2019

Gi 1/0/2: Faulty cable near the far end (away from the switch port);

Gi 1/0/7: Faulty cable near the far end (away from the switch port);

Gi 2/0/14: All good. All four pairs terminated correctly;

Gi 3/0/20: All good. All four pairs terminated correctly;

Gi 3/0/23: Faulty cable near the far end (away from the switch port); and

Gi 5/0/7: Faulty cable near the far end (away from the switch port)

479130 · ‎03-14-2019

Note: If I disconnect device from faulty switchport, then connect a non PoE device to it (with the same LAN cable), cycle switchport with shut / no shut.

Reconnect original PoE device to switchport, everything works fine again.

I have done this process to all faulty ports and all works right now.

We are going to update switchstack to Denali cat3k_caa-universalk9.16.03.07.SPA.bin

I will report back if this issue is fixed with firmware upgrade

479130 · ‎04-01-2019

Did an upgrade to cat3k_caa-universalk9.16.03.07.SPA.bin

Now all PoE requiring ports are marked as faulty on Switch1,

I've tried to cycle them with shut/no shut with a non PoE device connected to it but goes bad as soon as you re-plug PoE device

In fact there is a new error now from terminal indicating Power controller overheat

Does entire switch needs to be replaced? or just power supply?

#show power inline

Module Available Used Remaining

(Watts) (Watts) (Watts)

------ --------- -------- ---------

1 720.0 0.0 720.0

Interface Admin Oper Power Device Class Max

(Watts)

--------- ------ ---------- ------- ------------------- ----- ----

Gi1/0/1 auto off 0.0 n/a n/a 30.0

Gi1/0/2 auto faulty 0.0 n/a n/a 30.0

Gi1/0/3 auto faulty 0.0 n/a n/a 30.0

Gi1/0/4 off off 0.0 n/a n/a 30.0

Gi1/0/5 auto off 0.0 n/a n/a 30.0

Gi1/0/6 auto off 0.0 n/a n/a 30.0

Gi1/0/7 auto faulty 0.0 n/a n/a 30.0

Gi1/0/8 auto faulty 0.0 n/a n/a 30.0

Gi1/0/9 auto off 0.0 n/a n/a 30.0

Gi1/0/10 auto off 0.0 n/a n/a 30.0

Gi1/0/11 auto off 0.0 n/a n/a 30.0

Gi1/0/12 auto off 0.0 n/a n/a 30.0

Gi1/0/13 auto off 0.0 n/a n/a 30.0

Gi1/0/14 auto off 0.0 n/a n/a 30.0

Gi1/0/15 auto off 0.0 n/a n/a 30.0

Gi1/0/16 auto off 0.0 n/a n/a 30.0

Gi1/0/17 auto off 0.0 n/a n/a 30.0

Gi1/0/18 auto off 0.0 n/a n/a 30.0

Gi1/0/19 auto off 0.0 n/a n/a 30.0

Gi1/0/20 auto off 0.0 n/a n/a 30.0

Gi1/0/21 auto off 0.0 n/a n/a 30.0

Gi1/0/22 auto faulty 0.0 n/a n/a 30.0

Gi1/0/23 auto off 0.0 n/a n/a 30.0

Gi1/0/24 auto off 0.0 n/a n/a 30.0

#show environment power switch 1

SW PID Serial# Status Sys Pwr PoE Pwr Watts

-- ------------------ ---------- --------------- ------- ------- -----

1A PWR-C1-350WAC ART2003F757 OK Good Good 350

1B PWR-C1-715WAC DCA1949G04Z OK Good Good 715

Apr 1 13:18:48: %ILPOWER-5-IEEE_DISCONNECT: Interface Gi1/0/2: PD removed

Apr 1 13:18:49: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/2, changed state to down

Apr 1 13:18:50: %ILPOWER-7-DETECT: Interface Gi2/0/14: Power Device detected: IEEE PD

Apr 1 13:18:50: %ILPOWER-7-DETECT: Interface Gi1/0/2: Power Device detected: IEEE PD

Apr 1 13:18:51: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet2/0/11, changed state to down

Apr 1 13:18:52: %LINK-3-UPDOWN: Interface GigabitEthernet2/0/11, changed state to down

Apr 1 13:18:57: %ILPOWER-5-IEEE_DISCONNECT: Interface Gi2/0/14: PD removed

Apr 1 13:18:57: %ILPOWER-3-CONTROLLER_PORT_ERR: Controller port error, Interface Gi2/0/14: Power given, but Power Controller does not report Power Good

Apr 1 13:18:59: %LINK-3-UPDOWN: Interface GigabitEthernet2/0/11, changed state to up

Apr 1 13:18:59: %ILPOWER-5-IEEE_DISCONNECT: Interface Gi1/0/2: PD removed

Apr 1 13:18:59: %ILPOWER-3-CONTROLLER_PORT_ERR: Controller port error, Interface Gi1/0/2: Power given, but Power Controller does not report Power Good

Apr 1 13:19:00: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet2/0/11, changed state to up

Apr 1 13:19:07: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/3, changed state to down

Apr 1 13:19:07: %ILPOWER-5-IEEE_DISCONNECT: Interface Gi2/0/14: PD removed

Apr 1 13:19:07: %ILPOWER-5-IEEE_DISCONNECT: Interface Gi1/0/3: PD removed

Apr 1 13:19:07: %ILPOWER-3-CONTROLLER_PORT_ERR: Controller port error, Interface Gi1/0/3: Power Controller reports power supply over heat

Apr 1 13:19:08: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/3, changed state to down

Apr 1 13:19:09: %ILPOWER-5-IEEE_DISCONNECT: Interface Gi1/0/2: PD removed

Apr 1 13:19:10: %ILPOWER-7-DETECT: Interface Gi1/0/2: Power Device detected: IEEE PD

Apr 1 13:19:18: %ILPOWER-5-IEEE_DISCONNECT: Interface Gi1/0/2: PD removed

Apr 1 13:19:18: %ILPOWER-3-CONTROLLER_PORT_ERR: Controller port error, Interface Gi1/0/2: Power given, but Power Controller does not report Power Good

Apr 1 13:19:28: %ILPOWER-5-IEEE_DISCONNECT: Interface Gi1/0/2: PD removed

Apr 1 13:19:29: %ILPOWER-7-DETECT: Interface Gi1/0/2: Power Device detected: IEEE PD

Apr 1 13:19:31: %ILPOWER-5-POWER_GRANTED: Interface Gi1/0/2: Power granted

Apr 1 13:19:35: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/2, changed state to up

Apr 1 13:19:36: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/2, changed state to up

Apr 1 13:19:46: %ILPOWER-5-IEEE_DISCONNECT: Interface Gi1/0/2: PD removed

Apr 1 13:19:47: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/2, changed state to down

Apr 1 13:19:48: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/2, changed state to down

Apr 1 13:19:49: %ILPOWER-7-DETECT: Interface Gi1/0/2: Power Device detected: IEEE PD

Apr 1 13:19:51: %ILPOWER-7-DETECT: Interface Gi2/0/14: Power Device detected: IEEE PD

Apr 1 13:19:51: %ILPOWER-5-POWER_GRANTED: Interface Gi1/0/2: Power granted

Apr 1 13:22:52: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/7, changed state to down

Apr 1 13:22:53: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/7, changed state to down

Apr 1 13:23:01: %ILPOWER-5-IEEE_DISCONNECT: Interface Gi1/0/7: PD removed

Apr 1 13:23:04: %ILPOWER-5-IEEE_DISCONNECT: Interface Gi1/0/7: PD removed

Apr 1 13:23:04: %ILPOWER-3-CONTROLLER_PORT_ERR: Controller port error, Interface Gi1/0/7: Power Controller reports power supply over heat

Leo Laohoo · ‎04-01-2019

Your horizontal cabling is faulty.

479130 · ‎04-02-2019

sorry what do you mean for "horizontal cabling" ? :)

Leo Laohoo · ‎04-02-2019

Cable run from the patch panel to the wall port.

479130 · ‎04-04-2019

Sorry while I appreciate your help I still believe this is not a cabling issue.

I have plugged a PoE VoIP phone directly to the switch1 int gig 1/0/7 with new LAN cable and got this output

Apr 4 15:08:44: %ILPOWER-5-IEEE_DISCONNECT: Interface Gi1/0/7: PD removed

Apr 4 15:08:44: %ILPOWER-3-CONTROLLER_PORT_ERR: Controller port error, Interface Gi1/0/7: Power Controller reports power supply over heat

Apr 4 15:08:44: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/7, changed state to down

sure, this cant be cable, I'm inclined to think its a hardware fault now

C3NKO · ‎07-19-2019

Were you able to resolve the issue by any chance, having a similar issue with a 3850 running 16.3.8 . None of the ports say faulty on mine.

Module Available Used Remaining
(Watts) (Watts) (Watts)
------ --------- -------- ---------
1 600.0 0.0 600.0
Interface Admin Oper Power Device Class Max
(Watts)
--------- ------ ---------- ------- ------------------- ----- ----
Gi1/0/1 auto off 0.0 n/a n/a 60.0
Gi1/0/2 auto off 0.0 n/a n/a 60.0
Gi1/0/3 auto off 0.0 n/a n/a 60.0
Gi1/0/4 auto off 0.0 n/a n/a 60.0
Gi1/0/5 auto off 0.0 n/a n/a 60.0
Gi1/0/6 auto off 0.0 n/a n/a 60.0
Gi1/0/7 auto off 0.0 n/a n/a 60.0
Gi1/0/8 auto off 0.0 n/a n/a 60.0
Gi1/0/9 auto off 0.0 n/a n/a 60.0
Gi1/0/10 auto off 0.0 n/a n/a 60.0
Gi1/0/11 auto off 0.0 n/a n/a 60.0
Gi1/0/12 auto off 0.0 n/a n/a 60.0
Gi1/0/13 auto off 0.0 n/a n/a 60.0
Gi1/0/14 auto off 0.0 n/a n/a 60.0
Gi1/0/15 auto off 0.0 n/a n/a 60.0
Gi1/0/16 auto off 0.0 n/a n/a 60.0
Gi1/0/17 auto off 0.0 n/a n/a 60.0
Gi1/0/18 auto off 0.0 n/a n/a 60.0
Gi1/0/19 auto off 0.0 n/a n/a 60.0
Gi1/0/20 auto off 0.0 n/a n/a 60.0
Gi1/0/21 auto off 0.0 n/a n/a 60.0
Gi1/0/22 auto off 0.0 n/a n/a 60.0
Gi1/0/23 auto off 0.0 n/a n/a 60.0
Gi1/0/24 auto off 0.0 n/a n/a 60.0
Gi1/0/25 auto off 0.0 n/a n/a 60.0
Gi1/0/26 auto off 0.0 n/a n/a 60.0
Gi1/0/27 auto off 0.0 n/a n/a 60.0

Leo Laohoo · ‎07-19-2019

3850 supports TDR. What is the result of a TDR test on the cable run?

SahilUppal87258 · ‎09-30-2020

We are also facing same issue.
Were you able to resolve it with iOS upgrade?

We are running 03.07.05E

I am assuming it is ASIC level fault.

Leo Laohoo · ‎09-30-2020

Kindly open a new thread so we can diagnose this properly.