cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
22318
Views
16
Helpful
54
Replies

PSA: Avoid CBS350 Switches - Critical POE Defect

d0nni3q84
Level 1
Level 1

Hello Community,

 

I'm writing to let you know that there is a critical POE defect in the CBS350 switches and you should avoid them if you require stable POE operations. This apparent software defect causes POE ports to randomly fault after 8 to 12 hours and the powered device will go offline. Cisco does not have a resolution, workaround, or timeline to address this situation.

 

Please help Cisco prioritize this issue by opening a TAC case if you are experiencing this issue. Additionally, please escalate this for additional visibility through your sales channels.

 

All the best!

54 Replies 54

You have an 8-port PoE switch on which PoE is disabled on all PoE ports and to which no PDs have ever been connected on any port at any point during its current uptime, and all ports just randomly faulted? Are the ports faulted because you've manually set the power limit to 0W, as some ostensible attempt at hardening them?

The timestamps of your posts in the other thread reveal that you changed power limit mode and disabled inrush test prior to opening your TAC case.

Post the startup-config for that switch, cycle it, and then re-post that output.


@cyberconsultants wrote:

You have an 8-port PoE switch on which PoE is disabled on all PoE ports and to which no PDs have ever been connected on any port at any point during its current uptime, and all ports just randomly faulted?


Yes, that is correct.

 


@cyberconsultants wrote:

Are the ports faulted because you've manually set the power limit to 0W, as some ostensible attempt at hardening them?

No, the ports fault regardless of the power limit or even if it is cleared. Your postulate that I am trying to harden the switch is grossly incorrect. Even with the default POE settings, the strange and erratic behavior takes place.

 

SW-3#show clock

22:33:07 UTC Mar 3 2022
Time from SNTP is enabled
Operational Time Source: SNTP (last synchronized 00:13:51 ago)
Time from Browser is disabled


SW-3#show power inline

Power-limit mode: Port based
Usage threshold: 95%
Trap: Disable
Legacy Mode: Enable
Inrush Test: Disable
Guard-Band Value: Static (2W)

Unit Module         Nominal   Consumed     Temp     SW            PSE chipset
                    Power (W) Power (W)    (C)      Version       HW Revision
---- -------------- --------- ------------ -------- ------------- -----------------
1    CBS350-8P-E-2G 60        0 (0%)       47       0.2.0.17       TPS2388 - 0x40C4


Interface    Admin       Oper         Power (W)     Class     Device     Priority
---------- ---------- ----------- ----------------- ----- -------------- --------
gi1        Never      Fault       0.0               -                    low
gi2        Never      Fault       0.0               -                    low
gi3        Never      Fault       0.0               -                    low
gi4        Never      Fault       0.0               -                    low
gi5        Never      Fault       0.0               -                    low
gi6        Never      Fault       0.0               -                    low
gi7        Never      Fault       0.0               -                    low
gi8        Never      Fault       0.0               -                    low


SW-3#configure
SW-3(config)#interface range Gi1-8
SW-3(config-if-range)#power inline auto
SW-3(config-if-range)#power inline never
SW-3(config-if-range)#end


SW-3#show power inline

Power-limit mode: Port based
Usage threshold: 95%
Trap: Disable
Legacy Mode: Enable
Inrush Test: Disable
Guard-Band Value: Static (2W)

Unit Module         Nominal   Consumed     Temp     SW            PSE chipset
                    Power (W) Power (W)    (C)      Version       HW Revision
---- -------------- --------- ------------ -------- ------------- -----------------
1    CBS350-8P-E-2G 60        0 (0%)       47       0.2.0.17       TPS2388 - 0x40C4


Interface    Admin       Oper         Power (W)     Class     Device     Priority
---------- ---------- ----------- ----------------- ----- -------------- --------
gi1        Never      Off         0.0               -                    low
gi2        Never      Off         0.0               -                    low
gi3        Never      Off         0.0               -                    low
gi4        Never      Off         0.0               -                    low
gi5        Never      Off         0.0               -                    low
gi6        Never      Off         0.0               -                    low
gi7        Never      Off         0.0               -                    low
gi8        Never      Off         0.0               -                    low


SW-3#show power inline GigabitEthernet 1

Interface    Admin       Oper         Power (W)     Class     Device     Priority
---------- ---------- ----------- ----------------- ----- -------------- --------
gi1        Never      Off         0.0               -                    low


Port Status:               Port is off. User setting
Port standard:             802.3AT
Admin power limit:         0.0 watts
Time range:
Max Power Allocation:      0.0 watts
Spare pair:                Disabled
Negotiated power:          0.0 watts (None)
Current (mA):              0
Voltage(V):                0.0
Overload Counter:          68
Short Counter:             6
Denied Counter:            0
Absent Counter:            0
Invalid Signature Counter: 0

SW-3#show clock 22:34:07 UTC Mar 3 2022 Time from SNTP is enabled Operational Time Source: SNTP (last synchronized 00:14:51 ago) Time from Browser is disabled SW-3#

 

Hardening, manual power budgeting—you're right, the reason any port would have its power limit manually set at 0W (and which already has its PoE mode administratively disabled) is beyond unclear and I'm speculating.

Set limit mode to class, or at least set all individual port power limits back to default, re-enable inrush test, commit/copy to su, post su config, cycle the switch (not just PoE mode on all PoE ports), and then re-post show power inline and a show power inline [interface]. If you can show a random fault at that point, preferably in continuous output like you've done above, then other configuration settings aside from limit mode and/or port power limit or, I suppose, software bugs might be considered.

You originally describe the issue as:


@d0nni3q84 wrote:
This apparent software defect causes POE ports to randomly fault after 8 to 12 hours and the powered device will go offline.

Are we talking faults with PDs connected, or faults completely at random? If completely at random, then take the steps above. Or try a factory reset for that matter. Faults with PDs connected versus completely random faults are obviously entirely different troubleshoots.


@cyberconsultants wrote:

or, I suppose, software bugs might be considered.

Thanks so much @cyberconsultants, for your efforts to bring a notion of doubt to this situation under the guise of assistance. Nonetheless, I hope the below output provides proof beyond a reasonable doubt that the aforementioned erratic behavior exists. I'm using the power inline never command on the individual ports is for two reasons: (1) to workaround known issue CSCvu81814 and (2) the default admin power limit is 30W which oversubscribes the switch's POE capability.

 


@cyberconsultants wrote:

Are we talking faults with PDs connected, or faults completely at random?


Both, see below.

 

SW-3#show clock | include UTC
13:59:54 UTC Mar 4 2022

SW-3#show system | include Up System Up Time (days,hour:min:sec): 00,09:02:54
SW-3#show power inline Power-limit mode: Class based Usage threshold: 95% Trap: Disable Legacy Mode: Enable Inrush Test: Enable Guard-Band Value: Static (2W) Unit Module Nominal Consumed Temp SW PSE chipset Power (W) Power (W) (C) Version HW Revision ---- -------------- --------- ------------ -------- ------------- ----------------- 1 CBS350-8P-E-2G 60 0 (0%) 46 0.2.0.17 TPS2388 - 0x40C4 Interface Admin Oper Power (W) Class Device Priority ---------- ---------- ----------- ----------------- ----- -------------- -------- gi1 Never Off 0.0 - low gi2 Auto Searching 0.0 - low gi3 Never Off 0.0 - low gi4 Never Off 0.0 - low gi5 Never Off 0.0 - low gi6 Never Fault 0.0 - low gi7 Never Off 0.0 - low gi8 Never Fault 0.0 - low
SW-3#show power inline GigabitEthernet 6 Interface Admin Oper Power (W) Class Device Priority ---------- ---------- ----------- ----------------- ----- -------------- -------- gi6 Never Fault 0.0 - low Port Status: Port is off. Overload state Port standard: 802.3AT Admin power limit (for port power-limit mode): 30.0 watts Time range: Max Power Allocation: 30.0 watts Spare pair: Disabled Negotiated power: 0.0 watts (None) Current (mA): 0 Voltage(V): 0.0 Overload Counter: 24 Short Counter: 1 Denied Counter: 0 Absent Counter: 2 Invalid Signature Counter: 0
SW-3#show interfaces status GigabitEthernet 6 Flow Link Back Mdix Port Type Duplex Speed Neg ctrl State Pressure Mode -------- ------------ ------ ----- -------- ---- ----------- -------- ------- gi6 1G-Copper -- -- -- -- Down -- --
SW-3#show power inline GigabitEthernet 8 Interface Admin Oper Power (W) Class Device Priority ---------- ---------- ----------- ----------------- ----- -------------- -------- gi8 Never Fault 0.0 - low Port Status: Port is off. Overload state Port standard: 802.3AT Admin power limit (for port power-limit mode): 30.0 watts Time range: Max Power Allocation: 30.0 watts Spare pair: Disabled Negotiated power: 0.0 watts (None) Current (mA): 0 Voltage(V): 0.0 Overload Counter: 35 Short Counter: 0 Denied Counter: 0 Absent Counter: 4 Invalid Signature Counter: 0
SW-3#show interfaces status GigabitEthernet 8 Flow Link Back Mdix Port Type Duplex Speed Neg ctrl State Pressure Mode -------- ------------ ------ ----- -------- ---- ----------- -------- ------- gi8 1G-Copper -- -- -- -- Down -- --
SW-3#show startup-config | exclude ^username config-file-header SW-3 v3.1.1.7 / RCBS3.1_930_871_081 CLI v1.0 file SSD indicator encrypted @ ssd-control-start ssd config ssd file passphrase control unrestricted no ssd file integrity control ssd-control-end cb0a3fdb1f3a1af4e4430033719968c0 ! ! unit-type-control-start unit-type unit 1 network gi uplink none unit-type-control-end ! spanning-tree priority 8192 port jumbo-frame vlan database vlan 1010,1013-1014 exit voice vlan oui-table add 0001e3 Siemens_AG_phone voice vlan oui-table add 00036b Cisco_phone voice vlan oui-table add 00096e Avaya voice vlan oui-table add 000fe2 H3C_Aolynk voice vlan oui-table add 0060b9 Philips_and_NEC_AG_phone voice vlan oui-table add 00d01e Pingtel_phone voice vlan oui-table add 00e075 Polycom/Veritel_phone voice vlan oui-table add 00e0bb 3Com_phone no eee enable hostname SW-3 no logging console passwords aging 0 ip ssh server snmp-server location Family snmp-server contact noc@quindar7.net no ip http server sntp server 10.0.10.1 poll no sntp server pool.ntp.org no sntp server time-a.timefreq.bldrdoc.gov no sntp server time-b.timefreq.bldrdoc.gov no sntp server time-c.timefreq.bldrdoc.gov no sntp server time-pnp.cisco.com ! interface vlan 1 name Default no ip address dhcp shutdown ! interface vlan 1010 name Management ip address 10.0.10.13 255.255.255.128 ! interface vlan 1013 name IoT ! interface vlan 1014 name Media ! interface GigabitEthernet1 description UPS-3 switchport access vlan 1010 switchport trunk native vlan none switchport trunk allowed vlan none power inline never ! interface GigabitEthernet2 description IOT-ZWAVE switchport access vlan 1013 switchport trunk native vlan none switchport trunk allowed vlan none ! interface GigabitEthernet3 speed 10 no negotiation description IOT-POOL switchport access vlan 1013 switchport trunk native vlan none switchport trunk allowed vlan none power inline never ! interface GigabitEthernet4 description IOT-HVAC switchport access vlan 1013 switchport trunk native vlan none switchport trunk allowed vlan none power inline never ! interface GigabitEthernet5 description SND-FAMILY switchport access vlan 1014 switchport trunk native vlan none switchport trunk allowed vlan none power inline never ! interface GigabitEthernet6 description BLU-FAMILY switchport access vlan 1014 switchport trunk native vlan none switchport trunk allowed vlan none power inline never ! interface GigabitEthernet7 description ATV-FAMILY switchport access vlan 1014 switchport trunk native vlan none switchport trunk allowed vlan none power inline never ! interface GigabitEthernet8 shutdown description [UNUSED] switchport access vlan none switchport trunk native vlan none switchport trunk allowed vlan none power inline never ! interface GigabitEthernet9 shutdown description [UNUSED] switchport access vlan none switchport trunk native vlan none switchport trunk allowed vlan none ! interface GigabitEthernet10 description SW-1.Gi14 ip dhcp snooping trust switchport mode trunk switchport access vlan none switchport trunk native vlan none switchport trunk allowed vlan 1010,1013-1014 ! interface Port-Channel1 shutdown description [UNUSED] switchport access vlan none switchport trunk native vlan none switchport trunk allowed vlan none ! interface Port-Channel2 shutdown description [UNUSED] switchport access vlan none switchport trunk native vlan none switchport trunk allowed vlan none ! interface Port-Channel3 shutdown description [UNUSED] switchport access vlan none switchport trunk native vlan none switchport trunk allowed vlan none ! interface Port-Channel4 shutdown description [UNUSED] switchport access vlan none switchport trunk native vlan none switchport trunk allowed vlan none ! interface Port-Channel5 shutdown description [UNUSED] switchport access vlan none switchport trunk native vlan none switchport trunk allowed vlan none ! interface Port-Channel6 shutdown description [UNUSED] switchport access vlan none switchport trunk native vlan none switchport trunk allowed vlan none ! interface Port-Channel7 shutdown description [UNUSED] switchport access vlan none switchport trunk native vlan none switchport trunk allowed vlan none ! interface Port-Channel8 shutdown description [UNUSED] switchport access vlan none switchport trunk native vlan none switchport trunk allowed vlan none ! exit ip dhcp snooping ip dhcp snooping vlan 1010 ip dhcp snooping vlan 1013 ip dhcp snooping vlan 1014 ip default-gateway 10.0.10.1

 

For posterity: the only PoE issue ("defect"?) OP seems to have presented is with Ubiquiti PDs.


@cyberconsultants wrote:

For posterity: the only PoE issue ("defect"?) OP seems to have presented is with Ubiquiti PDs.


This is not true. I have also presented a case where a PoE port with no device connected at all goes into the Port is off. Overload state status.

CoreyP319
Cisco Employee
Cisco Employee

Hello d0nni3q84,

 

Thank you for raising this issue. I've not experienced the described issue with the PoE enabled CBS350 running in my home network. I've shared your post with the team and they are in turn sharing it internally to increase awareness while we investigate further. We'll provide more details or a workaround should they become available.

 

If anyone else is impacted, please open a case with TAC. It does help raise awareness on issues.

Hello @CoreyP319,

 

Please share the exact switch model and firmware version that you have running in your home network where you do not observe this behavior.

 

Thank you!

jimstewart77
Level 1
Level 1

I have seen ports on mine that have PoE live but a non-PD device (computer) connected show PoE fault.  The port however continues to work perfectly and the port itself isn't in fault, just the PoE.

Ports with PoE disabled and plugged into a non-PD device show PoE disabled.

I didn't give much thought to that fault state as it just seems to be a reporting oddity.

Same behavior here. Since the last firmware our Lancom devices are working, before there was definitely a problem with PoE.


@jimstewart77 wrote:

I have seen ports on mine that have PoE live but a non-PD device (computer) connected show PoE fault.  The port however continues to work perfectly and the port itself isn't in fault, just the PoE.


Yes, that's a known issue and benign, where the status is Port is off. Short condition and the Short Counter continuously increments. I'm not worried about the scenario, since as you said, the port sill operates.

d0nni3q84
Level 1
Level 1

During beta testing with Cisco TAC, I can confirm this issue is resolved in Software version 3.1.1.10 and MCU version 0xF0. Unfortunately, I do not have a release timeline available; however, I encourage you to reach out to Cisco TAC if you are experiencing this issue.

Klauzz
Level 1
Level 1

I am using a CBS350-8P-E-2G. At the moment just one POE device is connected. It's an Aruba AP22 Access Point.

 

I am also facing the Overload issue:

 

 

Interface    Admin       Oper         Power (W)     Class     Device     Priority
---------- ---------- ----------- ----------------- ----- -------------- --------
gi1        Never      Fault       0.0               0                    low


Port Status:               Port is off. Overload state
Port standard:             802.3AT
Admin power limit:         30.0 watts
Time range:
Max Power Allocation:      30.0 watts
Spare pair:                Disabled
Negotiated power:          0.0 watts (None)
Current (mA):              0
Voltage(V):                0.0
Overload Counter:          6
Short Counter:             0
Denied Counter:            1
Absent Counter:            0
Invalid Signature Counter: 0

 

 

The statistic looks like this:

 
Cisco.jpg

 

Is the denied counter also part of the known issue?

 

Thanks

 

Michael

I personally did not observe the Denied counter incrementing; however, it does mean that the switch did not provide power to the port since the requested power exceeded a limit. What is the output of the following commands?

show power inline GigabitEthernet 8
show run interface GigabitEthernet 8

Thanks for your reply. The output looks like this:

 

AzCiscoSwitch#show power inline GigabitEthernet 8

Interface Admin Oper Power (W) Class Device Priority ---------- ---------- ----------- ----------------- ----- -------------- -------- gi8 Auto On 5.641 (16.5) 3 high Port Status: Port is on. Valid resistor detected Port standard: 802.3AT Admin power limit: 30.0 watts Time range: Link partner standard: 802.3AF Max Power Allocation: 30.0 watts Spare pair: Disabled Negotiated power: 16.500 watts (LLDP) Current (mA): 107 Voltage(V): 53.0 Overload Counter: 17 Short Counter: 0 Denied Counter: 268 Absent Counter: 12 Invalid Signature Counter: 0
AzCiscoSwitch#show run interface GigabitEthernet 8
interface GigabitEthernet8
 description GzAccessPoint
 switchport mode trunk
 switchport access vlan 10
 power inline priority high
!

Thanks

 

Michael