Buy or Renew
Buy or Renew
Cisco + Splunk: It’s a new day for your data. Learn more.
 
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
cancel
Start a conversation
22368
Views
16
Helpful
54
Replies

PSA: Avoid CBS350 Switches - Critical POE Defect

Go to solution
d0nni3q84
Level 1
Level 1

‎03-02-2022 07:48 AM

Hello Community,

 

I'm writing to let you know that there is a critical POE defect in the CBS350 switches and you should avoid them if you require stable POE operations. This apparent software defect causes POE ports to randomly fault after 8 to 12 hours and the powered device will go offline. Cisco does not have a resolution, workaround, or timeline to address this situation.

 

Please help Cisco prioritize this issue by opening a TAC case if you are experiencing this issue. Additionally, please escalate this for additional visibility through your sales channels.

 

All the best!

7 people had this problem
Labels:
0 Helpful
54 Replies 54

Go to solution
cyberconsultants
Level 1
Level 1
In response to d0nni3q84

‎03-03-2022 02:05 PM

You have an 8-port PoE switch on which PoE is disabled on all PoE ports and to which no PDs have ever been connected on any port at any point during its current uptime, and all ports just randomly faulted? Are the ports faulted because you've manually set the power limit to 0W, as some ostensible attempt at hardening them?

The timestamps of your posts in the other thread reveal that you changed power limit mode and disabled inrush test prior to opening your TAC case.

Post the startup-config for that switch, cycle it, and then re-post that output.

0 Helpful

Go to solution
d0nni3q84
Level 1
Level 1
In response to cyberconsultants

‎03-03-2022 02:35 PM - edited ‎03-03-2022 02:39 PM

@cyberconsultants wrote:

You have an 8-port PoE switch on which PoE is disabled on all PoE ports and to which no PDs have ever been connected on any port at any point during its current uptime, and all ports just randomly faulted?

Yes, that is correct.

 

@cyberconsultants wrote:

Are the ports faulted because you've manually set the power limit to 0W, as some ostensible attempt at hardening them?

No, the ports fault regardless of the power limit or even if it is cleared. Your postulate that I am trying to harden the switch is grossly incorrect. Even with the default POE settings, the strange and erratic behavior takes place.

 

SW-3#show clock

22:33:07 UTC Mar 3 2022
Time from SNTP is enabled
Operational Time Source: SNTP (last synchronized 00:13:51 ago)
Time from Browser is disabled


SW-3#show power inline

Power-limit mode: Port based
Usage threshold: 95%
Trap: Disable
Legacy Mode: Enable
Inrush Test: Disable
Guard-Band Value: Static (2W)

Unit Module         Nominal   Consumed     Temp     SW            PSE chipset
                    Power (W) Power (W)    (C)      Version       HW Revision
---- -------------- --------- ------------ -------- ------------- -----------------
1    CBS350-8P-E-2G 60        0 (0%)       47       0.2.0.17       TPS2388 - 0x40C4


Interface    Admin       Oper         Power (W)     Class     Device     Priority
---------- ---------- ----------- ----------------- ----- -------------- --------
gi1        Never      Fault       0.0               -                    low
gi2        Never      Fault       0.0               -                    low
gi3        Never      Fault       0.0               -                    low
gi4        Never      Fault       0.0               -                    low
gi5        Never      Fault       0.0               -                    low
gi6        Never      Fault       0.0               -                    low
gi7        Never      Fault       0.0               -                    low
gi8        Never      Fault       0.0               -                    low


SW-3#configure
SW-3(config)#interface range Gi1-8
SW-3(config-if-range)#power inline auto
SW-3(config-if-range)#power inline never
SW-3(config-if-range)#end


SW-3#show power inline

Power-limit mode: Port based
Usage threshold: 95%
Trap: Disable
Legacy Mode: Enable
Inrush Test: Disable
Guard-Band Value: Static (2W)

Unit Module         Nominal   Consumed     Temp     SW            PSE chipset
                    Power (W) Power (W)    (C)      Version       HW Revision
---- -------------- --------- ------------ -------- ------------- -----------------
1    CBS350-8P-E-2G 60        0 (0%)       47       0.2.0.17       TPS2388 - 0x40C4


Interface    Admin       Oper         Power (W)     Class     Device     Priority
---------- ---------- ----------- ----------------- ----- -------------- --------
gi1        Never      Off         0.0               -                    low
gi2        Never      Off         0.0               -                    low
gi3        Never      Off         0.0               -                    low
gi4        Never      Off         0.0               -                    low
gi5        Never      Off         0.0               -                    low
gi6        Never      Off         0.0               -                    low
gi7        Never      Off         0.0               -                    low
gi8        Never      Off         0.0               -                    low


SW-3#show power inline GigabitEthernet 1

Interface    Admin       Oper         Power (W)     Class     Device     Priority
---------- ---------- ----------- ----------------- ----- -------------- --------
gi1        Never      Off         0.0               -                    low


Port Status:               Port is off. User setting
Port standard:             802.3AT
Admin power limit:         0.0 watts
Time range:
Max Power Allocation:      0.0 watts
Spare pair:                Disabled
Negotiated power:          0.0 watts (None)
Current (mA):              0
Voltage(V):                0.0
Overload Counter:          68
Short Counter:             6
Denied Counter:            0
Absent Counter:            0
Invalid Signature Counter: 0


SW-3#show clock
22:34:07 UTC Mar 3 2022
Time from SNTP is enabled
Operational Time Source: SNTP (last synchronized 00:14:51 ago)
Time from Browser is disabled
SW-3#

 

0 Helpful

Go to solution
cyberconsultants
Level 1
Level 1
In response to d0nni3q84

‎03-03-2022 03:54 PM

Hardening, manual power budgeting—you're right, the reason any port would have its power limit manually set at 0W (and which already has its PoE mode administratively disabled) is beyond unclear and I'm speculating.

Set limit mode to class, or at least set all individual port power limits back to default, re-enable inrush test, commit/copy to su, post su config, cycle the switch (not just PoE mode on all PoE ports), and then re-post show power inline and a show power inline [interface]. If you can show a random fault at that point, preferably in continuous output like you've done above, then other configuration settings aside from limit mode and/or port power limit or, I suppose, software bugs might be considered.

You originally describe the issue as:

@d0nni3q84 wrote:
This apparent software defect causes POE ports to randomly fault after 8 to 12 hours and the powered device will go offline.

Are we talking faults with PDs connected, or faults completely at random? If completely at random, then take the steps above. Or try a factory reset for that matter. Faults with PDs connected versus completely random faults are obviously entirely different troubleshoots.

0 Helpful

Go to solution
d0nni3q84
Level 1
Level 1
In response to cyberconsultants

‎03-04-2022 06:27 AM

@cyberconsultants wrote:

or, I suppose, software bugs might be considered.

Thanks so much @cyberconsultants, for your efforts to bring a notion of doubt to this situation under the guise of assistance. Nonetheless, I hope the below output provides proof beyond a reasonable doubt that the aforementioned erratic behavior exists. I'm using the power inline never command on the individual ports is for two reasons: (1) to workaround known issue CSCvu81814 and (2) the default admin power limit is 30W which oversubscribes the switch's POE capability.

 

@cyberconsultants wrote:

Are we talking faults with PDs connected, or faults completely at random?

Both, see below.

 

SW-3#show clock | include UTC
13:59:54 UTC Mar 4 2022


SW-3#show system | include Up
System Up Time (days,hour:min:sec):       00,09:02:54


SW-3#show power inline
Power-limit mode: Class based
Usage threshold: 95%
Trap: Disable
Legacy Mode: Enable
Inrush Test: Enable
Guard-Band Value: Static (2W)

Unit Module         Nominal   Consumed     Temp     SW            PSE chipset
                    Power (W) Power (W)    (C)      Version       HW Revision
---- -------------- --------- ------------ -------- ------------- -----------------
1    CBS350-8P-E-2G 60        0 (0%)       46       0.2.0.17       TPS2388 - 0x40C4


Interface    Admin       Oper         Power (W)     Class     Device     Priority
---------- ---------- ----------- ----------------- ----- -------------- --------
gi1        Never      Off         0.0               -                    low
gi2        Auto       Searching   0.0               -                    low
gi3        Never      Off         0.0               -                    low
gi4        Never      Off         0.0               -                    low
gi5        Never      Off         0.0               -                    low
gi6        Never      Fault       0.0               -                    low
gi7        Never      Off         0.0               -                    low
gi8        Never      Fault       0.0               -                    low


SW-3#show power inline GigabitEthernet 6
Interface    Admin       Oper         Power (W)     Class     Device     Priority
---------- ---------- ----------- ----------------- ----- -------------- --------
gi6        Never      Fault       0.0               -                    low


Port Status:               Port is off. Overload state
Port standard:             802.3AT
Admin power limit (for port power-limit mode): 30.0 watts
Time range:
Max Power Allocation:      30.0 watts
Spare pair:                Disabled
Negotiated power:          0.0 watts (None)
Current (mA):              0
Voltage(V):                0.0
Overload Counter:          24
Short Counter:             1
Denied Counter:            0
Absent Counter:            2
Invalid Signature Counter: 0


SW-3#show interfaces status GigabitEthernet 6
                                             Flow Link          Back   Mdix
Port     Type         Duplex  Speed Neg      ctrl State       Pressure Mode
-------- ------------ ------  ----- -------- ---- ----------- -------- -------
gi6      1G-Copper      --      --     --     --  Down           --     --


SW-3#show power inline GigabitEthernet 8

Interface    Admin       Oper         Power (W)     Class     Device     Priority
---------- ---------- ----------- ----------------- ----- -------------- --------
gi8        Never      Fault       0.0               -                    low


Port Status:               Port is off. Overload state
Port standard:             802.3AT
Admin power limit (for port power-limit mode): 30.0 watts
Time range:
Max Power Allocation:      30.0 watts
Spare pair:                Disabled
Negotiated power:          0.0 watts (None)
Current (mA):              0
Voltage(V):                0.0
Overload Counter:          35
Short Counter:             0
Denied Counter:            0
Absent Counter:            4
Invalid Signature Counter: 0


SW-3#show interfaces status GigabitEthernet 8
                                             Flow Link          Back   Mdix
Port     Type         Duplex  Speed Neg      ctrl State       Pressure Mode
-------- ------------ ------  ----- -------- ---- ----------- -------- -------
gi8      1G-Copper      --      --     --     --  Down           --     --


SW-3#show startup-config | exclude ^username
config-file-header
SW-3
v3.1.1.7 / RCBS3.1_930_871_081
CLI v1.0
file SSD indicator encrypted
@
ssd-control-start
ssd config
ssd file passphrase control unrestricted
no ssd file integrity control
ssd-control-end cb0a3fdb1f3a1af4e4430033719968c0
!
!
unit-type-control-start
unit-type unit 1 network gi uplink none
unit-type-control-end
!
spanning-tree priority 8192
port jumbo-frame
vlan database
vlan 1010,1013-1014
exit
voice vlan oui-table add 0001e3 Siemens_AG_phone
voice vlan oui-table add 00036b Cisco_phone
voice vlan oui-table add 00096e Avaya
voice vlan oui-table add 000fe2 H3C_Aolynk
voice vlan oui-table add 0060b9 Philips_and_NEC_AG_phone
voice vlan oui-table add 00d01e Pingtel_phone
voice vlan oui-table add 00e075 Polycom/Veritel_phone
voice vlan oui-table add 00e0bb 3Com_phone
no eee enable
hostname SW-3
no logging console
passwords aging 0
ip ssh server
snmp-server location Family
snmp-server contact noc@quindar7.net
no ip http server
sntp server 10.0.10.1 poll
no sntp server pool.ntp.org
no sntp server time-a.timefreq.bldrdoc.gov
no sntp server time-b.timefreq.bldrdoc.gov
no sntp server time-c.timefreq.bldrdoc.gov
no sntp server time-pnp.cisco.com
!
interface vlan 1
 name Default
 no ip address dhcp
 shutdown
!
interface vlan 1010
 name Management
 ip address 10.0.10.13 255.255.255.128
!
interface vlan 1013
 name IoT
!
interface vlan 1014
 name Media
!
interface GigabitEthernet1
 description UPS-3
 switchport access vlan 1010
 switchport trunk native vlan none
 switchport trunk allowed vlan none
 power inline never
!
interface GigabitEthernet2
 description IOT-ZWAVE
 switchport access vlan 1013
 switchport trunk native vlan none
 switchport trunk allowed vlan none
!
interface GigabitEthernet3
 speed 10
 no negotiation
 description IOT-POOL
 switchport access vlan 1013
 switchport trunk native vlan none
 switchport trunk allowed vlan none
 power inline never
!
interface GigabitEthernet4
 description IOT-HVAC
 switchport access vlan 1013
 switchport trunk native vlan none
 switchport trunk allowed vlan none
 power inline never
!
interface GigabitEthernet5
 description SND-FAMILY
 switchport access vlan 1014
 switchport trunk native vlan none
 switchport trunk allowed vlan none
 power inline never
!
interface GigabitEthernet6
 description BLU-FAMILY
 switchport access vlan 1014
 switchport trunk native vlan none
 switchport trunk allowed vlan none
 power inline never
!
interface GigabitEthernet7
 description ATV-FAMILY
 switchport access vlan 1014
 switchport trunk native vlan none
 switchport trunk allowed vlan none
 power inline never
!
interface GigabitEthernet8
 shutdown
 description [UNUSED]
 switchport access vlan none
 switchport trunk native vlan none
 switchport trunk allowed vlan none
 power inline never
!
interface GigabitEthernet9
 shutdown
 description [UNUSED]
 switchport access vlan none
 switchport trunk native vlan none
 switchport trunk allowed vlan none
!
interface GigabitEthernet10
 description SW-1.Gi14
 ip dhcp snooping trust
 switchport mode trunk
 switchport access vlan none
 switchport trunk native vlan none
 switchport trunk allowed vlan 1010,1013-1014
!
interface Port-Channel1
 shutdown
 description [UNUSED]
 switchport access vlan none
 switchport trunk native vlan none
 switchport trunk allowed vlan none
!
interface Port-Channel2
 shutdown
 description [UNUSED]
 switchport access vlan none
 switchport trunk native vlan none
 switchport trunk allowed vlan none
!
interface Port-Channel3
 shutdown
 description [UNUSED]
 switchport access vlan none
 switchport trunk native vlan none
 switchport trunk allowed vlan none
!
interface Port-Channel4
 shutdown
 description [UNUSED]
 switchport access vlan none
 switchport trunk native vlan none
 switchport trunk allowed vlan none
!
interface Port-Channel5
 shutdown
 description [UNUSED]
 switchport access vlan none
 switchport trunk native vlan none
 switchport trunk allowed vlan none
!
interface Port-Channel6
 shutdown
 description [UNUSED]
 switchport access vlan none
 switchport trunk native vlan none
 switchport trunk allowed vlan none
!
interface Port-Channel7
 shutdown
 description [UNUSED]
 switchport access vlan none
 switchport trunk native vlan none
 switchport trunk allowed vlan none
!
interface Port-Channel8
 shutdown
 description [UNUSED]
 switchport access vlan none
 switchport trunk native vlan none
 switchport trunk allowed vlan none
!
exit
ip dhcp snooping
ip dhcp snooping vlan 1010
ip dhcp snooping vlan 1013
ip dhcp snooping vlan 1014
ip default-gateway 10.0.10.1

 

0 Helpful

Go to solution
cyberconsultants
Level 1
Level 1
In response to d0nni3q84

‎03-04-2022 06:47 AM

For posterity: the only PoE issue ("defect"?) OP seems to have presented is with Ubiquiti PDs.

0 Helpful

Go to solution
d0nni3q84
Level 1
Level 1
In response to cyberconsultants

‎03-04-2022 07:53 PM

@cyberconsultants wrote:

For posterity: the only PoE issue ("defect"?) OP seems to have presented is with Ubiquiti PDs.

This is not true. I have also presented a case where a PoE port with no device connected at all goes into the Port is off. Overload state status.

0 Helpful

Go to solution
CoreyP319
Cisco Employee
Cisco Employee

‎03-03-2022 03:56 PM

Hello d0nni3q84,

 

Thank you for raising this issue. I've not experienced the described issue with the PoE enabled CBS350 running in my home network. I've shared your post with the team and they are in turn sharing it internally to increase awareness while we investigate further. We'll provide more details or a workaround should they become available.

 

If anyone else is impacted, please open a case with TAC. It does help raise awareness on issues.

0 Helpful

Go to solution
d0nni3q84
Level 1
Level 1
In response to CoreyP319

‎03-03-2022 05:23 PM

Hello @CoreyP319,

 

Please share the exact switch model and firmware version that you have running in your home network where you do not observe this behavior.

 

Thank you!

0 Helpful

Go to solution
jimstewart77
Level 1
Level 1

‎03-04-2022 02:57 AM

I have seen ports on mine that have PoE live but a non-PD device (computer) connected show PoE fault.  The port however continues to work perfectly and the port itself isn't in fault, just the PoE.

Ports with PoE disabled and plugged into a non-PD device show PoE disabled.

I didn't give much thought to that fault state as it just seems to be a reporting oddity.

0 Helpful

Go to solution
kolping-mw
Level 1
Level 1
In response to jimstewart77

‎03-04-2022 04:25 AM

Same behavior here. Since the last firmware our Lancom devices are working, before there was definitely a problem with PoE.

0 Helpful

Go to solution
d0nni3q84
Level 1
Level 1
In response to jimstewart77

‎03-04-2022 06:06 AM

@jimstewart77 wrote:

I have seen ports on mine that have PoE live but a non-PD device (computer) connected show PoE fault.  The port however continues to work perfectly and the port itself isn't in fault, just the PoE.

Yes, that's a known issue and benign, where the status is Port is off. Short condition and the Short Counter continuously increments. I'm not worried about the scenario, since as you said, the port sill operates.

0 Helpful

Go to solution
d0nni3q84
Level 1
Level 1

‎03-21-2022 08:19 PM

During beta testing with Cisco TAC, I can confirm this issue is resolved in Software version 3.1.1.10 and MCU version 0xF0. Unfortunately, I do not have a release timeline available; however, I encourage you to reach out to Cisco TAC if you are experiencing this issue.

Go to solution
Klauzz
Level 1
Level 1

‎05-01-2022 10:07 AM

I am using a CBS350-8P-E-2G. At the moment just one POE device is connected. It's an Aruba AP22 Access Point.

 

I am also facing the Overload issue:

 

 

Interface    Admin       Oper         Power (W)     Class     Device     Priority
---------- ---------- ----------- ----------------- ----- -------------- --------
gi1        Never      Fault       0.0               0                    low


Port Status:               Port is off. Overload state
Port standard:             802.3AT
Admin power limit:         30.0 watts
Time range:
Max Power Allocation:      30.0 watts
Spare pair:                Disabled
Negotiated power:          0.0 watts (None)
Current (mA):              0
Voltage(V):                0.0
Overload Counter:          6
Short Counter:             0
Denied Counter:            1
Absent Counter:            0
Invalid Signature Counter: 0

 

 

The statistic looks like this:

 
Cisco.jpg

 

Is the denied counter also part of the known issue?

 

Thanks

 

Michael

0 Helpful

Go to solution
d0nni3q84
Level 1
Level 1
In response to Klauzz

‎05-02-2022 10:50 PM

I personally did not observe the Denied counter incrementing; however, it does mean that the switch did not provide power to the port since the requested power exceeded a limit. What is the output of the following commands?

show power inline GigabitEthernet 8
show run interface GigabitEthernet 8
0 Helpful

Go to solution
Klauzz
Level 1
Level 1
In response to d0nni3q84

‎05-04-2022 01:11 AM

Thanks for your reply. The output looks like this:

 

AzCiscoSwitch#show power inline GigabitEthernet 8

Interface    Admin       Oper         Power (W)     Class     Device     Priority
---------- ---------- ----------- ----------------- ----- -------------- --------
gi8        Auto       On          5.641 (16.5)      3                    high


Port Status:               Port is on. Valid resistor detected
Port standard:             802.3AT
Admin power limit:         30.0 watts
Time range:
Link partner standard:     802.3AF
Max Power Allocation:      30.0 watts
Spare pair:                Disabled
Negotiated power:          16.500 watts (LLDP)
Current (mA):              107
Voltage(V):                53.0
Overload Counter:          17
Short Counter:             0
Denied Counter:            268
Absent Counter:            12
Invalid Signature Counter: 0
AzCiscoSwitch#show run interface GigabitEthernet 8
interface GigabitEthernet8
 description GzAccessPoint
 switchport mode trunk
 switchport access vlan 10
 power inline priority high
!

Thanks

 

Michael

0 Helpful
Customers Also Viewed These Support Documents