SG300-28P - POE not correctly supported on all ports - possible firmware or hardware issue

[Cinemaffect]

So, I spent some time this weekend troubleshooting the issues I've had  with the new SG300-28P switch and POE to many of my devices in the  office.  As a recap, I cannot utilize all of the 24 POE ports on the switch  for POE purposes.  Really only every other port [with a few odd  combinations thrown in between]. In addition, the SG300-28P switch, on occasion, is sending POE to non-POE devices [e.g. my Ruckus Zone Director 1106].

Here are my POE devices [all 802.3 af-compliant]:

• 3 Ruckus 7982 access points
• 1 Pakedge access point
• 2 home-automation controllers
• 2 Polycom voip phones

I called Cisco support several times in regards to this problem, and they figured it was a hardware issue - a faulty switch.  So, Cisco sent me a replacement SG300-28P, which I  hooked up today.  The exact problem still occurs.  Default configuration  [fresh out of the box].  No way I can land, for example, the 3 Ruckus  7982 AP's on ports 1, 2, and 3 [or ports 1,13, and 2].  I have to put  them on ports 1, 3, and 5 in order for them to power up.  In addition, I  can't plug any other POE devices on the ports either between or below  them.   I had to skip another port bay.  This is very odd behavior!!   Two Cisco SG300-28P's in a row with the same problem.

However, I also had one of the new Cisco SG300-10P switches in my  possession for a recent project of ours.  I decided to hook up the same  POE devices to this switch.  ALL POE devices were recognized and  worked!  No need to skip a port.  And it didn't matter what device was  plugged in first or not.  I am now convinced that it is either a  hardware issue [bad power supply/transformer?] inside all of the  SG300-28P switches, or a firmware issue. 

Both of the SG300-28P switches were running firmware 1.1.2 [the  latest on Cisco's website].  So, I decided to install an older firmware  version on the SG300-28P switch that I'm returning [installed 1.1.1.8].   Here's what I found out.  I could then plug 2 POE devices [e.g. two  Ruckus AP's] in adjacent horizontal ports, but not three in a row.  In  addition, not all adjacent ports.  It's funky. For example, I could plug  an access point in ports 20 and 21, but not in 21 and 22.  No rhyme or  reason in how it worked.  And I still couldn't plug an access point in  adjacent vertical ports [e.g. ports 1 and 13].  BUT...

It's interesting that the same exact switch that would not initially  allow 2 horizontally-adjacent POE ports to be utilized WOULD allow 2  horizontally-adjacent POE ports to be utilized when running a different  firmware version.   It's also interesting to note that when plugged into  a "non-working" POE  port, the SG300-28P would actually make a small whining noise.  Very  subtle noise; I could hear it when approx. 1ft away from the switch.   The noise was not noticeable when ports were skipped [and POE actually  worked].  Therefore, I believe that Cisco has some SG300-28P firmware  bugs [at least in the last two versions of firmware] that is not truly  allowing all 24 ports to utilize POE correctly.  This problem does not  exist with the SG300-10P switch.

I'm really interested to hear what Cisco's reply and findings on this  matter would be.  And would welcome a reply from one of their senior  support team members/managers who could actually experiment with this,  too.   In addition, I'd like to know when they think a solution could be  created if it's firmware-related.  If hardware-related, I don't think  I'll be recommending any 28P switches in our projects.  Perhaps just the  regular SG300-28 with a separate SG300-10P.  It's a shame because the  SG300-28P is more of a bargain when compared to the two separate  components.

[fturner124]

I've been having the same problem with SG300-28P and several ZF7982 APs for a while now.  May have it figured out, but would like someone else to confirm. 

Try setting "Administrative Power Allocation (mW)" on unused ports to 0.  The AP that wouldn't boot up, immediately came online after changing this setting.

I had also already changed the priority to critical on active ports, and disabled POE on all inactive ports.  These changes didn't resolve the issue, but may have been part of the solution.

[DARRELL MILAM]

I also have been having this issue but with Cisco 1142 access points....changing the Administrative power allocation DID NOT fix my problem.  Does anyone have any other suggestions.  I have also updated the controller and switch firmware. Set ports to critical, disabled Poe on all unused ports. Moved all connections to every other connection meaning I'm only using less than 12 connections per switch even though the switch shows it has plenty of poer left.  I have also noticed the sometimes the power is on the access point but it will not connect to the wireless controller,  but the big issue is the switch stops powering the ap.  rebooting the switch will sometimes bring them back on but then drop power to another port and so me time it makes no difference in the port that is powered off. To get it up you can sometimes enable the Poe on a different port then move the connection and it will come back on. After you get all the ports operating a few hours later and sometimes days it will drop another.  I can not see a pattern or anything in the switch log or controller that corresponds.  Any suggestions other than replacing

[pcscontrols]

My problem sounded exactly like what you are seeing and it went COMPLETELY away when I switched to UNSHEILDED cables.  Inspect the RJ45 connector jack at the cable end you are plugging into the Cisco, if it has a metal jacket on/around it my problem was fixed when I removed the metal shield/jacket.

[DARRELL MILAM]

Thank you for your reply randy

But we are not using shielded cables

[Nagaraja Thanthry]

Hello Darrell,

Can you please reach out to SBSC and open a Serivce Request in this regard? Please ask the SBSC Engineer to escalate the case to the next level so one of our Escalation Engineers can take a look at the problem.

Thanks,

Nagaraja

[uschhs-cisco]

Getting on this thread to validate that this problem persists. We are experiencing the same symptoms. I've would up running power supplies and extension cords from wall outlets to power our Ruckus 7982 access points as the PoE, even when distributed across transformers has proven unreliable occassionally causing the access point to restart.

Not good for continuity. Worked with Ruckus for several weeks trying to update their firmware to be less sensistive to the fluctuating power to no avail.Love to hear from Cisco on a firmware update to address this obvious, and long standing issue.

[Nagaraja Thanthry]

Hello Andrew,

When you say that the connection is not reliable, are you seeing any port flaps? Do you have EEE turned on on the switch? If yes, can you turn off EEE and see if that helps? If you are seeing issues even after turning off EEE, please reach out to SBSC and open a Service Request. One of our Engineers will be happy to take a look at the settings and see if we can find the root cause.

Thanks,

Nagaraja

[lokibjensen]

I too am jumping on this thread to confirm the same issue with the 28P's and Ruckus 7982's. My case number is: 625315271

I will ask the tech if he's using shielded cables and I will try setting the Administrative power on unused ports to 0. Anyone know off-hand what the command for that is?

[lokibjensen]

Quick update, the command to set adminitrative power to 0 is "power inline limit 0".

I applied this to most ports, except for one's with AP's and this did not work for me.

[lokibjensen]

Another update related to a tip above, we are not using shielded cable and still experiencing the issue. Then there's also this response from support. Is this some sort of joke????? CAT3???

"The issue you are facing is a know bug with the SG300-28P switch. I have reached out to the developers regarding your issue and awaiting their response for what should be out next step. The workaround recommended for this issue is to use Cat3 cable on site.

I will keep you posted as soon as I hear back from the developers."

If Cisco knows this is an issue then why is it being sold? It is supposed to be a PoE switch with 24 PoE ports. Obviously there is a limit to how much power per port but we are not even coming close to what the limits are supposed to be. At this point Cisco should honor the sale and send an SG300-28MP as replacement if that will fix the issue. There is NO WAY I'm putting CAT3 in and besides the building is already pre-wired.

Is anybody monitoring these posts?

[DARRELL MILAM]

I'm having the same problem the only way we could get it to partially work was to spread the Poe used ports around.  Please look at this thread https://supportforums.cisco.com/message/3731819#3731819

[Tom Watts]

Hi lokibjensen.

Another update related to a tip  above, we are not using shielded cable and still experiencing the issue.  Then there's also this response from support. Is this some sort of  joke????? CAT3???

"The issue you are facing is a know bug with the  SG300-28P switch. I have reached out to the developers regarding your  issue and awaiting their response for what should be out next step. The  workaround recommended for this issue is to use Cat3 cable on site.

I will keep you posted as soon as I hear back from the developers."

No, it is not some sort of joke. The reason Cat3 should resolve the problem is because it was the relevant wire-type for legacy POE. Cat3 has less twists in the wire pairs. Additionally Cat3 uses 2 wire pairs while Cat5 uses 4 wire pairs.

The technical matter of Cat3 is it can handle 16mhz while Cat5 is like 100mhz. Although it is not a favorable solution due to speed concerns (10mbit vs 100+ mbit).

Most of the problem comes from the concept of integrating support for legacy POE.

If  Cisco knows this is an issue then why is it being sold? It is supposed  to be a PoE switch with 24 PoE ports. Obviously there is a limit to how  much power per port but we are not even coming close to what the limits  are supposed to be. At this point Cisco should honor the sale and send  an SG300-28MP as replacement if that will fix the issue. There is NO WAY  I'm putting CAT3 in and besides the building is already pre-wired.

You have an invalid arugment that if there is an issue why is it sold. You may be frustrated which is agreeable and understandable but if your contention if no vendor has a bug or defect, you're in the wrong business.  This device is advertised for every port POE capable and it is thoroughly documented since it's inception of how much it supports per port. Quite frankly it does work. This is an isolated instance with a very specific symptom. The issue seems to be there is not a valid signature detected. It is very questionable if this has to do with the cross-implementation of 802.3at support with the AP. As you may notice most/any devices that are not 802.3at compatible do not have a problem. Since 802.3at is not standardized (or barely standardized, i'd have to double check) the implementations still are likely sloppy and incomplete.

Is anybody monitoring these posts?

Yes. The best thing to do is open a ticket with the support center and add more leverage to the cause. The support forums are provided as a courtesy and are complimentary. This is not the official channel of communication or documentation.

-Tom
Please mark answered for helpful posts

[lokibjensen]

Sorry if I offended you.  However, asking somebody to use CAT3 as a workaround for a device on a Gigabit switch, in 2013, is quite frankly unreasonable.  I understand why CAT3 will work but that doesn't mean it should ever be used in any modern deployment unless it is satisfactory to the client to have an outdated network when they paid for a new one.  I know my clients would fire me in a second if I even approached them with such a solution.

As for the switch working or not, it apparently cannot handle what it is advertised to do in this instance and the instances mentioned above in this thread.  I believe it's advertised as being able to handle up to just over 15W per port and not to exceed something near 180W (off the top of my head I can't remember the exact numbers).  As the switch obviously cannot do that in the instances on this thread (or even come close), and the issue has been known for about a year now, I find it alarming that Cisco would still sell the switch as advertised.

The reason I posted the "workaround" here was to provide the courtesy to the fellow members of this community so that they would know where we stand on this issue, especially as I had not seen a reply to anybody's comments or requests since January 29th, about two months ago.

Thank you.

[Tom Watts]

Hi Lokibjensen, it's really understandable the confusion and frustration. I was the original TAC engineer that sent this problem to the tier 2 team. From the best of my recollection there wasn't really anything to see. There was a header 0 for the POE and then the signature counter incremented.

I do not know what Cisco/Tier 2/Development has learned since the initial reports of this. My suspicion is it has something to do with how the Ruckus 7982 may set up the parameter for the POE negotiation. Since it increments an invalid signature I believe it is sending too much juice on the line and the switch doesn't like it. Hence the Cat3 wire can't support the juice therefore it works.

I do not feel it is a POE leak. I do feel it is a flawed implementation of POE. It's just a matter of by whom and why. Since all of the SB switches work very well with every other 802.3af compliant devices, I believe it has something to do with Ruckus' integration of 802.3at and interop with 802.3af when the AP realized the 802.3at is not present.I also think Ruckus may have a limitation to using a "phantom power" only. Another concept is mid-span. Gigabit switch uses all 4 pairs. Midspan will short the 4-5, 7-8 pairs.So if you're using mid-spans stick with 10/100 (How would your client respond to that?)

Ruckus does have some issues with product such as Adtran Netvanta 1234 switch where it does not power correctly. Another switch is the HP procurve 2520 has some issues with the Ruckus AP's.

-Tom
Please mark answered for helpful posts

[lokibjensen]

Please forgive my ignorance....what do you mean by, "Another concept is mid-span. Gigabit switch uses all 4 pairs. Midspan will short the 4-5, 7-8 pairs.So if you're using mid-spans stick with 10/100 (How would your client respond to that?)"