SX350X-24F problem

[charalampos Perakis]

I have a problem with port flapping on connections from my SMC server to my Cisco switch.

Server board is a Supermicro X11SDW-4C-TP13F+ which has 4x onboard x722 10g SFP+ LAN ports.

Switch is a SX350X-24F, and I am using SFPP-PC03 DAC copper twinax passive cables bought from fs.com for the connection.

Server is running windows 2012 R2 with the latest updates installed, and I have installed the latest Intel LAN

driver (25.5). The onboard cards have been flashed with the latest firmware that SMC provided me ().

The Switch is running latest firmware (2.5.5.57).

On top of this I am experiencing unexpected low throughput over these links.

Please advise on what I should do...

The problem is that when I connect the 10G SFP+ lan ports to the switch, I get port flapping and naturally the port

goes down. I have tried many settings, all of which have been hit and miss.

Flow control (enable, disable, auto-negotiate).

Link speed    (1g, 10g, auto-negotiate).

EEE (green ethernet) (Enable, disable) on selected ports.

SR_IOV (enable, disable also from bios).

The above settings have been tried and tested on both the switch and lan cards. None of the settings seems to be

decisive as to whether it will enable the link to connect or not.

For example I have had links connect with flow control enabled as well as disabled.

A Supermicro SYS-1029U-E1CRTP2 (running esxi 7.0) which has intel's x710-am2 lan cards has no problem whatsoever with link flapping.

(same switch, same cables)...

Right now when rebooting the server 75% of the time links will be disabled because of flapping and I have to manually

reenable them from the switch web gui. As I said above, whether this will work is a hit and miss situation.

[marce1000]

            - Can you post examples of this problem as observed  from the logs ?

  M.

[charalampos Perakis]

Hi

The cable was connected to port 20 (which was down), I unplugged it and plugged it into port 19.

First with flow control enabled on both the port and the LAN and EEE disabled on the port. .

Second time with flow control disabled on both the port and LAN and EEE disabled on the port.

Third time with flow control disabled on both the port and LAN and EEE enabled on the port.

Logs

2147347941 2020-Nov-30 22:28:08 Warning %LINK-W-PORT_SUSPENDED: Port te1/0/19 suspended by link-flapping
2147347942 2020-Nov-30 22:28:08 Warning %LINK-W-Down: te1/0/19
2147347943 2020-Nov-30 22:28:00 Warning %STP-W-PORTSTATUS: te1/0/19: STP status Forwarding
2147347944 2020-Nov-30 22:27:58 Informational %LINK-I-Up: te1/0/19
2147347945 2020-Nov-30 22:27:33 Informational %PNPAGENT-I-PNPSRVRDETECT: PnP Server devicehelper.cisco.com was detected
2147347946 2020-Nov-30 22:27:19 Warning %LINK-W-PORT_SUSPENDED: Port te1/0/19 suspended by link-flapping
2147347947 2020-Nov-30 22:27:18 Warning %LINK-W-Down: te1/0/19
2147347948 2020-Nov-30 22:27:12 Warning %STP-W-PORTSTATUS: te1/0/19: STP status Forwarding
2147347949 2020-Nov-30 22:27:10 Informational %LINK-I-Up: te1/0/19
2147347950 2020-Nov-30 22:27:10 Warning %LINK-W-Down: te1/0/19
2147347951 2020-Nov-30 22:27:09 Warning %STP-W-PORTSTATUS: te1/0/19: STP status Forwarding
2147347952 2020-Nov-30 22:27:07 Informational %LINK-I-Up: te1/0/19
2147347953 2020-Nov-30 22:26:19 Warning %LINK-W-PORT_SUSPENDED: Port te1/0/19 suspended by link-flapping
2147347954 2020-Nov-30 22:26:19 Warning %LINK-W-Down: te1/0/19
2147347955 2020-Nov-30 22:26:09 Warning %STP-W-PORTSTATUS: te1/0/19: STP status Forwarding
2147347956 2020-Nov-30 22:26:08 Informational %NSFP-I-SFPGibicDetected: te1/0/19 SFP port is present, module type - 10G BASE-CX1
2147347957 2020-Nov-30 22:26:07 Informational %LINK-I-Up: te1/0/19
2147347958 2020-Nov-30 22:26:04 Informational %NSFP-I-SFPGibicRemoved: te1/0/20 SFP port is not present
2147347959 2020-Nov-30 22:24:19 Informational %AAA-I-CONNECT: New http connection for user admin, source 192.168.10.43 destination 192.168.10.250 ACCEPTED
2147347960 2020-Nov-30 22:16:13 Informational %PNPAGENT-I-PNPSRVRDETECT: PnP Server devicehelper.cisco.com was detected
2147347961 2020-Nov-30 22:14:03 Informational %PNPAGENT-I-PNPSRVRDETECT: PnP Server devicehelper.cisco.com was detected
2147347962 2020-Nov-30 22:08:19 Informational %PNPAGENT-I-BYEREQ: Bye PnP Request
2147347963 2020-Nov-30 22:08:17 Informational %PNPAGENT-I-RESPSUCCESS: PnP Response Success
2147347964 2020-Nov-30 22:08:17 Informational %PNPAGENT-I-DEVINFO: Device Info PnP Request with parameter all
2147347965 2020-Nov-30 22:08:15 Informational %PNPAGENT-I-PNPSRVRDETECT: PnP Server devicehelper.cisco.com was detected
2147347966 2020-Nov-30 22:06:05 Informational %PNPAGENT-I-RESPSUCCESS: PnP Response Success
2147347967 2020-Nov-30 22:06:03 Informational %PNPAGENT-I-PNPSRVRDETECT: PnP Server devicehelper.cisco.com was detected
2147347968 2020-Nov-30 22:03:52 Informational %PNPAGENT-I-RESPSUCCESS: PnP Response Success
2147347969 2020-Nov-30 22:03:52 Informational %PNPAGENT-I-BACKOFFCALLBACKAFTER: Backoff PnP Request with <callbackAfter> has received: 0 Hours 1 Minutes 0 Seconds
2147347970 2020-Nov-30 22:03:20 Informational %PNPAGENT-I-BYEREQ: Bye PnP Request
2147347971 2020-Nov-30 22:03:17 Informational %PNPAGENT-I-RESPSUCCESS: PnP Response Success
2147347972 2020-Nov-30 22:03:17 Informational %PNPAGENT-I-DEVINFO: Device Info PnP Request with parameter all
2147347973 2020-Nov-30 22:02:46 Informational %PNPAGENT-I-BYEREQ: Bye PnP Request
2147347974 2020-Nov-30 22:02:43 Informational %PNPAGENT-I-RESPSUCCESS: PnP Response Success
2147347975 2020-Nov-30 22:02:40 Informational %PNPAGENT-I-PNPSRVRDETECT: PnP Server devicehelper.cisco.com was detected
2147347976 2020-Nov-30 22:00:27 Informational %PNPAGENT-I-PNPSRVRDETECT: PnP Server devicehelper.cisco.com was detected
2147347977 2020-Nov-30 21:57:47 Informational %PNPAGENT-I-BYEREQ: Bye PnP Request
2147347978 2020-Nov-30 21:57:46 Informational %PNPAGENT-I-RESPSUCCESS: PnP Response Success
2147347979 2020-Nov-30 21:57:46 Informational %PNPAGENT-I-DEVINFO: Device Info PnP Request with parameter all
2147347980 2020-Nov-30 21:57:45 Informational %PNPAGENT-I-PNPSRVRDETECT: PnP Server devicehelper.cisco.com was detected
2147347981 2020-Nov-30 21:52:30 Informational %PNPAGENT-I-RESPSUCCESS: PnP Response Success
2147347982 2020-Nov-30 21:52:28 Informational %PNPAGENT-I-PNPSRVRDETECT: PnP Server devicehelper.cisco.com was detected
2147347983 2020-Nov-30 21:47:17 Informational %PNPAGENT-I-PNPSRVRDETECT: PnP Server devicehelper.cisco.com was detected
2147347984 2020-Nov-30 21:44:36 Informational %PNPAGENT-I-BYEREQ: Bye PnP Request
2147347985 2020-Nov-30 21:44:36 Informational %PNPAGENT-I-RESPSUCCESS: PnP Response Success
2147347986 2020-Nov-30 21:44:36 Informational %PNPAGENT-I-DEVINFO: Device Info PnP Request with parameter all
2147347987 2020-Nov-30 21:44:05 Informational %PNPAGENT-I-BYEREQ: Bye PnP Request
2147347988 2020-Nov-30 21:44:03 Informational %PNPAGENT-I-RESPSUCCESS: PnP Response Success
2147347989 2020-Nov-30 21:44:02 Informational %PNPAGENT-I-PNPSRVRDETECT: PnP Server devicehelper.cisco.com was detected
2147347990 2020-Nov-30 21:38:48 Informational %PNPAGENT-I-RESPSUCCESS: PnP Response Success

[charalampos Perakis]

this problem has been fixed by Cisco. Cisco ticket # CSCvu44725 but i don't have access

can someone help me...

[marce1000]

SX550X-12F: Port flap with Intel X722 LOM NICs on SFP ports, but not combo ports

CSCvu44725

Description

Symptom:
Some SFP ports may flap when connected to Intel X722 LOM. This does not happen when the X722 LOM is plugged into combo ports. Lowering the speed to 1000 may result in no change or the port may go down completely. Flaps observed on the node side may only intermittently register in the switch log (i.e., flapping is not always evident on the switch side).

Conditions:

Workaround:
Use combo port, if available.

Further Problem Description:

Customer Visible

Notifications

Save Bug

Open Support Case

Was the description about this Bug Helpful?

(0)

Details

Last Modified:

Dec 22,2020

Status:

Fixed

Severity:

3 Moderate

Product:

(1)

Cisco LAN Switch Software

Support Cases:

14

Known Affected Releases:

(2)

2.5.0.92

2.5.5.x

Known Fixed Releases:

(0)

Download software for  Cisco LAN Switch Software

Community Discussion on CSCvu44725 - Cisco Community

1

Discussion(s)

SX350X-24F problem

I have a problem with port flapping on connections from my SMC server to my Cisco switch. Server board is a Supermicro X11SDW-4C-TP13F+ which has 4x onboard x722 10g SFP+ LAN ports. Switch is a SX350X-24F, and I am using SFPP-PC03 DAC copper twinax passive cables bought from fs.com for the connection. � Server is running windows 2012 R2 with the latest updates installed, and I have installed the latest Intel LAN driver (25.5). The onboard cards have been flashed with the latest firmware that SMC ...

Author:charalampos Perakis | Posted:November 26, 2020 | Latest activity:December 23, 2020

[charalampos Perakis]

I tried connecting to the combo port in my sx350x-24f. This works without any port flapping.

Still my main problem is that the 4 combo ports in my sx350x-24f are being used to connect 4 clients through 10gbase-t rj45 cables. (one of the reasons I chose this switch in the first place).

So although this works, it is a work around NOT a solution, and it confirms that the problem lies with the Cisco Switch.

Is there going to be newer firmware to resolve this issue?

[alecj]

Same here.. if anyone has a direct link to this firmware patch that would be much appreciated

[apogeenet]

should not require a SmartNet contract.  Just sign in to Cisco.com, support, downloads, search for SX550X, and it is open for all to download. 

https://software.cisco.com/download/home/286319791/type/282463181/release/2.5.7.85

Good luck

[charalampos Perakis]

As a follow up to the problem I reported with port flapping between the SFP+ ports of X11SDW-4C-TP13F+ SFP+ and Cisco SX-350X-24F I have done some testing.

First off, In order to eliminate the possibility that the problem with port flapping is not caused by the cables,

I ordered 3 more different cables to test.

In particular

ML-DACS+3        MaxLink ML-DACS+3 SFP+ Direct Attach Cable, passive, DDM, 3m

XS+DA0003         MikroTik XS+DA0003 SFP/SFP+/SFP28 direct attach cable, 3m

S+DA0003           MikroTik S+DA0003 SFP+ 3m direct attach cable

All three exhibit the same behavior as the original cables I was using.  (SFPP-PC03 10G SFP+ Passive DAC twinax cables from www.fs.com)

On top of this I ordered a Mikrotik CSS610-8G-2S+IN to check if there would be a problem establishing a link to its SFP+ ports.

The onboard x722 have no problem whatsoever to connect without any port flapping to the 2 SFP+ ports of the Mikrotik switch.

I tried all the cables, they all connected without issues. (in all cases the link established was 10G).

To completely rule out the possibility that the cables where to blame, I also did loopback testing on both the X11SDW-4C-TP13F+ SFP+ ports as well as the cisco ports.

In more detail:

I did a loopback test on the Cisco switch, used both 0.5m as well as the 3m cables from FS.com. In both cases it would connect without any problems whatsoever (10g link).

Ι did a loopback test on the motherboard SFP+ LAN ports. 0.5m as well as the 3m cables. In both cases it would connect without any problems whatsoever (10g link).

Apparently the only way you can actually observe the problem I am experiencing, is to source a SX350X-24F and test connect it to the X11SDW-4C-TP13F+ SFP+ ports.

We cannot be sure if testing with another switch from the same line from CISCO (350X) will exhibit the same behavior.

As it is, I obviously cannot start buying more (quite expensive) switches to test.

As mentioned in my earlier correspondence, there seems to be other cases with x722 LANs SFP+ and link flapping on cisco switches.

Any advice solving this quite annoying issue?

[apogeenet]

Does disabling power save mode on the Intel adapter side resolve the issue?  On a Windows machine you would accomplish this by opening device manager -> find the network adapter -> right click and select properties -> Select Power Management -> Uncheck "Allow the computer to turn off the device to save power." On a Debian Linux OS, you'd usually find this in the /etc/default/tlp file.  Does setting speeds manually and disabling negotiation resolve the problems?  These are common issues with Intel link flap related symptoms.