11-26-2020 11:45 PM
I have a problem with port flapping on connections from my SMC server to my Cisco switch.
Server board is a Supermicro X11SDW-4C-TP13F+ which has 4x onboard x722 10g SFP+ LAN ports.
Switch is a SX350X-24F, and I am using SFPP-PC03 DAC copper twinax passive cables bought from fs.com for the connection.
Server is running windows 2012 R2 with the latest updates installed, and I have installed the latest Intel LAN
driver (25.5). The onboard cards have been flashed with the latest firmware that SMC provided me ().
The Switch is running latest firmware (2.5.5.57).
On top of this I am experiencing unexpected low throughput over these links.
Please advise on what I should do...
The problem is that when I connect the 10G SFP+ lan ports to the switch, I get port flapping and naturally the port
goes down. I have tried many settings, all of which have been hit and miss.
Flow control (enable, disable, auto-negotiate).
Link speed (1g, 10g, auto-negotiate).
EEE (green ethernet) (Enable, disable) on selected ports.
SR_IOV (enable, disable also from bios).
The above settings have been tried and tested on both the switch and lan cards. None of the settings seems to be
decisive as to whether it will enable the link to connect or not.
For example I have had links connect with flow control enabled as well as disabled.
A Supermicro SYS-1029U-E1CRTP2 (running esxi 7.0) which has intel's x710-am2 lan cards has no problem whatsoever with link flapping.
(same switch, same cables)...
Right now when rebooting the server 75% of the time links will be disabled because of flapping and I have to manually
reenable them from the switch web gui. As I said above, whether this will work is a hit and miss situation.
11-27-2020 12:52 AM
- Can you post examples of this problem as observed from the logs ?
M.
12-01-2020 09:41 AM
Hi
The cable was connected to port 20 (which was down), I unplugged it and plugged it into port 19.
First with flow control enabled on both the port and the LAN and EEE disabled on the port. .
Second time with flow control disabled on both the port and LAN and EEE disabled on the port.
Third time with flow control disabled on both the port and LAN and EEE enabled on the port.
Logs
2147347941 2020-Nov-30 22:28:08 Warning %LINK-W-PORT_SUSPENDED: Port te1/0/19 suspended by link-flapping
2147347942 2020-Nov-30 22:28:08 Warning %LINK-W-Down: te1/0/19
2147347943 2020-Nov-30 22:28:00 Warning %STP-W-PORTSTATUS: te1/0/19: STP status Forwarding
2147347944 2020-Nov-30 22:27:58 Informational %LINK-I-Up: te1/0/19
2147347945 2020-Nov-30 22:27:33 Informational %PNPAGENT-I-PNPSRVRDETECT: PnP Server devicehelper.cisco.com was detected
2147347946 2020-Nov-30 22:27:19 Warning %LINK-W-PORT_SUSPENDED: Port te1/0/19 suspended by link-flapping
2147347947 2020-Nov-30 22:27:18 Warning %LINK-W-Down: te1/0/19
2147347948 2020-Nov-30 22:27:12 Warning %STP-W-PORTSTATUS: te1/0/19: STP status Forwarding
2147347949 2020-Nov-30 22:27:10 Informational %LINK-I-Up: te1/0/19
2147347950 2020-Nov-30 22:27:10 Warning %LINK-W-Down: te1/0/19
2147347951 2020-Nov-30 22:27:09 Warning %STP-W-PORTSTATUS: te1/0/19: STP status Forwarding
2147347952 2020-Nov-30 22:27:07 Informational %LINK-I-Up: te1/0/19
2147347953 2020-Nov-30 22:26:19 Warning %LINK-W-PORT_SUSPENDED: Port te1/0/19 suspended by link-flapping
2147347954 2020-Nov-30 22:26:19 Warning %LINK-W-Down: te1/0/19
2147347955 2020-Nov-30 22:26:09 Warning %STP-W-PORTSTATUS: te1/0/19: STP status Forwarding
2147347956 2020-Nov-30 22:26:08 Informational %NSFP-I-SFPGibicDetected: te1/0/19 SFP port is present, module type - 10G BASE-CX1
2147347957 2020-Nov-30 22:26:07 Informational %LINK-I-Up: te1/0/19
2147347958 2020-Nov-30 22:26:04 Informational %NSFP-I-SFPGibicRemoved: te1/0/20 SFP port is not present
2147347959 2020-Nov-30 22:24:19 Informational %AAA-I-CONNECT: New http connection for user admin, source 192.168.10.43 destination 192.168.10.250 ACCEPTED
2147347960 2020-Nov-30 22:16:13 Informational %PNPAGENT-I-PNPSRVRDETECT: PnP Server devicehelper.cisco.com was detected
2147347961 2020-Nov-30 22:14:03 Informational %PNPAGENT-I-PNPSRVRDETECT: PnP Server devicehelper.cisco.com was detected
2147347962 2020-Nov-30 22:08:19 Informational %PNPAGENT-I-BYEREQ: Bye PnP Request
2147347963 2020-Nov-30 22:08:17 Informational %PNPAGENT-I-RESPSUCCESS: PnP Response Success
2147347964 2020-Nov-30 22:08:17 Informational %PNPAGENT-I-DEVINFO: Device Info PnP Request with parameter all
2147347965 2020-Nov-30 22:08:15 Informational %PNPAGENT-I-PNPSRVRDETECT: PnP Server devicehelper.cisco.com was detected
2147347966 2020-Nov-30 22:06:05 Informational %PNPAGENT-I-RESPSUCCESS: PnP Response Success
2147347967 2020-Nov-30 22:06:03 Informational %PNPAGENT-I-PNPSRVRDETECT: PnP Server devicehelper.cisco.com was detected
2147347968 2020-Nov-30 22:03:52 Informational %PNPAGENT-I-RESPSUCCESS: PnP Response Success
2147347969 2020-Nov-30 22:03:52 Informational %PNPAGENT-I-BACKOFFCALLBACKAFTER: Backoff PnP Request with <callbackAfter> has received: 0 Hours 1 Minutes 0 Seconds
2147347970 2020-Nov-30 22:03:20 Informational %PNPAGENT-I-BYEREQ: Bye PnP Request
2147347971 2020-Nov-30 22:03:17 Informational %PNPAGENT-I-RESPSUCCESS: PnP Response Success
2147347972 2020-Nov-30 22:03:17 Informational %PNPAGENT-I-DEVINFO: Device Info PnP Request with parameter all
2147347973 2020-Nov-30 22:02:46 Informational %PNPAGENT-I-BYEREQ: Bye PnP Request
2147347974 2020-Nov-30 22:02:43 Informational %PNPAGENT-I-RESPSUCCESS: PnP Response Success
2147347975 2020-Nov-30 22:02:40 Informational %PNPAGENT-I-PNPSRVRDETECT: PnP Server devicehelper.cisco.com was detected
2147347976 2020-Nov-30 22:00:27 Informational %PNPAGENT-I-PNPSRVRDETECT: PnP Server devicehelper.cisco.com was detected
2147347977 2020-Nov-30 21:57:47 Informational %PNPAGENT-I-BYEREQ: Bye PnP Request
2147347978 2020-Nov-30 21:57:46 Informational %PNPAGENT-I-RESPSUCCESS: PnP Response Success
2147347979 2020-Nov-30 21:57:46 Informational %PNPAGENT-I-DEVINFO: Device Info PnP Request with parameter all
2147347980 2020-Nov-30 21:57:45 Informational %PNPAGENT-I-PNPSRVRDETECT: PnP Server devicehelper.cisco.com was detected
2147347981 2020-Nov-30 21:52:30 Informational %PNPAGENT-I-RESPSUCCESS: PnP Response Success
2147347982 2020-Nov-30 21:52:28 Informational %PNPAGENT-I-PNPSRVRDETECT: PnP Server devicehelper.cisco.com was detected
2147347983 2020-Nov-30 21:47:17 Informational %PNPAGENT-I-PNPSRVRDETECT: PnP Server devicehelper.cisco.com was detected
2147347984 2020-Nov-30 21:44:36 Informational %PNPAGENT-I-BYEREQ: Bye PnP Request
2147347985 2020-Nov-30 21:44:36 Informational %PNPAGENT-I-RESPSUCCESS: PnP Response Success
2147347986 2020-Nov-30 21:44:36 Informational %PNPAGENT-I-DEVINFO: Device Info PnP Request with parameter all
2147347987 2020-Nov-30 21:44:05 Informational %PNPAGENT-I-BYEREQ: Bye PnP Request
2147347988 2020-Nov-30 21:44:03 Informational %PNPAGENT-I-RESPSUCCESS: PnP Response Success
2147347989 2020-Nov-30 21:44:02 Informational %PNPAGENT-I-PNPSRVRDETECT: PnP Server devicehelper.cisco.com was detected
2147347990 2020-Nov-30 21:38:48 Informational %PNPAGENT-I-RESPSUCCESS: PnP Response Success
12-23-2020 12:29 PM
this problem has been fixed by Cisco. Cisco ticket # CSCvu44725 but i don't have access
can someone help me...
12-23-2020 11:52 PM
Product: |
(1)
|
Known Affected Releases: |
(2)
|
|
01-18-2021 11:28 PM
I tried connecting to the combo port in my sx350x-24f. This works without any port flapping.
Still my main problem is that the 4 combo ports in my sx350x-24f are being used to connect 4 clients through 10gbase-t rj45 cables. (one of the reasons I chose this switch in the first place).
So although this works, it is a work around NOT a solution, and it confirms that the problem lies with the Cisco Switch.
Is there going to be newer firmware to resolve this issue?
07-28-2021 09:47 AM
Same here.. if anyone has a direct link to this firmware patch that would be much appreciated
07-28-2021 07:08 PM
should not require a SmartNet contract. Just sign in to Cisco.com, support, downloads, search for SX550X, and it is open for all to download.
https://software.cisco.com/download/home/286319791/type/282463181/release/2.5.7.85
Good luck
01-11-2021 12:37 AM
As a follow up to the problem I reported with port flapping between the SFP+ ports of X11SDW-4C-TP13F+ SFP+ and Cisco SX-350X-24F I have done some testing.
First off, In order to eliminate the possibility that the problem with port flapping is not caused by the cables,
I ordered 3 more different cables to test.
In particular
ML-DACS+3 MaxLink ML-DACS+3 SFP+ Direct Attach Cable, passive, DDM, 3m
XS+DA0003 MikroTik XS+DA0003 SFP/SFP+/SFP28 direct attach cable, 3m
S+DA0003 MikroTik S+DA0003 SFP+ 3m direct attach cable
All three exhibit the same behavior as the original cables I was using. (SFPP-PC03 10G SFP+ Passive DAC twinax cables from www.fs.com)
On top of this I ordered a Mikrotik CSS610-8G-2S+IN to check if there would be a problem establishing a link to its SFP+ ports.
The onboard x722 have no problem whatsoever to connect without any port flapping to the 2 SFP+ ports of the Mikrotik switch.
I tried all the cables, they all connected without issues. (in all cases the link established was 10G).
To completely rule out the possibility that the cables where to blame, I also did loopback testing on both the X11SDW-4C-TP13F+ SFP+ ports as well as the cisco ports.
In more detail:
I did a loopback test on the Cisco switch, used both 0.5m as well as the 3m cables from FS.com. In both cases it would connect without any problems whatsoever (10g link).
Ι did a loopback test on the motherboard SFP+ LAN ports. 0.5m as well as the 3m cables. In both cases it would connect without any problems whatsoever (10g link).
Apparently the only way you can actually observe the problem I am experiencing, is to source a SX350X-24F and test connect it to the X11SDW-4C-TP13F+ SFP+ ports.
We cannot be sure if testing with another switch from the same line from CISCO (350X) will exhibit the same behavior.
As it is, I obviously cannot start buying more (quite expensive) switches to test.
As mentioned in my earlier correspondence, there seems to be other cases with x722 LANs SFP+ and link flapping on cisco switches.
Any advice solving this quite annoying issue?
07-28-2021 06:25 PM
Does disabling power save mode on the Intel adapter side resolve the issue? On a Windows machine you would accomplish this by opening device manager -> find the network adapter -> right click and select properties -> Select Power Management -> Uncheck "Allow the computer to turn off the device to save power." On a Debian Linux OS, you'd usually find this in the /etc/default/tlp file. Does setting speeds manually and disabling negotiation resolve the problems? These are common issues with Intel link flap related symptoms.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide