cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Announcements

CISCO SWITCHES FOR SMALL and MEDIUM BUSINESS

Introducing the next generation of Cisco Small and Medium Business Switches. Cisco is refreshing its SMB Switch portfolio. Click here  to learn more.


5461
Views
0
Helpful
9
Replies
gergelyimre
Beginner

Cisco SG200-08 (SLM2008T-EU) IPv4 packet loss

Hi

I recently bought a SG200-08 8-port gigabit switch. The switch firmware is the latest (1.0.0.16). Everything works fine, except that packet loss I have when flood-pinging on the LAN.

I have 3 machines connected to the switch, they have 1000 full duplex links, they all have Ubuntu Linux installed on them, no limiting on the switch (as far as I can tell), no limiting on the machines (nothing like htb, cbq), no QOS set up on the switch, the machines are fairly OK hardware-wise (dual-core, quad-core, six-core).

And still, it doesn't matter which way I ping, I get a consistent packet loss of 5-7% when flood pinging:

gimre@voy:~$ sudo ping -n -f 172.16.21.4 -c 10000 -q

PING 172.16.21.4 (172.16.21.4) 56(84) bytes of data.

--- 172.16.21.4 ping statistics ---

10000 packets transmitted, 9247 received, 7% packet loss, time 14944ms

rtt min/avg/max/mdev = 0.036/0.062/0.290/0.035 ms, ipg/ewma 1.494/0.051 ms

The really weird thing is that it doesn't happen on IPv6, and it's a LOT faster too:
gimre@voy:~$ sudo ping6 -f -n 2001:470:c952:0:224:1dff:fe7c:c65f -c 180000 -q
PING 2001:470:c952:0:224:1dff:fe7c:c65f(2001:470:c952:0:224:1dff:fe7c:c65f) 56 data bytes
--- 2001:470:c952:0:224:1dff:fe7c:c65f ping statistics ---
180000 packets transmitted, 180000 received, 0% packet loss, time 10017ms
rtt min/avg/max/mdev = 0.036/0.047/0.206/0.017 ms, ipg/ewma 0.055/0.056 ms
I've checked the interface on the other machine and the traffic is indeed getting through, I'm pinging the right IP address every time.
I've looked at the switch's every possible setting but I can't find the problem. It is not that big of a deal, because I rarely have to use over 500pps over the LAN but still, I would like it to not have packet loss.
There's no packet loss at around 500pps:
gimre@voy:~$ sudo ping -n 172.16.21.4 -i 0.002 -q -c 10000
PING 172.16.21.4 (172.16.21.4) 56(84) bytes of data.
--- 172.16.21.4 ping statistics ---
10000 packets transmitted, 10000 received, 0% packet loss, time 19997ms
rtt min/avg/max/mdev = 0.037/0.115/0.281/0.040 ms, ipg/ewma 1.999/0.109 ms
But after that (notice the -i parameter):
gimre@voy:~$ sudo ping -n 172.16.21.4 -i 0.0019 -q -c 10000
PING 172.16.21.4 (172.16.21.4) 56(84) bytes of data.
--- 172.16.21.4 ping statistics ---
10000 packets transmitted, 9401 received, 5% packet loss, time 15416ms
rtt min/avg/max/mdev = 0.036/0.110/0.289/0.039 ms, ipg/ewma 1.541/0.099 ms
Looks like some limitation is going on, but I can't find it. There's no other equipment between the two computers, only the SG200-08. Just for fun I did another test with IPv6 and large packets:
gimre@voy:~$ sudo ping6 -f -n 2001:470:c952:0:224:1dff:fe7c:c65f -c 180000 -q -s 1472
PING 2001:470:c952:0:224:1dff:fe7c:c65f(2001:470:c952:0:224:1dff:fe7c:c65f) 1472 data bytes
--- 2001:470:c952:0:224:1dff:fe7c:c65f ping statistics ---
180000 packets transmitted, 179999 received, 0% packet loss, time 33625ms
rtt min/avg/max/mdev = 0.133/0.167/0.448/0.023 ms, ipg/ewma 0.186/0.173 ms
That's 5300+ pps (~around 60mbps) without problems.
Any help is really appreciated.
9 REPLIES 9
gergelyimre
Beginner

Seems like the empty lines got removed from my initial post, I hope it's still readable... sorry about that...

Nobody? Not a clue, a hint?

I have seen issues with jumbo frames on IPv4 for this model of switch we are currently testing that issue with the switch. I will look into the packet loss issue. It looks like you are using the basic size for your ping of 32 bytes with frame over head of 24 or so. So the frames look to be about 56 bytes. I have seen the switch stop passing packets at 504 bytes. I will be doing further testing today. The is not a documented bug at this time. Let me do some testing today. I will post back to let you know my results.

thanks,

Cisco Small Business Support Center

Randy Manthey

CCNA, CCNA - Security

Thanks a lot! Let me know if I can help with something. I'm away for the weekend but I'll be back Sunday afternoon. Thanks again.

I wasn't able to reproduce 100% but I used a SRW224G4-K9-NA and was able to do a flood test with no issues using packets at mtu 1472. The SLM2008T would not pass packets above 504 MTU period. The flood test I was using required me to use 1472 as the MTU size. The packet size issue has be escalated but I think it is also related to your issue, but I can't be sure with my testing tools. Please call into Cisco Small Business Support and open a case. Please reference my name and have the technician contact me. if I am not available when you call in they should email me the case number and I will take care of getting the case escalated.

Thanks,

Cisco Small Business Support Center

Randy Manthey

CCNA, CCNA - Security

1-866-606-1866

gergelyimre
Beginner

To clarify things, I've opened a case and it turns out this is not a bug, it seems to be a feature. The switch is limiting ICMP packets, that's it. Otherwise I got no problem with it, the transfer speeds are quite nice (tested between two computers connected to the switch, 1000mbps links):

root@voy:~# iperf -c 172.16.21.6 -t 60

------------------------------------------------------------
Client connecting to 172.16.21.6, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  3] local 172.16.21.1 port 53582 connected with 172.16.21.6 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec  6.58 GBytes    942 Mbits/sec

It still would be nice to have the option to turn the ICMP limiting off, but it's not too big of a deal.

Thanks Randy for the help.

Hello

How losing packages can this be a feature?  If this is really a feature, how can we disable it :S

The test you are doing does not help you to check if the switch loses packages, because tcp will resend the missing packages, you have to do a udp test.

I am also having a bad time with this model. I dont know if it is a bad unit, or there has been a quality drop compared with the slm2008

Here are my results:

cheap 1000mb switch

newpili:/home/ricardo# ping -f 192.168.2.44 -c 10000

PING 192.168.2.44 (192.168.2.44) 56(84) bytes of data.

--- 192.168.2.44 ping statistics ---

10000 packets transmitted, 10000 received, 0% packet loss, time 3271ms

rtt min/avg/max/mdev = 0.053/0.239/1.880/0.094 ms, ipg/ewma 0.327/0.243 ms

newpili:/home/ricardo# iperf -c 192.168.2.44 -u -b 1000m

------------------------------------------------------------

Client connecting to 192.168.2.44, UDP port 5001

Sending 1470 byte datagrams

UDP buffer size: 1.91 MByte (default)

------------------------------------------------------------

[  3] local 192.168.2.20 port 39779 connected with 192.168.2.44 port 5001

[ ID] Interval       Transfer     Bandwidth

[  3]  0.0-10.0 sec   962 MBytes   807 Mbits/sec

[  3] Sent 686497 datagrams

[  3] Server Report:

[  3]  0.0-10.0 sec   950 MBytes   797 Mbits/sec   0.015 ms 8612/686496 (1.3%)

[  3]  0.0-10.0 sec  1 datagrams received out-of-order

root@ontario:~# iperf -s -u -i 1

------------------------------------------------------------

Server listening on UDP port 5001

Receiving 1470 byte datagrams

UDP buffer size:  114 KByte (default)

------------------------------------------------------------

[  3] local 192.168.2.44 port 5001 connected with 192.168.2.20 port 39779

[ ID] Interval       Transfer     Bandwidth        Jitter   Lost/Total Datagrams

[  3]  0.0- 1.0 sec  94.2 MBytes   790 Mbits/sec   0.015 ms 1305/68474 (1.9%)

[  3]  1.0- 2.0 sec  95.4 MBytes   800 Mbits/sec   0.011 ms  733/68756 (1.1%)

[  3]  2.0- 3.0 sec  95.1 MBytes   798 Mbits/sec   0.012 ms  817/68648 (1.2%)

[  3]  3.0- 4.0 sec  95.2 MBytes   799 Mbits/sec   0.009 ms  897/68798 (1.3%)

[  3]  4.0- 5.0 sec  95.0 MBytes   797 Mbits/sec   0.011 ms  834/68575 (1.2%)

[  3]  5.0- 6.0 sec  95.0 MBytes   797 Mbits/sec   0.013 ms  791/68588 (1.2%)

[  3]  6.0- 7.0 sec  95.1 MBytes   798 Mbits/sec   0.021 ms  844/68671 (1.2%)

[  3]  7.0- 8.0 sec  95.1 MBytes   798 Mbits/sec   0.023 ms  828/68652 (1.2%)

[  3]  8.0- 9.0 sec  95.3 MBytes   800 Mbits/sec   0.011 ms  762/68762 (1.1%)

[  3]  0.0-10.0 sec   950 MBytes   797 Mbits/sec   0.016 ms 8612/686496 (1.3%)

[  3]  0.0-10.0 sec  1 datagrams received out-of-order

newpili:/home/ricardo# iperf -c 192.168.2.44

------------------------------------------------------------

Client connecting to 192.168.2.44, TCP port 5001

TCP window size: 16.0 KByte (default)

------------------------------------------------------------

[  3] local 192.168.2.20 port 47776 connected with 192.168.2.44 port 5001

[ ID] Interval       Transfer     Bandwidth

[  3]  0.0-10.0 sec  1.09 GBytes   937 Mbits/sec

sg200-08 switch

newpili:/home/ricardo# ping -f 192.168.2.44 -c 10000

PING 192.168.2.44 (192.168.2.44) 56(84) bytes of data.

....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

--- 192.168.2.44 ping statistics ---

10000 packets transmitted, 9340 received, 6% packet loss, time 15076ms

rtt min/avg/max/mdev = 0.080/0.300/0.580/0.117 ms, ipg/ewma 1.507/0.273 ms

newpili:/home/ricardo# iperf -c 192.168.2.44 -u -b 1000m

------------------------------------------------------------

Client connecting to 192.168.2.44, UDP port 5001

Sending 1470 byte datagrams

UDP buffer size: 1.91 MByte (default)

------------------------------------------------------------

[  3] local 192.168.2.20 port 58588 connected with 192.168.2.44 port 5001

[ ID] Interval       Transfer     Bandwidth

[  3]  0.0-10.0 sec   965 MBytes   809 Mbits/sec

[  3] Sent 688262 datagrams

[  3] Server Report:

[  3]  0.0-10.0 sec   951 MBytes   798 Mbits/sec   0.038 ms 9545/688261 (1.4%)

[  3]  0.0-10.0 sec  1 datagrams received out-of-order

^Croot@ontario:~# iperf -s -u -i 1

------------------------------------------------------------

Server listening on UDP port 5001

Receiving 1470 byte datagrams

UDP buffer size:  114 KByte (default)

------------------------------------------------------------

[  3] local 192.168.2.44 port 5001 connected with 192.168.2.20 port 58588

[ ID] Interval       Transfer     Bandwidth        Jitter   Lost/Total Datagrams

[  3]  0.0- 1.0 sec  95.2 MBytes   799 Mbits/sec   0.013 ms 1014/68923 (1.5%)

[  3]  1.0- 2.0 sec  95.1 MBytes   798 Mbits/sec   0.012 ms  966/68822 (1.4%)

[  3]  2.0- 3.0 sec  95.1 MBytes   798 Mbits/sec   0.014 ms  875/68711 (1.3%)

[  3]  3.0- 4.0 sec  95.1 MBytes   797 Mbits/sec   0.009 ms 1001/68812 (1.5%)

[  3]  4.0- 5.0 sec  95.2 MBytes   799 Mbits/sec   0.012 ms  895/68804 (1.3%)

[  3]  5.0- 6.0 sec  95.2 MBytes   799 Mbits/sec   0.009 ms 1039/68976 (1.5%)

[  3]  6.0- 7.0 sec  95.1 MBytes   798 Mbits/sec   0.014 ms  851/68704 (1.2%)

[  3]  7.0- 8.0 sec  95.2 MBytes   798 Mbits/sec   0.009 ms  937/68818 (1.4%)

[  3]  8.0- 9.0 sec  95.2 MBytes   799 Mbits/sec   0.010 ms  945/68888 (1.4%)

[  3]  9.0-10.0 sec  95.0 MBytes   797 Mbits/sec   0.012 ms 1023/68803 (1.5%)

[  3]  0.0-10.0 sec   951 MBytes   798 Mbits/sec   0.038 ms 9545/688261 (1.4%)

[  3]  0.0-10.0 sec  1 datagrams received out-of-order

newpili:/home/ricardo# iperf -c 192.168.2.44

------------------------------------------------------------

Client connecting to 192.168.2.44, TCP port 5001

TCP window size: 16.0 KByte (default)

------------------------------------------------------------

[  3] local 192.168.2.20 port 47779 connected with 192.168.2.44 port 5001

[ ID] Interval       Transfer     Bandwidth

[  3]  0.0-10.0 sec  1.09 GBytes   937 Mbits/sec

SLM2008

newpili:/home/ricardo# ping -f 192.168.2.44 -c 10000

PING 192.168.2.44 (192.168.2.44) 56(84) bytes of data.

--- 192.168.2.44 ping statistics ---

10000 packets transmitted, 10000 received, 0% packet loss, time 3311ms

rtt min/avg/max/mdev = 0.065/0.253/1.002/0.089 ms, ipg/ewma 0.331/0.254 ms

newpili:/home/ricardo# iperf -c 192.168.2.44 -u -b 1000m

------------------------------------------------------------

Client connecting to 192.168.2.44, UDP port 5001

Sending 1470 byte datagrams

UDP buffer size: 1.91 MByte (default)

------------------------------------------------------------

[  3] local 192.168.2.20 port 38780 connected with 192.168.2.44 port 5001

[ ID] Interval       Transfer     Bandwidth

[  3]  0.0-10.0 sec   963 MBytes   808 Mbits/sec

[  3] Sent 686811 datagrams

[  3] Server Report:

[  3]  0.0-10.0 sec   951 MBytes   798 Mbits/sec   0.015 ms 8110/686810 (1.2%)

[  3]  0.0-10.0 sec  1 datagrams received out-of-order

^Croot@ontario:~# iperf -s -u -i 1

------------------------------------------------------------

Server listening on UDP port 5001

Receiving 1470 byte datagrams

UDP buffer size:  114 KByte (default)

------------------------------------------------------------

[  3] local 192.168.2.44 port 5001 connected with 192.168.2.20 port 38780

[ ID] Interval       Transfer     Bandwidth        Jitter   Lost/Total Datagrams

[  3]  0.0- 1.0 sec  95.1 MBytes   797 Mbits/sec   0.015 ms  803/68607 (1.2%)

[  3]  1.0- 2.0 sec  95.2 MBytes   799 Mbits/sec   0.010 ms  816/68754 (1.2%)

[  3]  2.0- 3.0 sec  95.0 MBytes   797 Mbits/sec   0.010 ms  770/68545 (1.1%)

[  3]  3.0- 4.0 sec  95.2 MBytes   799 Mbits/sec   0.012 ms  822/68730 (1.2%)

[  3]  4.0- 5.0 sec  95.1 MBytes   798 Mbits/sec   0.013 ms  818/68672 (1.2%)

[  3]  5.0- 6.0 sec  95.1 MBytes   798 Mbits/sec   0.014 ms  885/68745 (1.3%)

[  3]  6.0- 7.0 sec  95.1 MBytes   797 Mbits/sec   0.011 ms  803/68616 (1.2%)

[  3]  7.0- 8.0 sec  95.3 MBytes   799 Mbits/sec   0.020 ms  860/68844 (1.2%)

[  3]  8.0- 9.0 sec  95.2 MBytes   799 Mbits/sec   0.012 ms  755/68674 (1.1%)

[  3]  9.0-10.0 sec  95.1 MBytes   797 Mbits/sec   0.012 ms  779/68583 (1.1%)

[  3]  0.0-10.0 sec   951 MBytes   798 Mbits/sec   0.015 ms 8110/686810 (1.2%)

[  3]  0.0-10.0 sec  1 datagrams received out-of-order


newpili:/home/ricardo# iperf -c 192.168.2.44

------------------------------------------------------------

Client connecting to 192.168.2.44, TCP port 5001

TCP window size: 16.0 KByte (default)

------------------------------------------------------------

[  3] local 192.168.2.20 port 47781 connected with 192.168.2.44 port 5001

[ ID] Interval       Transfer     Bandwidth

[  3]  0.0-10.0 sec  1.09 GBytes   937 Mbits/sec

Summary

RouterPing flood lossUDP bandwidthUDP LossTCP Bandwidth
Cheapo0%797 Mb1.3%937Mb
sg200-086%798 Mb1.4%937Mb
slm20080%798 Mb1.2%937Mb





mcr314519
Beginner

I have an SG200-26 switch for my home office.

4-5 Linux machines, some XEN servers.  One machine is the household NFS server with everything from our photo and mp3 collection, to a master linux.git tree. Ideally a storage VLAN would hold all the NFS traffic, but the wifi machines aren't on that LAN.

My test case is TCP, both IPv4 and IPv6. A media web server spits out mp3 on port 80 (over IPsec) to my various devices/offices, getting the stuff over NFS. The music would skip. I started blaming bufferbloat on the uplink; then I realized it was happening at my desktop over gigabit ethernet.  Rate limiting the media server to 400kb/s on the port-80 would significantly help.  My other test case is just playing a video from the NFS server either over NFS or over port-80 to VLC. The video would skip. My HDHomeRun (ATSC receiver) can not play to my desktop through the SG200.

If I put an old 100Mb/s Belkin switch between the devices which I'm trying to stream media, then things work. So, for instance, in my lab, I unplug the cable from my desktop and the HDHomeRun from the SG200, and plug it into the 16-port Belkin, and plug the Belkin in to the SG200, then things work fine.

So the 10-year old 100Mb/s switch outperforms the SG200.

I'm running switch firmware:  1.3.7.18 now.

 

The music got perhaps slightly better, but the video still fails.   I also upgraded the NIC in the NFS server from a Realtek to an e1000. I was then able to try turning the ethernet flow control on and off from both ends and see if that had an effect.

I also tried forcing the port on the NFS server to 100Mb/s both at the switch and machine side, and it had no effect: the SG200 can not keep up with 100Mb/s.  I haven't tried 10Mb/s, but I could do that.

I've done tcpdump traces at both ends (I'm also mcr@tcpdump.org, btw.), and I clearly see TCP losing about every 6th packet and then retransmitting it. 

On the advice of an IETF/Cisco colleague, I turned *off* all of the QoS mechanisms. WIth my original firmware, this had no effect.  With 1.3.7.18, it seemed to improve things significantly.

This seems ridiculous.

When I say "improve things significantly", I mean, the videos only step every 5 minutes, instead of every 1 minute.