06-04-2012 01:14 PM - edited 03-04-2019 04:33 PM
I have a 10Gb link between two sites about a 1k miles apart. I have a Cat 6506 on one end and a Nex 7018 on the other. I have a laptop plugged in to each side. I run iperf tests with various settings changing the window sizes and find that it really doesn’t make a huge difference to the outcome of the tests.
To get a baseline I plugged another laptop in to a different port on the switches IE: PC1 = 3/4 , PC2= 2/7 and run iperf and get the following results- 6506 = 550 Mb average, 7018 = 939 Mb average.
When I try a iperf test across the 10Gb link I get 17-18Mb. If I change the test and run it with 10 parallel threads I only get 17-18 Mb per thread but can get 700-800 Mb total bandwidth. My limiting factor is my hosts are connected via 1Gb links I will not get 10 Gb.
I am trying to figure out why I am only getting 17-18 Mb per thread across this 10 Gb link.
Thanks for any ideas anyone can provide,
06-04-2012 02:18 PM
Unless there are some layer 2 errors showing up on the link, Its doubtful that its the 10G.
http://cisconet.com/traffic-analysis/throughput/104-tcp-throughput-calculation-formula.html
Most times its TCP setup on the end stations that make things appear slow.
06-04-2012 04:18 PM
I have a 10Gb link between two sites about a 1k miles apart.
Stupid question: Dark fibre or via an ISP?
Can you post the output to the command "sh interface ten X/Y/Z"?
06-05-2012 08:46 AM
Thanks for the reply.
ISP
Ethernet5/8 is up
Dedicated Interface
Hardware: 10000 Ethernet, address: 44b3.dc84.b22f (bia 44b3.dc84.b22f)
Description: 10Gb link
MTU 9216 bytes, BW 10000000 Kbit, DLY 10 usec
reliability 255/255, txload 1/255, rxload 3/255
Encapsulation ARPA
Port mode is trunk
full-duplex, 10 Gb/s, media type is 10G
Beacon is turned off
Auto-Negotiation is turned off
Input flow-control is off, output flow-control is off
Rate mode is dedicated
Switchport monitor is off
EtherType is 0x8100
Last link flapped 1week(s) 6day(s)
Last clearing of "show interface" counters never
23 interface resets
30 seconds input rate 167858264 bits/sec, 17340 packets/sec
30 seconds output rate 7295136 bits/sec, 4024 packets/sec
Load-Interval #2: 5 minute (300 seconds)
input rate 134.38 Mbps, 14.29 Kpps; output rate 5.54 Mbps, 3.16 Kpps
RX
77214448366 unicast packets 217239922 multicast packets 247003260 broadcast packets
77678699778 input packets 86652678395790 bytes
0 jumbo packets 0 storm suppression packets
0 runts 0 giants 1 CRC 0 no buffer
2 input error 1 short frame 0 overrun 0 underrun 0 ignored
0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop
0 input with dribble 0 input discard
0 Rx pause
TX
20043125899 unicast packets 301631055 multicast packets 470815729 broadcast packets
20815572683 output packets 11397424890924 bytes
0 jumbo packets
0 output error 0 collision 0 deferred 0 late collision
0 lost carrier 0 no carrier 0 babble 0 output discard
0 Tx pause
06-05-2012 08:52 AM
Hmm. non zero counters for resets, input erros and crc.
I'd clear the counters at a specic time, then check 24 hours later.
Since the last clearing of show interfaces was "never" we don't know when those errors occured.
06-06-2012 02:25 PM
Here they are 29 hours later.
Ethernet5/8 is up
Dedicated Interface
Hardware: 10000 Ethernet, address: 44b3.dc84.b22f (bia 44b3.dc84.b22f)
Description: DMSP Beltline link
MTU 9216 bytes, BW 10000000 Kbit, DLY 10 usec
reliability 255/255, txload 1/255, rxload 2/255
Encapsulation ARPA
Port mode is trunk
full-duplex, 10 Gb/s, media type is 10G
Beacon is turned off
Auto-Negotiation is turned off
Input flow-control is off, output flow-control is off
Rate mode is dedicated
Switchport monitor is off
EtherType is 0x8100
Last link flapped 13:49:05
Last clearing of "show interface" counters 1d05h
1 interface resets
30 seconds input rate 105473208 bits/sec, 12087 packets/sec
30 seconds output rate 4216576 bits/sec, 3273 packets/sec
Load-Interval #2: 5 minute (300 seconds)
input rate 107.97 Mbps, 12.34 Kpps; output rate 7.11 Mbps, 3.72 Kpps
RX
1060231327 unicast packets 2602443 multicast packets 2611436 broadcast packets
1065445205 input packets 1090897061568 bytes
0 jumbo packets 0 storm suppression packets
0 runts 0 giants 0 CRC 0 no buffer
0 input error 0 short frame 0 overrun 0 underrun 0 ignored
0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop
0 input with dribble 0 input discard
0 Rx pause
TX
343355770 unicast packets 1503259 multicast packets 771391 broadcast packets
345630420 output packets 198922907428 bytes
0 jumbo packets
0 output error 0 collision 0 deferred 0 late collision
0 lost carrier 0 no carrier 0 babble 0 output discard
0 Tx pause
06-05-2012 04:50 PM
Sorry, you got a 10 Gbps link from an ISP????
You sure about that???
06-06-2012 02:50 PM
no low level link issues. I'd look to workstation tuning.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide