Maximising performance of Network speed on UCS-E Blade in Cisco 3945

[dastrix80]

Hi all

Ive got a 3945 with 4GB Ram, its got a 140D UCS-E M1 blade, with the Quad Core Xeon and 16GB Ram - 2 x 900GB SAS 10k drives in RAID1.

This router also has a 24 Port Gigabit Module in the an SM slot, with 3 x Ubiqiti AC Pro Access points.

My client, a 3x4MIMO @ 5ghz syncs at 1.3Gbps to the AC Pro AP. When copying a file (20GB in size) to the Xeon UCS-E blade, I get a max of about 450Mbits/50MB/s in transfer speed.

I'd expect far more.

Is what I am seeing limits of the Internal UCS-E interface into the Switch Module or something else?

How can I maximize the performance?

Cheers

kris

[Kirk J]

While you may be looking at a network transfer speed metrics, there are other more likely limiting factors including the OS/disk write speeds of the destination.

I would run both iometer and iperf to see what your local disk speeds are (using large block sizes to emulate your large file transfer) and see if iperf records much higher network throughput when  you remove the local disk subsystem from the equation.

Kirk...

[dastrix80]

Thanks Kirk.

Iperf indicates the following, slower than what I was seeing with a standard file transfer.

C:\Users\Kris\Downloads\iperf-3.1.3-win64>iperf3.exe -c 192.168.0.4
Connecting to host 192.168.0.4, port 5201
[  4] local 192.168.0.10 port 62048 connected to 192.168.0.4 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  24.8 MBytes   208 Mbits/sec
[  4]   1.00-2.00   sec  28.5 MBytes   239 Mbits/sec
[  4]   2.00-3.00   sec  27.2 MBytes   229 Mbits/sec
[  4]   3.00-4.00   sec  28.6 MBytes   240 Mbits/sec
[  4]   4.00-5.00   sec  28.2 MBytes   236 Mbits/sec
[  4]   5.00-6.00   sec  26.4 MBytes   222 Mbits/sec
[  4]   6.00-7.00   sec  31.5 MBytes   264 Mbits/sec
[  4]   7.00-8.00   sec  30.5 MBytes   256 Mbits/sec
[  4]   8.00-9.00   sec  31.8 MBytes   266 Mbits/sec
[  4]   9.00-10.00  sec  33.0 MBytes   277 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec   290 MBytes   244 Mbits/sec                  sender
[  4]   0.00-10.00  sec   290 MBytes   244 Mbits/sec                  receiver

iperf Done.

Ill do some disk tests as youve suggested...

[dastrix80]

Correct me if im wrong but these look pretty abysmal?

[dastrix80]

And then this in UDP

Accepted connection from 192.168.0.10, port 52831
[  5] local 192.168.0.4 port 5201 connected to 192.168.0.10 port 60853
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  5]   0.00-1.00   sec  61.4 MBytes   515 Mbits/sec  0.070 ms  211/8076 (2.6%)
[  5]   1.00-2.00   sec  56.9 MBytes   477 Mbits/sec  0.206 ms  2055/9337 (22%)
[  5]   2.00-3.00   sec  71.0 MBytes   596 Mbits/sec  0.185 ms  1327/10417 (13%)
[  5]   3.00-4.00   sec  80.7 MBytes   678 Mbits/sec  0.078 ms  29/10363 (0.28%)
[  5]   4.00-5.00   sec  79.7 MBytes   669 Mbits/sec  0.056 ms  314/10519 (3%)
iperf3: OUT OF ORDER - incoming packet = 17224 and received packet = 49981 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 17266 and received packet = 50023 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 17275 and received packet = 50032 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 17285 and received packet = 50042 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 17342 and received packet = 50099 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 17351 and received packet = 50108 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 17627 and received packet = 50383 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 18736 and received packet = 51492 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 18766 and received packet = 51522 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 18795 and received packet = 51551 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 18834 and received packet = 51590 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 18865 and received packet = 51621 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 18873 and received packet = 51630 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 22033 and received packet = 54791 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 22042 and received packet = 54800 AND SP = 5
[  5]   5.00-6.00   sec  74.3 MBytes   623 Mbits/sec  0.128 ms  68/9560 (0.71%)
iperf3: OUT OF ORDER - incoming packet = 33689 and received packet = 66418 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 33697 and received packet = 66426 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 34622 and received packet = 67351 AND SP = 5
[  5]   6.00-7.00   sec  73.7 MBytes   618 Mbits/sec  0.128 ms  15/9445 (0.16%)
iperf3: OUT OF ORDER - incoming packet = 35458 and received packet = 68186 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 37269 and received packet = 69995 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 37358 and received packet = 70084 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 38560 and received packet = 71278 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 38569 and received packet = 71287 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 38580 and received packet = 71298 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 38622 and received packet = 71340 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 38655 and received packet = 71372 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 38663 and received packet = 71380 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 38705 and received packet = 71422 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 38716 and received packet = 71433 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 39186 and received packet = 71902 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 39195 and received packet = 71911 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 39243 and received packet = 71958 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 39254 and received packet = 71969 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 39296 and received packet = 72011 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 39307 and received packet = 72022 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 39341 and received packet = 72055 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 39360 and received packet = 72074 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 39394 and received packet = 72108 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 39413 and received packet = 72127 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 39448 and received packet = 72162 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 39456 and received packet = 72170 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 39467 and received packet = 72181 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 39501 and received packet = 72215 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 39520 and received packet = 72234 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 39554 and received packet = 72267 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 39573 and received packet = 72285 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 39608 and received packet = 72320 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 39616 and received packet = 72328 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 39676 and received packet = 72388 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 41236 and received packet = 73935 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 41541 and received packet = 74239 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 41572 and received packet = 74269 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 41603 and received packet = 74296 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 41665 and received packet = 74362 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 41673 and received packet = 74370 AND SP = 5
[  5]   7.00-8.00   sec  71.3 MBytes   598 Mbits/sec  0.171 ms  133/9227 (1.4%)
iperf3: OUT OF ORDER - incoming packet = 46004 and received packet = 78684 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 51196 and received packet = 83877 AND SP = 5
[  5]   8.00-9.00   sec  72.8 MBytes   610 Mbits/sec  0.189 ms  35/9347 (0.37%)
iperf3: OUT OF ORDER - incoming packet = 53844 and received packet = 86526 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 54298 and received packet = 86979 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 58458 and received packet = 91151 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 58646 and received packet = 91340 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 61108 and received packet = 93807 AND SP = 5
[  5]   9.00-10.00  sec  72.4 MBytes   608 Mbits/sec  0.091 ms  39/9302 (0.42%)
[  5]  10.00-10.06  sec  4.18 MBytes   583 Mbits/sec  0.053 ms  6/541 (1.1%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  5]   0.00-10.06  sec  0.00 Bytes  0.00 bits/sec  0.053 ms  4232/96134 (4.4%)
[SUM]  0.0-10.1 sec  62 datagrams received out-of-order
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.0.10, port 52832
[  5] local 192.168.0.4 port 5201 connected to 192.168.0.10 port 59964
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  5]   0.00-1.00   sec  58.6 MBytes   491 Mbits/sec  0.105 ms  700/8202 (8.5%)
[  5]   1.00-2.00   sec  63.3 MBytes   531 Mbits/sec  0.091 ms  749/8850 (8.5%)
[  5]   2.00-3.00   sec  66.7 MBytes   560 Mbits/sec  0.080 ms  1018/9559 (11%)
[  5]   3.00-4.00   sec  82.5 MBytes   692 Mbits/sec  0.113 ms  50/10605 (0.47%)
iperf3: OUT OF ORDER - incoming packet = 11735 and received packet = 44491 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 11744 and received packet = 44500 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 11755 and received packet = 44511 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 11789 and received packet = 44545 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 11797 and received packet = 44550 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 11808 and received packet = 44564 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 11895 and received packet = 44651 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 11904 and received packet = 44660 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 11934 and received packet = 44689 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 11953 and received packet = 44709 AND SP = 5
[  5]   4.00-5.00   sec  80.6 MBytes   676 Mbits/sec  0.164 ms  133/10436 (1.3%)
[  5]   5.00-6.00   sec  82.8 MBytes   694 Mbits/sec  0.110 ms  13/10611 (0.12%)
iperf3: OUT OF ORDER - incoming packet = 26028 and received packet = 58784 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 26919 and received packet = 59671 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 27120 and received packet = 59872 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 29102 and received packet = 61855 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 29136 and received packet = 61889 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 29184 and received packet = 61937 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 29344 and received packet = 62097 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 31382 and received packet = 64134 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 31466 and received packet = 64218 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 31943 and received packet = 64695 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 32647 and received packet = 65401 AND SP = 5
[  5]   6.00-7.00   sec  78.2 MBytes   656 Mbits/sec  0.102 ms  58/10056 (0.58%)
iperf3: OUT OF ORDER - incoming packet = 37228 and received packet = 69980 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 40464 and received packet = 73216 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 41498 and received packet = 74249 AND SP = 5
[  5]   7.00-8.00   sec  72.4 MBytes   607 Mbits/sec  0.126 ms  257/9524 (2.7%)
iperf3: OUT OF ORDER - incoming packet = 45424 and received packet = 78175 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 45980 and received packet = 78731 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 46930 and received packet = 79680 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 47577 and received packet = 80326 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 48838 and received packet = 81585 AND SP = 5
[  5]   8.00-9.00   sec  75.4 MBytes   633 Mbits/sec  0.090 ms  32/9678 (0.33%)
iperf3: OUT OF ORDER - incoming packet = 56067 and received packet = 88810 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 58248 and received packet = 90991 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 58370 and received packet = 91113 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 58644 and received packet = 91387 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 58652 and received packet = 91395 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 58681 and received packet = 91424 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 61155 and received packet = 93898 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 61665 and received packet = 94408 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 62434 and received packet = 95176 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 62443 and received packet = 95185 AND SP = 5
[  5]   9.00-10.00  sec  74.6 MBytes   626 Mbits/sec  0.142 ms  62/9600 (0.65%)
[  5]  10.00-10.04  sec  3.10 MBytes   640 Mbits/sec  0.110 ms  13/410 (3.2%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  5]   0.00-10.04  sec  0.00 Bytes  0.00 bits/sec  0.110 ms  3085/97531 (3.2%)
[SUM]  0.0-10.0 sec  39 datagrams received out-of-order