Buy or Renew
Buy or Renew
Cisco + Splunk: It’s a new day for your data. Learn more.
 
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
cancel
Start a conversation
1666
Views
0
Helpful
4
Replies

Very low transfer speed issue on SUN4270 M2 server connected with nexus 5548 switch on 10GB fiber.

jayprakashk
Level 1
Level 1

‎07-23-2012 12:34 AM - edited ‎03-01-2019 07:08 AM

                   Hi,

I have 2 SUN 4270 M2 servers connected with Nexus 5548 switch over 10Gb fiber card. I am getting performance of just 60 MB per second while transfer of 5Gb file across 2 servers. The similar speed i use to get on 1Gb network also. Please suggest how to improve the tranfer speed. On servers, ports ET4 and ETH5 are bonded in bond0 with mode=1. The server envrionment will be used for OVS 2.2.2.

Below are the details of network configuration on server. I quick help will be highly appriciated--


[root@host1 network-scripts]# ifconfig eth4
eth4      Link encap:Ethernet  HWaddr 90:E2:BA:0E:22:4C
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:9000  Metric:1
          RX packets:5648589 errors:215 dropped:0 overruns:0 frame:215
          TX packets:3741680 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2492781394 (2.3 GiB)  TX bytes:3911207623 (3.6 GiB)

[root@host1 network-scripts]# ifconfig eth5
eth5      Link encap:Ethernet  HWaddr 90:E2:BA:0E:22:4C
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:9000  Metric:1
          RX packets:52961 errors:215 dropped:0 overruns:0 frame:215
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:3916644 (3.7 MiB)  TX bytes:0 (0.0 b)


[root@host1 network-scripts]# ethtool eth4
Settings for eth4:
        Supported ports: [ FIBRE ]
        Supported link modes:   1000baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  1000baseT/Full
                                10000baseT/Full
        Advertised auto-negotiation: Yes
        Speed: 10000Mb/s
        Duplex: Full
        Port: FIBRE
        PHYAD: 0
        Transceiver: external
        Auto-negotiation: on
        Supports Wake-on: d
        Wake-on: d
        Current message level: 0x00000007 (7)
        Link detected: yes
[root@host1 network-scripts]# ethtool eth5
Settings for eth5:
        Supported ports: [ FIBRE ]
        Supported link modes:   1000baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  1000baseT/Full
                                10000baseT/Full
        Advertised auto-negotiation: Yes
        Speed: 10000Mb/s
        Duplex: Full
        Port: FIBRE
        PHYAD: 0
        Transceiver: external
        Auto-negotiation: on
        Supports Wake-on: d
        Wake-on: d
        Current message level: 0x00000007 (7)
        Link detected: yes
[root@host1 network-scripts]#

[root@host1 network-scripts]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.2.4 (January 28, 2008)

Bonding Mode: fault-tolerance (active-backup)
Primary Slave: None
Currently Active Slave: eth4
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth4
MII Status: up
Link Failure Count: 0
Permanent HW addr: 90:e2:ba:0e:22:4c

Slave Interface: eth5
MII Status: up
Link Failure Count: 0
Permanent HW addr: 90:e2:ba:0e:22:4d

[root@host1 network-scripts]# modinfo ixgbe | grep ver
filename:       /lib/modules/2.6.18-128.2.1.4.44.el5xen/kernel/drivers/net/ixgbe/ixgbe.ko
version:        3.9.17-NAPI
description:    Intel(R) 10 Gigabit PCI Express Network Driver
srcversion:     31C6EB13C4FA6749DF3BDF5
vermagic:       2.6.18-128.2.1.4.44.el5xen SMP mod_unload Xen 686 REGPARM 4KSTACKS gcc-4.1

[root@host1 network-scripts]#brctl show
bridge name     bridge id               STP enabled     interfaces
vlan301         8000.90e2ba0e224c       no              bond0.301
vlan302         8000.90e2ba0e224c       no              vif1.0
                                                        bond0.302
vlan303         8000.90e2ba0e224c       no              bond0.303
vlan304         8000.90e2ba0e224c       no              bond0.304


[root@host2 test]# ifconfig eth5
eth5      Link encap:Ethernet  HWaddr 90:E2:BA:0F:C3:15
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:9000  Metric:1
          RX packets:4416730 errors:215 dropped:0 overruns:0 frame:215
          TX packets:2617152 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:190977431 (182.1 MiB)  TX bytes:3114347186 (2.9 GiB)

[root@host2 network-scripts]# ifconfig eth4
eth4      Link encap:Ethernet  HWaddr 90:E2:BA:0F:C3:15
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:9000  Metric:1
          RX packets:28616 errors:3 dropped:0 overruns:0 frame:3
          TX packets:424 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:4982317 (4.7 MiB)  TX bytes:80029 (78.1 KiB)

[root@host2 test]#
[root@host2 network-scripts]# ethtool eth4
Settings for eth4:
        Supported ports: [ FIBRE ]
        Supported link modes:   1000baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  1000baseT/Full
                                10000baseT/Full
        Advertised auto-negotiation: Yes
        Speed: 10000Mb/s
        Duplex: Full
        Port: FIBRE
        PHYAD: 0
        Transceiver: external
        Auto-negotiation: on
        Supports Wake-on: d
        Wake-on: d
        Current message level: 0x00000007 (7)
        Link detected: yes
[root@host2 test]# ethtool eth5
Settings for eth5:
        Supported ports: [ FIBRE ]
        Supported link modes:   1000baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  1000baseT/Full
                                10000baseT/Full
        Advertised auto-negotiation: Yes
        Speed: 10000Mb/s
        Duplex: Full
        Port: FIBRE
        PHYAD: 0
        Transceiver: external
        Auto-negotiation: on
        Supports Wake-on: d
        Wake-on: d
        Current message level: 0x00000007 (7)
        Link detected: yes
[root@host2 network-scripts]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.2.4 (January 28, 2008)

Bonding Mode: fault-tolerance (active-backup)
Primary Slave: None
Currently Active Slave: eth5
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth5
MII Status: up
Link Failure Count: 0
Permanent HW addr: 90:e2:ba:0f:c3:14

Slave Interface: eth4
MII Status: up
Link Failure Count: 0
Permanent HW addr: 90:e2:ba:0f:c3:15

[root@host2 network-scripts]# modinfo ixgbe | grep ver
filename:       /lib/modules/2.6.18-128.2.1.4.44.el5xen/kernel/drivers/net/ixgbe/ixgbe.ko
version:        3.9.17-NAPI
description:    Intel(R) 10 Gigabit PCI Express Network Driver
srcversion:     31C6EB13C4FA6749DF3BDF5
vermagic:       2.6.18-128.2.1.4.44.el5xen SMP mod_unload Xen 686 REGPARM 4KSTACKS gcc-4.1

[root@host2 network-scripts]#brctl show
bridge name     bridge id               STP enabled     interfaces
vlan301         8000.90e2ba0fc315       no              bond0.301
vlan302         8000.90e2ba0fc315       no              bond0.302
vlan303         8000.90e2ba0fc315       no              bond0.303
vlan304         8000.90e2ba0fc315       no              vif1.0
       bond0.304

Thanks....

Jay

Labels:
0 Helpful
4 Replies 4

Leo Laohoo
Hall of Fame
Hall of Fame

‎07-23-2012 04:04 PM

On the Nexus, do you see any "Output Drops" or "Pause" frames incrementing?

0 Helpful

jayprakashk
Level 1
Level 1
In response to Leo Laohoo

‎07-24-2012 07:44 AM

Thanks for response...

On Nexus, there is no Output Drops or Pause, but errors count is increasing when i transfer some data from one host to other host through nexus. Even on my server also, error count is increasing as below--

#ifconfig eth5

eth5      Link encap:Ethernet  HWaddr 90:E2:BA:0F:C3:15

          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:9000  Metric:1

          RX packets:5883257 errors:12 dropped:0 overruns:0 frame:12

          TX packets:6660592 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000

          RX bytes:3591022675 (3.3 GiB)  TX bytes:1306712675 (1.2 GiB)

Jay.

0 Helpful

Leo Laohoo
Hall of Fame
Hall of Fame
In response to jayprakashk

‎07-24-2012 03:23 PM

RX errors of that size is insignificant to the amount of total packets received.   But I'd still be worried though. 

0 Helpful

jayprakashk
Level 1
Level 1
In response to Leo Laohoo

‎07-30-2012 05:51 AM

Hi,

Thanks for reply..but the RX errors count is keep on increasing and the transfer speed between 2 servers are max 60MB/ps on 10GB FC card. Even on storage also, i am getting the same speed when i try to transfer data from server to storage on 10GB FC card. Servers and storage are connected through Nexus 5548 switch.

#ifconfig eth5

eth5      Link encap:Ethernet  HWaddr 90:E2:BA:0E:22:4C

          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:9000  Metric:1

          RX packets:21187303 errors:1330 dropped:0 overruns:0 frame:1330

          TX packets:17805543 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000

          RX bytes:624978785 (596.0 MiB)  TX bytes:2897603160 (2.6 GiB)

JP

0 Helpful
Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Customers Also Viewed These Support Documents