sg300-28 port-channel options

[Brendan Kearney]

i have an sg300-28 running the latest firmware, and would like some insight on port-channel options.  below are the port configs i have for a LAG to my router.  i am currently using 802.3ad with LACP.  my router is a linux machine pulling duty as a basic (no dynamic routing) router, firewall and internet gateway.  the bonding options on the routers side explained at

http://docs.fedoraproject.org/en-US/Fedora/16/html/System_Administrators_Guide/sec-Using_Channel_Bonding.html.  the mode is 4 or 802.3ad and the xmit_hash_policy is 2 or layer2+3.  i also have 2 servers setup in a similar fashion with 2 interfaces in a LAG.

when i run a bandwidth test, iperf, between the two servers, i only get 900+ mbps which indicates that the GB ports are running fine.  this also indicates to me that the traffic is not being "striped" across the port-channeled interfaces, thereby giving me the aggregated bandwidth of all interfaces in the LAG.  i have found no options to use the balance-xor mode to create port-channels, which as i understand it, would aggregate the total bandwidth of all interfaces in the LAG.  is the sg300 capable of creating a LAG that will combine the throughput of all the members of the LAG?  for example, create a 2 GB pipe when 2 interfaces are port-channeled?  is the balance-xor mode what would do this (regardless of the sg300's ability to do this)

interface gigabitethernet25

description "Port Channel to Router"                

channel-group 1 mode auto

lldp notifications enable

lldp optional-tlv port-desc sys-name sys-desc sys-cap 802.3-mac-phy 802.3-lag 802.3-max-frame-size

lldp management-address automatic

!

interface gigabitethernet26

description "Port Channel to Router"

channel-group 1 mode auto

lldp notifications enable

lldp optional-tlv port-desc sys-name sys-desc sys-cap 802.3-mac-phy 802.3-lag 802.3-max-frame-size

lldp management-address automatic

!

interface gigabitethernet27

description "Port Channel to Router"

channel-group 1 mode auto

lldp notifications enable

lldp optional-tlv port-desc sys-name sys-desc sys-cap 802.3-mac-phy 802.3-lag 802.3-max-frame-size

lldp management-address automatic

!

interface gigabitethernet28

description "Port Channel to Router"

channel-group 1 mode auto                           

lldp notifications enable

lldp optional-tlv port-desc sys-name sys-desc sys-cap 802.3-mac-phy 802.3-lag 802.3-max-frame-size

lldp management-address automatic

!

interface Port-channel1

description "Port Channel to Router"

switchport mode general

switchport general allowed vlan add 2-3,25,37,50,52,253-255 tagged

switchport general pvid 255

!

[David Hornstein]

Hi Brendan,

You said "i only get 900+ mbps which indicates that the GB ports are running fine.  this also indicates to me that the traffic is not being "striped" across the port-channeled interfaces, thereby giving me the aggregated bandwidth of all interfaces in the LAG.  i have found no options to use the balance-xor mode to create port-channels, which as i understand it, would aggregate the total bandwidth of all interfaces in the LAG."

As the Admin guide says on page 130, http://www.cisco.com/en/US/docs/switches/lan/csbms/sf30x_sg30x/administration_guide/78-19308-01.pdf

Load Balancing

Traffic forwarded to a LAG is load-balanced across the active member ports, thus achieving an effective bandwidth close to the aggregate bandwidth of all the active member ports of the LAG.

Traffic load balancing over the active member ports of a LAG is managed by a hash-based distribution function that distributes Unicast and Multicast traffic based on Layer 2 or Layer 3 packet header information.

The switch supports two modes of load balancing:

• By MAC Addresses—Based on the destination and source MAC addresses of all packets.
• By IP and MAC Addresses—Based on the destination and source IP addresses for IP packets, and destination and source MAC addresses for

non-IP packets.

   So,  an IP host running running IPERF maybe checking unicast throughput between the two IP hosts.  There will be a Source and Desination IP address in that test.  The switch will direct the traffic over one of the LAG port members .  It wont Round robin the unicast traffic over multiple LAG ports, if the Source and Desination IP address of the traffic is the same.  .

If the  PC  running IPerf,  had  another concurrent IPerf session to another or different IP host, the hash algorithm on the switch may direct that stream , maybe, over a different physical LAG interface.

So your comment about  achieving  900+mbps  sounds normal   Yes LAG spreads the load, the benefit comes when lots of hosts on  both sides of the switches.

You hay find with just two hosts on either side of a LAG, that the switch may run the traffic between two hosts over just one member of the LAG group.

regards Dave

[Brendan Kearney]

so how do i make the switch use all members of the LAG all the time, in a more "striped" fashion?  is that possible?

[Tom Watts]

Hi Brendan, exclusive xor is defined as a hash operation.

XOR is defined as

• SRC-DST-IP
• SRC-DST-MAC
• SRC-DST-PORT

The SX300 doesn't support balance by port, but as David said above, it does support by IP and MAC.

The switch supports 2 load balance toggles

config t

port-channel load-balance src-dst-mac-ip

config t

port-channel load-balance src-dst-mac

You should choose which will effectively work for your deployment

-Tom
Please rate helpful posts

[David Hornstein]

Hi Brendan,

To add to the concersation, one possible situation desribed in "IEEE Std 802.1AX -2008", appendix A,  is the Linux server to server Link aggregation link, almost a back to back direct connect link.

Because the Source MAC address (SA)  and Destination MAC address (DA) of the servers are always the same, it could just use a single link in the channel group to talk between servers.

Luckily for the linux , server and NAS distributions, the operating systems can offer  options such as;

• Round-robin (balance-rr)
• Active-backup (active-backup)
• XOR (balance-xor)
• Broadcast (broadcast)
• IEEE 802.3ad Dynamic link aggregation (802.3ad).
• Adaptive transmit load balancing (balance-tlb)
• Adaptive load balancing (balance-alb)

to allow for a even distribution of packets rather than all traffic running over a single link. Yep you have more options on the server when compared to the switch. As Tom said in hist posting above, on this family of switches we offer tw types of load balancing;

Load Balance Algorithms:within the 300 series

• MAC Address—Perform load balancing by source and destination MAC addresses on all packets.
• IP /  MAC Address—Perform load balancing by the source and destination IP addresses on IP packets, and by the source and destination MAC addresses on non-IP packets

But these days gigabit NIC cards in PC  and maybe 10GbE in a server, you will find that the traffic will spread / balance  over the individual ports in a LAG group..especially when we have increasing numbers of hosts in a network. Running a IPerf  test between two hosts i would expect the results you received.

regards Dave