cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
7007
Views
0
Helpful
2
Replies

LACP Link Aggregation not using full bandwith

PhilBieber
Level 1
Level 1

Hi all!

I have configured a LACP port-channel on my Cisco 4506 on one side and a Win 2k8R2 Fujitsu Server (Intel NIC) on the other. I have three interfaces on the server that join the etherchannel without any problems. Looking at the sh int po 9, I get a strange output:

HF-SW-01-02#sh int port-channel 9

Port-channel9 is up, line protocol is up (connected)

  Hardware is EtherChannel, address is 68ef.bdaa.78d8 (bia 68ef.bdaa.78d8)

  Description: BackUp Server 1

  MTU 1500 bytes, BW 3000000 Kbit, DLY 10 usec,

     reliability 255/255, txload 1/255, rxload 1/255

  Encapsulation ARPA, loopback not set

  Keepalive set (10 sec)

  Full-duplex, 1000Mb/s, media type is N/A

  input flow-control is on, output flow-control is unsupported

  Members in this channel: Gi3/41 Gi3/42 Gi3/43

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input never, output never, output hang never

  Last clearing of "show interface" counters never

  Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 0 bits/sec, 0 packets/sec

  5 minute output rate 3000 bits/sec, 6 packets/sec

     23017370 packets input, 34014684782 bytes, 0 no buffer

     Received 3140 broadcasts (1265 multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 input packets with dribble condition detected

     6365464 packets output, 2159208832 bytes, 0 underruns

     0 output errors, 0 collisions, 0 interface resets

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier

     0 output buffer failures, 0 output buffers swapped out

The etherchannel consists of three 1GBit/s interfaces, but it shows only 1000Mb/s in the sh int command. When copying files to two seperate servers on the same switch, I only ge an accumulated bandwith of 1Gb/s, while I would excpect 1Gb/s per Server (I tried copying betrween the servers and the got about 1Gb/s in each direction).

Here the relevant config sniplets:

interface Port-channel9

description BackUp Server 1

switchport

switchport access vlan 15

switchport mode access

bandwidth 3000000

end

interface GigabitEthernet3/41

description 4-1-1 - hf-srv-bu-001 Team CS 1 - 10.1.15.18

switchport access vlan 15

switchport mode access

channel-protocol lacp

channel-group 9 mode active

spanning-tree portfast

spanning-tree bpduguard enable

end

!

interface GigabitEthernet3/42

description 4-1-2 - hf-srv-bu-001 Team CS 1 - 10.1.15.18

switchport access vlan 15

switchport mode access

channel-protocol lacp

channel-group 9 mode active

spanning-tree portfast

spanning-tree bpduguard enable

end

!

interface GigabitEthernet3/43

description 4-1-3 - hf-srv-bu-001 Team CS 1 - 10.1.15.18

switchport access vlan 15

switchport mode access

channel-protocol lacp

channel-group 9 mode active

spanning-tree portfast

spanning-tree bpduguard enable

end

HF-SW-01-02#sh etherchannel 9 summary
Flags:  D - down        P - bundled in port-channel
        I - stand-alone s - suspended
        R - Layer3      S - Layer2
        U - in use      f - failed to allocate aggregator

        M - not in use, minimum links not met
        u - unsuitable for bundling
        w - waiting to be aggregated
        d - default port


Number of channel-groups in use: 7
Number of aggregators:           7

Group  Port-channel  Protocol    Ports
------+-------------+-----------+-----------------------------------------------
9      Po9(SU)         LACP      Gi3/41(P)   Gi3/42(P)   Gi3/43(P)


HF-SW-01-02#sh etherchannel 9 detail
Group state = L2
Ports: 3   Maxports = 8
Port-channels: 1 Max Port-channels = 1
Protocol:   LACP
Minimum Links: 0
                Ports in the group:
                -------------------
Port: Gi3/41
------------

Port state    = Up Mstr Assoc In-Bndl
Channel group = 9           Mode = Active          Gcchange = -
Port-channel  = Po9         GC   =   -             Pseudo port-channel = Po9
Port index    = 0           Load = 0x00            Protocol =   LACP

Flags:  S - Device is sending Slow LACPDUs   F - Device is sending fast LACPDUs.
        A - Device is in active mode.        P - Device is in passive mode.

Local information:
                            LACP port     Admin     Oper    Port        Port
Port      Flags   State     Priority      Key       Key     Number      State
Gi3/41    SA      bndl      32768         0x9       0x9     0x6F        0x3D

Partner's information:

                  LACP port                        Admin  Oper   Port    Port
Port      Flags   Priority  Dev ID          Age    key    Key    Number  State
Gi3/41    SA      255       0019.99ca.a2b7   4s    0x0    0x11   0xF     0x3D

Age of the port in the current state: 0d:00h:22m:10s

Port: Gi3/42
------------

Port state    = Up Mstr Assoc In-Bndl
Channel group = 9           Mode = Active          Gcchange = -
Port-channel  = Po9         GC   =   -             Pseudo port-channel = Po9
Port index    = 1           Load = 0x00            Protocol =   LACP

Flags:  S - Device is sending Slow LACPDUs   F - Device is sending fast LACPDUs.
        A - Device is in active mode.        P - Device is in passive mode.

Local information:
                            LACP port     Admin     Oper    Port        Port
Port      Flags   State     Priority      Key       Key     Number      State
Gi3/42    SA      bndl      32768         0x9       0x9     0x70        0x3D

Partner's information:

                  LACP port                        Admin  Oper   Port    Port
Port      Flags   Priority  Dev ID          Age    key    Key    Number  State
Gi3/42    SA      255       0019.99ca.a2b7   7s    0x0    0x11   0xD     0x3D

Age of the port in the current state: 0d:00h:22m:09s

Port: Gi3/43
------------

Port state    = Up Mstr Assoc In-Bndl
Channel group = 9           Mode = Active          Gcchange = -
Port-channel  = Po9         GC   =   -             Pseudo port-channel = Po9
Port index    = 2           Load = 0x00            Protocol =   LACP

Flags:  S - Device is sending Slow LACPDUs   F - Device is sending fast LACPDUs.
        A - Device is in active mode.        P - Device is in passive mode.

Local information:
                            LACP port     Admin     Oper    Port        Port
Port      Flags   State     Priority      Key       Key     Number      State
Gi3/43    SA      bndl      32768         0x9       0x9     0x71        0x3D

Partner's information:

                  LACP port                        Admin  Oper   Port    Port
Port      Flags   Priority  Dev ID          Age    key    Key    Number  State
Gi3/43    SA      255       0019.99ca.a2b7   7s    0x0    0x11   0xE     0x3D

Age of the port in the current state: 0d:00h:22m:11s

                Port-channels in the group:
                ---------------------------

Port-channel: Po9    (Primary Aggregator)

------------

Age of the Port-channel   = 0d:02h:39m:10s
Logical slot/port   = 11/9          Number of ports = 3
Port state          = Port-channel Ag-Inuse
Protocol            =   LACP
Port security       = Disabled

Ports in the Port-channel:

Index   Load   Port     EC state        No of bits
------+------+------+------------------+-----------
  0     00     Gi3/41   Active             0
  1     00     Gi3/42   Active             0
  2     00     Gi3/43   Active             0

Time since last port bundled:    0d:00h:22m:11s    Gi3/43
Time since last port Un-bundled: 0d:00h:22m:17s    Gi3/43

Have I configured something incorrectly? I would like to be able to use three seperate sessions of about 1Gb/s each simultaneously.

A few words reagrding our setup:
We have two 4506e connected with an 8 x 1GB/s etherchannel. The server mentioned above (back up server) is connected to both switches with 3 links of 1GB/s each, also there are two smaller servers, each connected with 1GB/s to one of the switches.

I would like to be able to copy a file from both smaller servers to the back up server simultaneously with a combined speed of 2GB/s (or at least close to it). I am able to get speeds of 1GB/s when copying one file simultaneously.

I would greately appreciate any insight on this matter...

Cheers

Phil

2 Replies 2

Did you change the Load-balancing-Algorithm? I remember that the 4500 uses a default of src-dest-ip, which should be ok in your scenario. Check that with "sh etherchannel load-balance"

The Problem ist that with only a small amount of systems you have a high propability of ending up on the same link.

You can control that with the command "test etherchannel load-balance interface port-channel ...". There you provide the IP-addresses of your servers and see which link would be picked.

Karsten,

thank you for your reply.

I tried the default load balancing mehtod and have just tried to issue your command. Unfortunately, my 4506e (IP Base) does not offer to test any etherchannel related config.

I have tried switching to a SRC & DST Port load balancing method (port-channl load-balance src-dst-port), but unfortunately without any success....

Cheers

Phil

      

[EDIT:

I don't know how relevant this is, but I just looked at the show etherchannel load-balance command. I noticed, it uses the MAC for non IP traffic. The Etherchannel is configured as a L2 channel, but I am using IPv4 for the data transfer... Not sure, which load balancing method the 4506e would use...

HF-SW-01-02#sh etherchannel load-balance
EtherChannel Load-Balancing Configuration:
        src-dst-port

EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source XOR Destination MAC address
  IPv4: Source XOR Destination TCP/UDP (layer-4) port number
  IPv6: Source XOR Destination IP address

]

[EDIT2:

HF-SW-01-02#$rm software etherchannel po 9 map ip 10.1.15.18 10.1.15.10

Map port for Ip 10.1.15.18, 10.1.15.10 is Gi3/42(Po9)

NOTE: Software forwarded traffic will use Gi3/42(Po9)

HF-SW-01-02#$rm software etherchannel po 9 map ip 10.1.15.18 10.1.15.12

Map port for Ip 10.1.15.18, 10.1.15.12 is Gi3/41(Po9)

NOTE: Software forwarded traffic will use Gi3/42(Po9)

HF-SW-01-02#$rm software etherchannel po 9 map ip 10.1.15.19 10.1.15.12

Map port for Ip 10.1.15.19, 10.1.15.12 is Gi3/42(Po9)

NOTE: Software forwarded traffic will use Gi3/42(Po9)

HF-SW-01-02#$rm software etherchannel po 9 map ip 10.1.15.10 10.1.15.18

Map port for Ip 10.1.15.10, 10.1.15.18 is Gi3/42(Po9)

NOTE: Software forwarded traffic will use Gi3/42(Po9)

HF-SW-01-02#$rm software etherchannel po 9 map ip 10.1.15.10 10.1.15.19

Map port for Ip 10.1.15.10, 10.1.15.19 is Gi3/43(Po9)

NOTE: Software forwarded traffic will use Gi3/42(Po9)

HF-SW-01-02#$rm software etherchannel po 9 map ip 10.1.15.12 10.1.15.18

Map port for Ip 10.1.15.12, 10.1.15.18 is Gi3/41(Po9)

NOTE: Software forwarded traffic will use Gi3/42(Po9)

HF-SW-01-02#$rm software etherchannel po 9 map ip 10.1.15.12 10.1.15.19

Map port for Ip 10.1.15.12, 10.1.15.19 is Gi3/42(Po9)

NOTE: Software forwarded traffic will use Gi3/42(Po9)

When issueing the same command with a random IP + Port combination, the results are similar... Is there a way to optimize the load balancing behavoiur?

]

Nachricht geändert durch Philipp Bieber

Review Cisco Networking for a $25 gift card