Quick link aggregation

[victor_87]

Hello,

I been trying to use LACP for link aggregation. However, i want the Link aggregation to be very fast and hence enabled the Fast mode on channel members, but the convergence time is still over 30 sec when i try to test this by shutting down all members and bring them back online. Is this expected ? Or would i be missing something?

Below is the typical config on the switchport on both sides.

switchport

switchport trunk encapsulation dot1q

switchport mode trunk

no ip address

lacp rate fast

channel-group 1 mode active

[Gregory Snipes]

Are you also configuring the po1 interface? You do not even really need the trunking commands on the port, once you place it in a channel group it is the po1 configuration that is important.

[Steve Fuller]

Hi,

What is the link aggregate configured between? Is it between two switches or a switch and a server?

The reason I ask is that 30-seconds sound like spanning tree taking the link from blocking to forwarding. If this is to a server running link aggregation you should also configure the spanning-tree portfast or spanning-tree portfast edge command under the physical and port-channel interfaces.

Regards

[victor_87]

Hello,

Thanks for the quick responses.

@Gregory

Yeah, the Port channel interface is configured. No issues with that part.

sh etherchannel 1 summary

Flags:  D - down        P - bundled in port-channel

        I - stand-alone s - suspended

        H - Hot-standby (LACP only)

        R - Layer3      S - Layer2

        U - in use      f - failed to allocate aggregator

        M - not in use, minimum links not met

        u - unsuitable for bundling

        w - waiting to be aggregated

Number of channel-groups in use: 1

Number of aggregators:           1

Group  Port-channel  Protocol    Ports

------+-------------+-----------+-----------------------------------------------

1      Po1(SU)         LACP      Gi1/1(P)   Gi1/5(P)   Gi1/6(D)   Gi1/7(P)  

                                 Gi1/8(P)  

@Steve

The Etherchannel is with another switch and im trunking so spanning tree portfast is not allowed. I to was thinking it's the Spanning tree delaying this, is there any other workaround to make this fast, else there is no point using the fast mode i guess.

[Steve Fuller]

Hi,

Can you post the output of the show lacp 1 neighbor and show lacp 1 internal command? Do you see a Port State and Partner Port State with a value of 0x3F or 0x3D?

Could you also add the following commands to the port-channel interface configuration?

!

logging event link-status

logging event spanning-tree status

logging event bundle status

!

This would help us see the timing between the links becoming operational, being bundled and the spanning-tree state.

Regards

Message was edited by: Steve Fuller Request to add logging commands included.

[ALIAOF_]

So you have a port channel lets say two ports from each switch.  You are turning off one port and it is taking over 30 seconds for traffic to start working again over the second remaining link in that port channel?

Also which version of spanning tree are you running?

[paul driver]

Hello

Your etherchannel looks okay, Also stp sees a port-channel as one link, so any physical changes to the interfaces within should not effect the port-channel connection as long as their is a tleast one interface up on either side of the interconnect

For fast convergence when you test shutting down the whole port-channel -I would suggest change the spanning-tree to RSTP (802.1w)

This will need to be done on both switches otherwise you will not get the rapid converence  that rstp offers.

please reviiew this cco doc for clarity.

spanning-tree mode rstp

res

Paul

Please don't forget to rate any posts that have been helpful.

Thanks.

[victor_87]

Thanks for your Responses.

I did realize that Spanning tree is delaying stuff. Running RSTP brought down the delay to 10 sec now. Trying to run some debugs to figure out why it is still 10sec.

Can the convergence time get any better?

Victor

[Steve Fuller]

Hi Victor,

Looking at the trunk configuration you could add the switchport nonegotiate command to the port-channel and physical interfaces as this will disable DTP on the link. That may help a little.

Did you add the logging commands I mentioned yesterday as this would help understand how quickly the links are being bundled into the port-channel and getting to the STP forwarding state?

I tested in my lab and when I shut/no shut the port-channel interface I can see that STP is forwarding around 3-seconds after the link comes up:

[..]

May 14 15:41:46.854 BST: %LINK-SP-3-UPDOWN: Interface GigabitEthernet3/16, changed state to up

May 14 15:41:47.378 BST: %LINK-SP-3-UPDOWN: Interface GigabitEthernet3/8, changed state to up]

May 14 15:41:49.543 BST: %EC-SP-5-BUNDLE: Interface Gi3/8 joined port-channel Po1

May 14 15:41:49.551 BST: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 11 moving from disabled to blocking

May 14 15:41:49.575 BST: %EC-SP-5-BUNDLE: Interface Gi3/16 joined port-channel Po1

May 14 15:41:49.583 BST: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 11 moving from blocking to forwarding

[..]

What is the test you're actually doing and how are you measuring the convergence?

Regards

[victor_87]

Hi Steve,

Sorry for the delayed response. I was away for a bit.

Regarding the testing, we are currently using ON mode of etherchannels which is quick and does the Job well. The problem with ON is that there is no control messages/negotiation hence when there is a weird problem on the link and traffic is not passing thru, the port is not removed from the port channel immediately and this causes loss of data.

Trying to use a negotiation protocol for the port channel and still keep the convergence time as minimal as possible with LACP.

I have enabled the logging suggested by you. The output is further below. Im just measuring the convergence on the fly by counting the approx seconds it takes for pings across the switch to start after i unshut it.

with DTP on it is currently taking around 10 sec and with DTP OFF it takes about 6-7 secs. This is for LACP.

With the ON mode for etherchannel, and with DTP ON, it takes about 6 seconds flat.

With DTP ON

===========

switch(config-if)#shut

switch(config-if)#

*May 30 04:35:32: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel1, changed state to down

*May 30 04:35:32: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 1 moving from blocking to disabled

*May 30 04:35:32: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 505 moving from forwarding to disabled

*May 30 04:35:32: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 510 moving from forwarding to disabled

*May 30 04:35:32: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 622 moving from forwarding to disable

switch(config-if)#

switch(config-if)#d

*May 30 04:35:32: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 810 moving from forwarding to disabled

*May 30 04:35:32: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface Port-channel1, changed state to down

*May 30 04:35:33: %LINK-3-UPDOWN: Interface Port-channel1, changed state to down

*May 30 04:35:33: %LINK-SP-3-UPDOWN: Interface Port-channel1, changed state to down

*May 30 04:35:33: %LINK-5-CHANGED: Interface Port-channel1, changed state to administratively down

*May 30 04:35:33: %LINK-SP-5-CHANGED: Interface Port-channel1, changed state to administratively down

switch(config-if)#

switch(config-if)#no shut

switch(config-if)#

*May 30 04:35:39: %LINK-3-UPDOWN: Interface Port-channel1, changed state to down

*May 30 04:35:39: %LINK-SP-3-UPDOWN: Interface Port-channel1, changed state to down

*May 30 04:35:48: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 1 moving from disabled to blocking

*May 30 04:35:48: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 505 moving from disabled to blocking

*May 30 04:35:48: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 510 moving from disabled to blocking

*May 30 04:35:48: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 622 moving from disabled to blocking

*May 30 04:35:48: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 810 moving from disabled to blocking

*May 30 04:35:48: %LINK-3-UPDOWN: Interface Port-channel1, changed state to up

*May 30 04:35:48: %LINK-SP-3-UPDOWN: Interface Port-channel1, changed state to up

*May 30 04:35:48: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel1, changed state to up

*May 30 04:35:48: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface Port-channel1, changed state to up

*May 30 04:35:49: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 505 moving from blocking to forwarding

*May 30 04:35:49: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 810 moving from blocking to forwarding

*May 30 04:35:49: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 510 moving from blocking to forwarding

*May 30 04:35:49: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 622 moving from blocking to forwarding

switch(config-if)#

switch(config-if)#

*May 30 04:36:03: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 1 moving from blocking to learning

*May 30 04:36:18: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 1 moving from learning to forwarding

With DTP negotiation turned OFF

==========================

switch(config-if)#shut

switch(config-if)#

*May 30 04:47:41: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel1, changed state to down

*May 30 04:47:41: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 1 moving from blocking to disabled

*May 30 04:47:41: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 505 moving from forwarding to disabled

*May 30 04:47:41: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 510 moving from forwarding to disabled

*May 30 04:47:41: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 622 moving from forwarding to disable

switch(config-if)#

switch(config-if)#

switch(config-if)#

switch(config-if)#d

*May 30 04:47:41: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 810 moving from forwarding to disabled

*May 30 04:47:41: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface Port-channel1, changed state to down

*May 30 04:47:41: %LINK-3-UPDOWN: Interface Port-channel1, changed state to down

*May 30 04:47:41: %LINK-5-CHANGED: Interface Port-channel1, changed state to administratively down

*May 30 04:47:41: %LINK-SP-3-UPDOWN: Interface Port-channel1, changed state to down

switch(config-if)#

switch(config-if)#

*May 30 04:47:41: %LINK-SP-5-CHANGED: Interface Port-channel1, changed state to administratively down

switch(config-if)#

switch(config-if)#

switch(config-if)#

switch(config-if)#

switch(config-if)#no shut

switch(config-if)#

*May 30 04:47:58: %LINK-3-UPDOWN: Interface Port-channel1, changed state to down

*May 30 04:47:58: %LINK-SP-3-UPDOWN: Interface Port-channel1, changed state to down

*May 30 04:48:03: %LINK-3-UPDOWN: Interface Port-channel1, changed state to up

*May 30 04:48:03: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 1 moving from disabled to blocking

*May 30 04:48:03: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 505 moving from disabled to blocking

*May 30 04:48:03: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 510 moving from disabled to blocking

*May 30 04:48:03: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 622 moving from disabled to blocking

*May 30 04:48:03: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 810 moving from disabled to blocking

*May 30 04:48:03: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel1, changed state to up

*May 30 04:48:03: %LINK-SP-3-UPDOWN: Interface Port-channel1, changed state to up

*May 30 04:48:03: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface Port-channel1, changed state to up

*May 30 04:48:04: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 510 moving from blocking to forwarding

*May 30 04:48:04: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 622 moving from blocking to forwarding

*May 30 04:48:04: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 505 moving from blocking to forwarding

*May 30 04:48:04: %SPANTREE-SP-6-PORT_STATE: Port Po1 instance 810 moving from blocking to forwarding

[ALIAOF_]

Check and make sure all switches are running RSTP, also it is good to hard code the port speed and duplex settings.  You are running LACP, can you try to instead setup both sides to "mode on" and test....

[victor_87]

I am running RSTP.

One more question:

Is LACP slower than No Negotiation?

When i have channel members configured in LACP Active/passive modes, it takes about 10 sec for the channels to go online.

But when i have them configured for No negotiation (On mode), the channel comes online within 5-6 secs.

Is this expected? i did try the FAST mode for LACP but still the same results basically.

[ALIAOF_]

Yes the negotiation will cause a delay indeed as compared to "mode on" because when you have the "mode on" channel comes up right away there is nothing to negotiate.