Two L2L in Etherchannel for voice and data

[filiberto.aguirre]

Hi all !!!

End user has two L2L between HQ and Wearehouse, and wants to have redundancy with load balance.

My first idea is to set an etherchannel between these two L2L with  load-balance src-dst-ip.

Traffic carry in this etherchannel will be data and voice.

I realized that the delay in these L2L is not the same, a little difference.

I worry if this could affect voice traffic when load balance takes place.

L2L  PRIMARY

CORE_W#ping 192.168.100.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.100.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/4/9 ms

L2L SECONDARY

CORE_W#ping 192.168.200.10

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.200.10, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 8/8/9 ms

I want to use the following config ( I just show the config for one side)

vlan 2,10,18,20
!
vlan 23
name SERVERS
!
vlan 24
name DATOS
!
vlan 25
name VOICE

port-channel load-balance src-dst-ip

!
interface Port-channel1
switchport access vlan 10
switchport mode access
!

!
interface GigabitEthernet2/0/23
description Enlace L2L-PRIMARY
switchport access vlan 10
switchport mode access
channel-group 1 mode on
duplex full
spanning-tree portfast
!

interface GigabitEthernet2/0/24
description Enlace L2L- SECONDARY
switchport access vlan 10
switchport mode access
channel-group 1 mode on
duplex full
spanning-tree portfast
!

Any aditional recommendation?

Regards

[Giuseppe Larosa · Accepted solution]

Hello Filiberto,

I would recommend the use of LACP that is able to detect failures on member links because the switches exchange LACP control frames on each member link.

This is specially useful if there are two lan extension services in the middle.

use channel -group 1 mode active

instead of

channel-group 1 mode on

Hope to help

Giuseppe

[filiberto.aguirre]

Hi Giuseppe,

Thanks a lot for your quick response.

I have just tried both option (mode on and active) , definitely active option worked much better.

Thanks once again.

Regards

[filiberto.aguirre]

Hi Giuseppe,

I have the following issue with etherchannel, it had taken place twice since I set; the following messages are displayed, I do not understand why. When this message appears all the communication is lost beetween main office and branch.

These logs are displayed in Branch Office switch, in Main Office switch no loggs messages are shown.

Sep 27 12:36:12.155: %SW_MATM-4-MACFLAP_NOTIF: Host 001f.9d45.c4c8 in vlan 10 is flapping between port Gi1/0/23 and port Po1

Sep 27 12:36:27.364: %SW_MATM-4-MACFLAP_NOTIF: Host 001f.9d45.c4c8 in vlan 10 is flapping between port Gi1/0/23 and port Po1

Sep 27 12:36:40.794: %SW_MATM-4-MACFLAP_NOTIF: Host 001f.9d45.c4c8 in vlan 10 is flapping between port Po1 and port Gi1/0/23

Sep 27 12:36:56.632: %SW_MATM-4-MACFLAP_NOTIF: Host 001f.9d45.c4c8 in vlan 10 is flapping between port Gi1/0/23 and port Po1

G1 1/0/23 is the switch port that correspond to L2L of carrier B in Branch Office and the MAC  001f.9d45.c4c8  correspond to interface VLAN 10 in

main office.

I show part of the config in both switches (Branch and Main Office)

BRANCH OFFICE

port-channel load-balance src-dst-ip

vlan 2,10,18,20,23,24,26,27,28,36,50

interface Port-channel1
switchport access vlan 10
switchport mode access

interface GigabitEthernet1/0/23
description Enlace L2L - CARRIER B
switchport access vlan 10
switchport mode access
duplex full
spanning-tree portfast
channel-group 1 mode active

interface GigabitEthernet1/0/24
description Enlace L2L CARRIER A
switchport access vlan 10
switchport mode access
duplex full
spanning-tree portfast
channel-group 1 mode active

interface Vlan10
description L2L
ip address 192.168.100.3 255.255.255.0
ip helper-address 10.1.5.201
ip helper-address 10.1.5.202
no ip proxy-arp

************************************

MAIN OFFICE

port-channel load-balance src-dst-ip

vlan 2,3,4,5,6,7,8,9,10,11,12,15, 18,20,192

interface Port-channel1
switchport access vlan 10
switchport mode access

interface GigabitEthernet2/0/20
description Enlace L2L CARRIER A
switchport access vlan 10
switchport mode access
duplex full
channel-group 1 mode active
spanning-tree portfast

!
interface GigabitEthernet2/0/24
description ENLACE LAN to LAN CARRIER B
switchport access vlan 10
switchport mode access
duplex full
channel-group 1 mode active
spanning-tree portfast

!
interface Vlan10
description Red L2L
ip address 192.168.100.1 255.255.255.0
ip helper-address 10.1.5.202
ip helper-address 10.1.5.201
no ip proxy-arp

any reason of this loggs? why communication is lost if the other L2L is up ?

regards

[Giuseppe Larosa]

Hello Filiberto,

the errors are strange.

>> Sep 27 12:36:12.155: %SW_MATM-4-MACFLAP_NOTIF: Host 001f.9d45.c4c8 in vlan 10 is flapping between port Gi1/0/23 and port Po1

This would mean that gi1/0/23 was seen as out of the Po1 bundle and the switch complains of seen frames sourced by  the same MAC address on both po1 and gi1/0/23 coming from main office.

The expected behaviour is different: if for any reason LACP control frames are not received on member link gi1/0/23 the link is suspended = like it is disabled and you should see a log message like LACP suspended interface gi1/0/23.

I would suggest the following:

look in the log messages before and after the error messages for LACP related messages to see if any event happened. I would expect to see one message before telling the link gi1/0/23 should be suspended for lack of LACP messages and later a message telling gi1/0/23 has been added again to the bundle.

Use command

show etherchannel detail

and look for the times the member links joined Po1 to see if the involved interface gi1/0/23 has joined the bundle last time just after the issue seen in the logs.

What model is the branch office switch and what is the IOS version running on it?

It might have some SW bug affecting LACP operations

The reason for the connectivity issue is that on main office the bundle was still considered formed by two links, and the branch office switch has seen a bundle po1 with a single link and gi1/0/23 separated.

But this should not happen, LACP should have removed the link from the bundle and should have suspended it, both devices should have agreed on using only the surviving link. Later the link is re-added to the bundle when LACP frames are exchanged again.

Hope to help

Giuseppe

[filiberto.aguirre]

Hi Guiseppe,

I really appreciate your time for answering to this issue.

These are the last logs when the  problem took place. what is strange is that there is no logs of "change state to down" for disconnection of port Gi1/0/23 (L2L secondary).

The IOs version that is running in branch office is c3750e-universalk9-mz.150-1.SE2; I have already checked bug toolkit

  but I did not find any but related to etherchannel.

I double check config of etherchannel in both side, I wonder if I should remove  spanning-tree portfast command. ???

Oct  3 16:06:05.155: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/11, changed state to down

Oct  3 16:06:05.180: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/12, changed state to down

Oct  3 16:06:07.638: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/11, changed state to up

Oct  3 16:06:07.722: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/12, changed state to up

Oct  3 16:06:08.644: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/11, changed state to up

Oct  3 16:06:08.728: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/12, changed state to up

Oct  3 16:07:06.048: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/9, changed state to down

Oct  3 16:07:07.054: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/9, changed state to down

Oct  3 16:07:09.462: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/9, changed state to up

Oct  3 16:07:10.468: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/9, changed state to up

Oct  3 16:07:34.988: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/11, changed state to down

Oct  3 16:07:35.995: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/11, changed state to down

Oct  3 16:07:38.537: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/11, changed state to up

Oct  3 16:07:39.543: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/11, changed state to up

Oct  3 16:07:40.189: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/10, changed state to down

Oct  3 16:07:41.196: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/10, changed state to down

Oct  3 16:07:43.763: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/10, changed state to up

Oct  3 16:07:44.769: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/10, changed state to up

Oct  3 16:08:09.113: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/12, changed state to down

Oct  3 16:08:10.120: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/12, changed state to down

Oct  3 16:08:12.645: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/12, changed state to up

Oct  3 16:08:13.651: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/12, changed state to up

Oct  3 16:57:16.207: %SW_MATM-4-MACFLAP_NOTIF: Host 001f.9d45.c4c8 in vlan 10 is flapping between port Gi1/0/23 and port Po1

Oct  3 16:58:01.908: %SW_MATM-4-MACFLAP_NOTIF: Host 001f.9d45.c4c8 in vlan 10 is flapping between port Gi1/0/23 and port Po1

Oct  3 16:59:06.140: %SW_MATM-4-MACFLAP_NOTIF: Host 001f.9d45.c4c8 in vlan 10 is flapping between port Gi1/0/23 and port Po1

Oct  3 17:00:42.802: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/24, changed state to down   --> we disconnet the cable in this port (L2L primary)

Oct  3 17:00:42.810: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel1, changed state to down

Oct  3 17:00:43.808: %LINK-3-UPDOWN: Interface Port-channel1, changed state to down

Oct  3 17:00:43.808: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/24, changed state to down

Oct  3 17:00:53.019: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/24, changed state to up  ---> connect once again  the cable

Oct  3 17:00:55.813: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/24, changed state to up

Oct  3 17:00:56.811: %LINK-3-UPDOWN: Interface Port-channel1, changed state to up

Oct  3 17:00:57.817: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel1, changed state to up

[Giuseppe Larosa]

Hello Filiberto,

being the bundle a link between switches I agree that it would be wise to remove the spanning-tree portfast command on all member links.

Your switch is a new platform with a new IOS.

The log messages do not show any event related to Gi1/0/23.

However, I would like to point out the following messages:

Oct  3 17:00:42.802: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/24, changed state to down   --> we disconnet the cable in this port (L2L primary)

Oct  3 17:00:42.810: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel1, changed state to down

Oct  3 17:00:43.808: %LINK-3-UPDOWN: Interface Port-channel1, changed state to down

If I have correctly understood your comment you have removed the cable only on Gi1/0/24, and bundle interface po1 has gone down. This is a confirmation that gi1/0/23 was not considered part of the bundle or the po1 should have been stayed up with a single member link down.

Later when the cable is plugged again in gi1/0/24 po1 comes up again.

The command show etherchannel or show etherchannel detail shows the last time a member link joined the bundle can you check for both member links.

Hope to help

Giuseppe