Difference between L2 vs L3 etherchannel

[Horacio Gutierrez]

Hello,

I have to 6509 core switches that I am running HSRP between them.  I have two ethernet channels between the two switches, which are channel 1 and channel 2.  My question why would I have a Layer 2 ethernet channnel and also Layer  3 ehternet channel between these two switches instead of only Layer 3 or Layer 2 ethernet channels.  Also I have an MPLS network.  Please let me know what you guys think. Below is the config.

Thanks.

interface Port-channel1

description to Core

ip address 10.89.255.12 255.255.255.244

ip pim sparse-dense-mode

ip hello-interval eigrp 1226 1

ip hold-time eigrp 1226 3

flowcontrol receive on

flowcontrol send on

hold-queue 250 in

hold-queue 200 out

!

interface Port-channel2

description to Core/ (VLAN traffic)

switchport

switchport access vlan 999

switchport trunk encapsulation dot1q

switchport trunk native vlan 999

switchport trunk allowed vlan 700-705,888,999

switchport mode trunk

!

[smitesh kharecha]

Hi,

It depends on the requirement.

If you want the spanning-tree across the network till the core (as per your config), then you will need L2 etherchannel.

And if you don't need that, then L3 etherchannel is a good idea.

Regards,

Smitesh

[Vaibhava Varma]

Hi Horacio

L2 Channel will be required between the Core Switches to pass the HSRP Hellos and also to provide redundant L2 Path for few VLANs which are terminated in a cross-linked way from the LAN side. So basically its needed for getting the required L2 path needed in a typical LAN infrastructure.

Now we can use the same L2 Port-Channel and to create a L3 Link (using dedicated VLANs) between the core switches in case of providing upstream routing redundancy. If we are using HSRP Track for the Upstream WAN Links the L3 Link will not be required at all.

The reason which I can think of having a separate L2 and L3 port-channel might be from the need to have separate links for LAN redundant traffic and WAN redundant traffic so that either them of being physically down we have atleast one of them working fine. Such a case can be that upstream link of HSRP master breaks and we are not using track and same time the L2 link between the devices also break down and if now the L3 link was built on top of same L2 link we would have completely lost traffic. But if we have a separate L3 port-channel here it would save us a bad day.Again its per the design of the network and considerations for traffic separation which will count for the actual implementation.

Hope this helps you for your question

Regards

Varma

[Horacio Gutierrez]

Hello Varma,

I am bit confuse in your explanation. Let's say Varma my L3 channel goes down what will happen to L2 channel. Why do we need an L2 channel and especially L3 channel at all?

[Vaibhava Varma]

Hi Horacio

Lets consider this setup as below :

                           ISPPE1                                                       ISPPE2

                              !                                                                    !

                          WANLink1                                                   WANLink2

                              !                                                                     !

                          CoreSW1 -----------------------L3-Port_Channel-----CoreSW2

                                          -----------------------L2-Port_Channel-----

                             !                               HSRP_GW                           !

                             !                                                                          !

                                                             LAN_Server

Now for the LAN_Server to have a redundant GW we will be suing HSRP whereby CoreSW1 is Master. The HSRP Hellos will be passing via the L2-Port_Channel.And the WANLink1 is not being tracked under HSRP. Now LAN_Server will always go to CoreSW1 for Forward Traffic and if the WANLink1 goes down same time we have no upstream connectivity anymore. Therefore we need alternate L3 routing path via CoreSW2 which means we have two options:

1. Use a dedicated VLAN between CoreSW1 and CoreSW2 over the L2_PortChannel and run routing for same. But this is susceptible to loss of alternate routing path if the L2_PortChannel and WANLink1 are being down at same time

2. Use a dedicated L3_PortChannel between CoreSW1 and CoreSw2 which will provide one more level of redundancy for alternate routing path between CoreSW1 and CoreS2.

This is one scenario where the separation of L2 and L3 path between Core_Switches help to have uninterrupted traffic flow provided there is no HSRP Tracking happening. Again even sometimes when there is HSRP Track and the WAN Uplinks are passing via intermediate Transport media which are susceptible of detecting link faliures slowly the alternate L3 routing path will help us a lot. There is a use of BFD for Routing-Protcol faster failure detection as well as HSRP_Track.

So there are couple of ways for this but in my personal opinion alternate routing paths can for sure help us if the WAN Uplinks failure does gets detected slowly.

Hope this clarifies my viewpoint.

Regards

Varma

[Horacio Gutierrez]

Hello Varma,

So let me get this right the L2 Portchannel is dedicated for HSRP Hellos packets and the L3 Portchannel is used for packets being routed from core 1 to core 2 if the WAN link 1 goes down.  However, what if the L2 goes down and no more HSRP hellos packets are going accross where now core 2 is the active switch. Varma, will L3 Portchannel be able to still foward LAN server packets out the new active core 2 gateway with core 1 down even if the L2 Portchannel is down?

Thanks.

[Vaibhava Varma · Accepted solution]

Hi Horacio

Yes even if the L2-PortChannel breaks down between the Core_Devices the L3 traffic forwarding will still happen over the L3-PortChannel to the new Active Core-2 GW.

This is exactly where having a separte L3-PortChannel will provide uninterrupted traffic forwarding when both the L2-PortChannnel and one of the WAN Uplink is down. The HSRP can't switch and the WAN uplink is down so the track is not working anymore and we need to have no blackholing of traffic and then the separate L3-Port Channel does the forwarding.

Regards

Varma

[Jon Marshall]

Varma

Yes even if the L2-PortChannel breaks down between the Core_Devices the L3 traffic forwarding will still happen over the L3-PortChannel to the new Active Core-2 GW.

If the HSRP hasn't switched then that assumes that the L3 vlan interface on core1 is still up for the server. Assumiing the active NIC is connected to core1 i don't think this will work ie.

server1 is in vlan 10 (192.168.5.10) and sends packet to a remote network 192.168.20.10

packet goes to core1 (vlan 10 HSRP active) and core1 does a lookup for 192.168.20.10 and sees a route via core2. Packet is sent to core2 and then to WAN2 and out to remote network.

Packet comes back to WAN2 and then to core2 with a destination of 192.168.5.10. But core2 has a L3 interface for vlan 10 so it sees it as a directly connected route so it can't route the packet back to core1, it has to switch it but it can't switch it because the port-channel is down.

Perhaps i have misunderstood what you mean by "HSRP can't switch". Do you mean HSRP on the core switches or HSRP on the WAN routers ?

Jon

[Vaibhava Varma]

Hi Jon

The L2-Port Channel has to be there always and having a separate L3 Port-Channel is more helpful for alternate routing paths rather than creating a L3 Link (using dedicated vlans) for inter-vlan routing or to reach remote network not connected to core switches.

As you have mentiobed above for the case under consideration of having the L2-Port-Channel going down and at the same time the Uplink WAN link also going down the reverse traffic would be still down in this particular scenario of Server being directly connected to Core_SW. This is applicable only when we are taking a scenario of Server directly connected to Core_Switches.

Lets take a case where Server is dual homed to an Access L2 Switch which is dual Homed to the two Core_Switches. Now HSRP hellos have two paths ie via L2-Port_Channel betweeen the Core_Switches and via the L2 Switch itself. Under Normal scenario of using STP with Core_Switch1 as STP Master, the downlink from Core_SW2 to Access Switch would be blocked via STP and  the moment the L2_Port_Channel breaks off ,both the downlinks to the Access_Switch are operational and the forward traffic will go via Core_SW1 as mentioned by you above via the alternate L3 Port_Channel and also now for reverse traffic Core_SW2 will have the Server MAC address learnt via its direct L2 Downlink to Access_Switch and traffic will keep flowing.

So thinking more on this I think  its better to have a dedicated L3-Port Channel such that even If the L2-Port_Channel is down the alternate L3_Port-Channel can ensure uninterrupted traffic flow.

Regards

Varma

[Jon Marshall]

Horacio

The L2 potchannel is not just for HSRP eg.

server1 in vlan 10 connected to core1 and core2. core1 is HSRP active for vlan 10.

server2 in vlan 11 connected to core1 and core2. core2 is HSRP active for vlan 11

server1 active NIC is connected to core1

server2 active NIC is connected to core2

server1 sends packet to server2. packet goes to core1 because that is vlan 10 HSRP active and this is the active NIC for server2. core1 routes the packet onto vlan 11. Then does a mac-address lookup for server2. Because active NIC on server2 is connected to core2 the packet is switched across the L2 porchannel to core2 and then sent to server2.

return packet from server2 - packet goes to core 2 because vlan 11 is HSRP active and active NIC is connected to core2. packet is routed by core2 onto vlan 10 then switched across portchannel to core1 and then sent to server1.

Wihout the L2 portchannel the 2 servers would not be able to communicate. The L2 portchannel is there for inter-vlan routing on the core switches.  So, no L2 portchannel, no inter-vlan routing.

The L3 etherchannel is used for routes received from the WAN. It is purely there for reachability to remote networks, ie networks not on the core switches.

Personally i wouldn't design it this way. I would connect each WAN router to both core switches used L3 routed links then there is no need for a L3 etherchannel.

Jon

[Vaibhava Varma]

Hi Jon

When we say of connecting the WAN routers in a cross-linked way to the Core_Switches do we mean to connect the ISP_WAN_Router to both core_switches. If they are ISP_Routers I think this might be not an optimal consideration as we need to incur the cost for the 2 extra links we need to setup here which could be easily avoided using an L3-Port Channel.

Do you agree on this ?

Regards

Varma

[Jon Marshall]

Varma

Yes i did mean connect each WAN router to both core switches rather than run a L3 etherchannel.

I agree there is an extra cost involved but as always it comes back to cost vs redundancy. If you have paid for 2 MPLS connections then the cost of an extra ethernet interface in each router is minimal and why would you not want to use both links eg.

with a L3 etherchannel in place as in your setup the HSRP active switch only sees one path to remote locations across the MPLS network. If you connect each router to both switches then the HSRP active switch sees 2 equal cost paths and can use both links.

The L3 etherchannel is a perfectly valid design but if you are paying for both MPLS links it makes no sense to only use one of them for outbound traffic.

If you are only using one of the links and the other is a backup then yes the L3 etherchannel is a more valid choice. 

Jon

[Vaibhava Varma]

Hi Jon

I agree with you if we have no cost limitations we do not need to have the L3-Port Channel and just dual-home each core Switch to botht the ISPs but I would say from my personal experience that if we can achieve the required redundancy without incurring any extra capex/opex its always good to do that.

Thanks a lot for the great insight and a valuable discussion.

Regards

Varma

[Jon Marshall]

Varma

but I would say from my personal experience that if we can achieve the required redundancy without incurring any extra capex/opex its always good to do that.

Absolutely, couldn't agree more. Every design is a tradeoff between cost and functionality. I was just trying to say that sometimes it can be a false saving but again that does come down to the design requirements,

Good discussion, i agree.

Jon

[Horacio Gutierrez]

Hello Jon,

I thought L2 PortChannel was only use on Layer 2 switches not Layer 3 switches so how would your design work. Where would you do your inter vlan routing.  I guess I am bit confuse on the inter Vlan routing.  Can you give me an example.

Thanks.

The L2 potchannel is not just for HSRP eg.

server1 in vlan 10 connected to core1 and core2. core1 is HSRP active for vlan 10.

server2 in vlan 11 connected to core1 and core2. core2 is HSRP active for vlan 11

server1 active NIC is connected to core1

server2 active NIC is connected to core2