Any way to bundle 2 different layer 2 isp connections

[Louis Adams]

Hi,

Please take a look at my basic diagram.  I have (2) layer 2 circuits form 2 different carriers connecting my catalyst 9000's in two different datacenters.  Currently, one is active and the other is in standby mode via being blocked by STP.  I am trying to figure out how to make use of both circuits.  When i issue a show cdp neighbor command I see the other switch, the carrier switches pass all layer 2 info.  The problem I see is if I where to bundle theses circuits in an etherchannel is (since they are not directly connected to each other), if a circuit fails somewhere down the line the switch would not see that the port is down and continue to send traffic out the port.  Is it possible to take advantage of both carrier circuits?

[Jon Marshall]

You won't be able to use etherchannel as they are different switches between your DCs. 

It really depends on whether you have common vlans in both DCs because if you do at best you could send some vlans down one link and some down the other. 

But if each DC has it's own vlans/IP subnets then you could route between the DCs and then you could use both links and with a dynamic routing protocol a failure of one link would just move all traffic to the other one. 

Jon

[Louis Adams]

Hey Jon,

Thanks for reply, yes they do have same vlans on both sides.  So if I can't do etherchannel won't vlan 1 cause a loop if i try to only allow vlans over one port vs another?  I am assuming VLAN 1 has to be configured to pass on both ports?  Wouldn't spanning tree block that port or would it be only blocked for vlan 1 and open for other allowed ports as long as they are different vlans.  

[Jon Marshall · Accepted solution]

You have to allow all vlans across both links and let STP block per vlan otherwise if one of the links goes down some of your vlans would stop working. 

But you should be able to use STP parameters to control which vlans use which link when both are up and although it isn't perfect by any means if you have a rough idea of how much traffic per vlan you may be able to balance the load fairly evenly. 

Of course some people argue you may not want to use both circuits if it means a circuit failure means the other one is then overloaded but if you are paying for both circuits regardless of the traffic on them you may as well use them both. 

Jon

[Louis Adams]

Yeah that makes sense, thanks for info.   I have also been told to look at a SD wan appliance for a possible solution.  

[Joseph W. Doherty]

As @Jon Marshall notes, Cisco's PVST would allow you to determine which VLANs are blocked per DC-to-DC link, but unblock if there's a link failure,  (More on link failure, in a moment.)

Additionally, MST, could also be used much like Cisco's PVST.  MST should be supported on your Cat 9Ks, and would also work with non-Cisco equipment (as it isn't proprietary as is PVST.)

". . . the carrier switches pass all layer 2 info."

You may want to double check that.  I know several years ago, Comcast's Metro L2 offerings did filter out a few (low level) frame types.

". . . if a circuit fails somewhere down the line the switch would not see that the port is down and continue to send traffic out the port."

You may also want to double check that - especially asking if it's an option.  (Several years ago, working with OTN providing L2 "p2p" links, OTN could drop the end connections if there was a break in path [it had to be enabled to do so, though].)

If a provider's internal physical link failure cannot be detected, I suspect STP (or its variants), may be "slow" detecting the path break (or its restoration), as your port doesn't change up/down or down/up.

Further, without your port having up/down changes, unsure how well Etherchannel would do in this situation either.

As Jon also mentions, if you convert the links to L3, then a dynamic router protocol could detect and route around a path break, and further, Cisco routing often supports ECMP, so you could use both links (with possible a better load balance than STP, which will only control whole VLANs  [i.e. Etherchannel or ECMP are per flow, not per VLAN]).

[Louis Adams]

Thanks very much for the Information.  Helps alot.