Layer 2 extension between 3 datacenters with load balancing

[sandu_nastas1]

Hi there,

The scenario is the following : there are 3 distant sites that need to be interconnected as in the image attached. On every site - there are 2 layer3 switches. 2 of the sites are interconnected via 2 telecom links an the third is connected to each of the 2 using one link.

The first 2 sites are datacenters and the third is a user site.

The question is how to use all the links in this topology in load balancing mode and avoid creating loops. If we enable Spanning Tree - it will brake all the loops, thus leaving only on channel between each site in active.

The price of these channels is very expensive and we would like to use them in active/active environment

Any help would be appreciated

[Reza Sharifi]

Hi,

If you have 2 layer-3 switches in each location, why do you want to extend layer-2?

If these are layer-3 switches, you can simply use a /30 between each router/switch and use a routing protocol like OSPF.  In this case, you don't have to worry about loops.

HTH

[sandu_nastas1]

Hi,

Thank you for the reply

The reason for extending Layer 2 is data replication and Clustering. We have some clusters that need to be in the same broadcast domain for high availability reasons. So there is no question about layer 3

[Jerry Ye]

OTV can accomplish what you are thinking. Please take a look at the following link:

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9402/solution_overview_c22-574939.html

Regards,

jerry

[sandu_nastas1]

Heh, Thanks for the response. Nice article.

The problem is, though, that OTV exists only on Nexus and 6500 series. And we'd like to use 4500 and 3750 switches, cause the network isn't this big (about 15 servers and 150-200 users)

I even thought about VSS, but it's also maintained by these 2 platforms only.

Any other ideas ?:)

[Jerry Ye]

If this is a cost concern, you can try the new C4500-X. Data sheet said it supports VSS (I have never tried it).

Regards,

jerry

[Mr_Helpful]

What is the current layer-3 topology ? Where is routing taking place; I'm sure you don't have only one large VLAN spanning all three sites.

For example: what is the gateway for your servers in both DC's to get to the clients, and what is the gateway for the clients to get to the DC's ?

[sandu_nastas1]

John, the routing takes place on the switches. You're right, there isn't just one vlan spanning all three sites. There are about 10 vlans in the whole network. But they need to be spanned across the interconnection trunk links, so there would be the possibility to have for example VLAN 10 devices on each site and they'd be in one broadcast domain. Even if they are geographically distant. There are, for instance, server clusters, with VRRP adresses that should be in the same vlan.

[Mr_Helpful · Accepted solution]

Well, depending on the circumstances and your business policies, it might be an option to use different switches as the default-gateway in the various VLANs. It is not a very clean solution, but depending on your needs it may work.

I mean... suppose you have clients in VLAN 5 and 6 in the bottom location, and their default gateway is the IP-address of one of the datacenter-switches. Instead of haveing both VLANs point to the same datacenter, give the siwthc in the left datacenter an IP in VLAN5 that is the gateway-address, and let the right datacenter handle VLAN6. Sure enough, all relevant switches have an IP address in all VLANs, otherwise you can't route. It's just the VRRP virtual addresses that you want to spread.

Again, depending on your precise topology, this might be a nightmare-scenario, or it might not even accomplish any load-balancing at all. So this advise comes without warranties...

But look at some statistics (I am not a mathematician, so don't trust this reasoning) assuming you have spread your servers equally over both datacenters :

1. if you are actively using only one link from the bottom client-site... about 50% of the traffic arriving at the datacenter will be destined for the other site, crossing your datacenter-link.

2. if you are actively using both links from the bottom client-site... still about 50% of the traffic will arrive at the wrong datacenter, so still 50% of your traffic will cross your datacenter-link.

Now, if you can do anything to optimise traffic-flow from the client to the right datacenter (maybe in layer 3?), you may be able to lower that 50%, making it actually worth the effort to use both links.

Point being: in my opinion it is not by definition 'more optimal' to use links active-active instead of active-passive.

Long story, but I hope you get the idea I tried to explain...

;-)

[sandu_nastas1]

Hi John,

This may be worth trying. Although it seems a little too complicated. My idea was to adjust spanning tree priorities in order to force STP block different ports for different VLANs.

But anyways, I think we'll use the VSS with 4500s that should be available later in 2012 ( I hope).

About the traffic, we'll see how many active/active clusters we have. If 90% of the servers will be active/passive, it's obvious that there's no reason to implement load balancing. But I'd like to have a topology that would permit to implement it when there's a need, without major changes.

But thanks for the point though