Two redundant links between two L3 switches

Lovro Lazarin · ‎07-29-2010

Hi all,

First of all, hello to everybody in the community :-)!

Now the problem. I have two switches on two different locations. switches are 3750 and 3560, connected with two links- P2P fiber link (C1), and also some kind of P2P link (C2), but data is passing through ISP cloud.

The switches are in two separate VTP domains, both have their VLANs, we only need connection from one side to other, and route to specific networks, we configured static routes for that. Also for that purpose is crated on one side VLAN55 and on the other side VLAN5. With one link C1 everything is working fine, but when we connect other link C2, we have network loss and no connection to the other side.

So to the question.. what is the best way to configure this links to be redundant?? I understand that there is two approach, on the L2 (STP) or on the L3 (OSPF,EIGRP).

Also any advice on troubleshooting would bi nice.

Thanks in advance and kind regards,

Lovro

Jon Marshall · ‎07-29-2010

Lovro

How were the links configured when you connected up C2 and did you lose all connectivity on both links. If so were you connecting as L2 and running STP ?

If the links are the same speed and you do not need the same vlan at both sites then i would go for L3 routed links and use EIGRP/OSPF to have equal cost load-balancing over both links.

Jon

Lovro Lazarin · ‎07-30-2010

Hi Jon,

Tnx for the reply, and I have to say that I'm only CCNA, i'm little rusty on configuring routers and switches, and this is the first time I'm configuring this kind of communication.

No special configuration was made for those two links, Access mode, and VLAN only. The STP is enabled and it is in PVST mode. When we connect C2 we lose connectivity on both links. But also I did a little test and when I connect only C2 there is no communication with remote location. I have to check with ISP if this link is up, if it's not up it would be really hard to explain why is there loss of connectivity on C1 when I connect C2?

Router1

VLAN 5

IP address 10.28.18.1

__________

Router2

VLAN55

IP address 10.28.18.10

On both sides both links are in the same VLAN, router1 in VLAN 5, and router2 VLAN55

How would you configure load balancing ? Can you please post some config examples?

Tnx,

Lovro

Lovro Lazarin · ‎07-30-2010

Update..

Both links C1 and C2 are L2!

Debugging the problem with ISP we found out why there was a problem with connectivity on C2. The switch in their cloud is configured max mac count to 5. So when switch learns 5 mac addresses, the rest is blocked.

I need help how to configure our switches to ''represent'' it self with just one mac something like router, or somehow block mac addresses, so that ISP's switch don't fill it's maximum of 5 MACs?

am I making any sence :-) ?

Thanks,

Lovro

Nagaraja Thanthry · ‎07-30-2010

Hello,

One solution would be to go for a routed networks.Convert the ports connected to the ISP as L3 ports (no switchport command) and then give an IP address on each end (eg: 10.1.1.1 and 10.1.1.2 on respective sides). Now, turn on routing on both switches and run a routing protocol (RIP/EIGRP/OSPF). Now the ISP switch will see the MAC of the port that is directly connected. Only drawback of this solution is that you will not have layer 2 connectivity between devices in remote location and your location.

Hope this helps.

Regards,

NT

Eric Boadu · ‎07-30-2010

Lovro,

Have you consider trunking between the two switches? If not, then try this solution. Assuming each site has its own internet access and/or you need connectivity between your LAN via fiber p2p link.

1. On your L3 switch at C1 and C2 (routed port) assign layer 3 subnet and route your VLAN IP schema via that link. Example: Gi0/48 ip address 10.10.10.1 255.255.255.252 for C1 and Gi0/48 ip address 10.10.10.2 255.255.255.252 for C2. You should be able to ping C1 and C2 L3 address now from your switches.

2. On C1 switch: IP route 0.0.0.0 0.0.0.0 192.168.16.x <

3. On C1 switch: IP route 10.28.0.0 255.255.255.0 10.10.10.2 for your L3 subnet at C2 switch

4. On C2 switch: IP route 00.0.0 0.0.0.0 192.168.24.x <

5. On C2 switch: IP route 10.28.0.0 255.255.255.0 10.10.10.1.

6. This will solve your issue and both sites should be able to access internet and still access each other LAN via the 10.10.10.x L3 link.

7. Hope this help please rate this post.

Eric

waltermavely · ‎07-30-2010

Hi

Let me brief your both site there is a layer 3 cisco switch its connected through ISP as L2 link

Site-1 Site-2

c1(fa0/1)--------------------------------c1 (fa0/1)

3750 3560

c2 (fa0/2)--------------------------------c2 (fa0/2)

Hope this is your scenario

If the link speed is the same for all interface why dont your bundle the port with etherchannel

here is the config L3 etherchannel

Site-1

interface range fastethernet 0/1 - 2

no switchport

channel-group 5 mode on

interface port-channel 5

no shut

ip address x.x.x.x

Site-1

interface range fastethernet 0/1 - 2

no switchport

channel-group 5 mode on

interface port-channel 5

no switchport

ip address x.x.x.x

then config routing

L2 etherchannel config

Site-1

global config mode

Vlan 5

interface range fastethernet 0/1 - 2

switchport

switchport mode access

switchport access vlan 5

channel-group 5 mode on

interface port-channel 5

switchport

switchport access vlan 5

switchport mode access

interface vlan 5

ip address x.x.x

Site-2

global config mode

Vlan 5

interface range fastethernet 0/1 - 2

switchport

switchport mode access

switchport access vlan 5

channel-group 5 mode on

interface port-channel 5

switchport

switchport access vlan 5

switchport mode access

interface vlan 5

ip address x.x.x.x

then config routing

hope this is helpfull for you

regards

walter

Lovro Lazarin · ‎08-02-2010

Tnx all for helping me on this.

Sorry for the late response, I'm on my holidays, so it will have to wait till I come back to work to try your suggestions.

I have to say that most likely I'll try to go with etherchannel solution, or L3 suggestion from Nagaraja.

@Eric, we have a little more complicated topology so that is not possible..

@Walter, thanks for posting config example.

I'll get back to you with results, when I go back to work, and that would be around 15th of august :-)

Regards,

Lovro

P.S. Nagaraja, I excidently gave you only 1 star, looks like I can't correct that:-(

Nagaraja Thanthry · ‎08-02-2010

Hello Lovro,

Not an issue. As long as the solution helps, that matters.

Regards,

NT

Lovro Lazarin · ‎08-26-2010

Hi all,

Finally I managed to find time to try your suggestion. I tried with L2 andL3 etherchannel, non of them worked like it should. I also attached networkscheme for better explanation. Here are the problems that I encuntered.

- L3 Etherchannel -

* I configured everything by the book, fallowing your and instructions from ciscohttp://www.cisco.com/en/US/docs/switches/lan/catalyst3560/software/release/12.2_25_se/configuration/guide/swethchl.html#wpxref25850. After that, I added static routes, but strange thing I couldn't see them in ship route static! Anyway.. I managed to get communication on C1 link. Now the statuses of the ports were Gi 0/11 <P>as bundled in Ethc, and Gi 0/2 <s> as suspended.

* I tested connectivity by manually disabling port. Problem was that traffic was not routed on C2. In fact I couldn’t get connectivity on C2. I disabled Gi0/11 and also Fa 1/0/2 on the other side, but nothing.

* When I enabled ports on both sides, the connectivity was not restored. I had to disable Gi0/2 and Fa 1/0/6, so that other ports get back in the

state. Only then communication was restored. So there was no automation!

- L2 etherchannel –

* Same as previous, after I configured ethc, I added static routes, but now the routes was visible in routing table ??

* When testing connectivity a managed to get communication on both the C1and C2 line. But again, with no automation. I had to disable one port so that other becomes

, on both sides.

Finally I think that etherchannel is not working because if you examine network scheme C2 is passing trough ISP equipment. Maybe PAgPpackets was blocked, or something like that.

Any way I decided to try with simple L3 routing. Now…, I would much appreciate if someone could post config example based on the network scheme that I provided.

Thanks to all that are trying to help me on this, and I apologize for such long and exhausting post :-)

Kind regards,

Lovro

Lovro Lazarin · ‎09-07-2010

Anyone?

Any suggestions ?

Thanks and regards,

Lovro

mdavid.cisco · ‎06-10-2020

With your two L3 switches you could try connecting it using the example below:

C3750 C3850

SW1 g1/0/1 ----------VLAN 100------------- SW2 g1/0/1

SW1 g1/0/2 ----------VLAN 200 (cost: 40)---- SW2 g1/0/2

Configure the switch ports as trunk ports. Setting the native vlan respectively as shown above. You can then restrict what other VLANs to go through the trunk for L2 traffic (say if you want a specific vlan that's on SW1 to stretch to SW2). Then for L3 make sure you create SVI for VLANs 100 and 200. Enable a routing protocol e.g. OSPF and use the SVI as interface to advertise networks. You could also apply OSPF costing on the SVI to artificially influence traffic e.g. you could put a cost on SVI VLAN200 of 40, making the SVI VLAN100 as the preferred path. Essentially making VLAN 100 as the main link and the VLAN 200 as the backup. You could even expand this beyond a point-to-point connection to a many-to-many using a single OSPF Area.

HTH