2 remote locations, 2 MPLS links between

[Damir Reic]

Hello,

maybe this is simple question for someone with MPLS experience. I have 2 locations that are connected with 2 MPLS links by 2 different ISPs. Some services go over one link, some over another link to remote location. I presume there is a static route on each server with next hop 1st or 2nd MPLS router so that's how traffic is divided. Let's presume that is truth and when 1 MPLS link goes down services on that link become unavailable. It's obvious i need to use dynamic routing protocol so path is corrected when one of the links go down. Also i need to make sure that after that broken link is restored previous path is auto restored. Which routing protocol should i implement?

[Bilal Nawaz]

Hello, If this is from the customer towards the provider, I recommend EIGRP or OSPF - some use BGP but I think its overkill. With this you can form a neighborship with the MPLS provider and their VRF's (assuming its MPLS VPN) will advertise the rest of your network to your router(s).

Any MPLS link that goes down, it will stop advertising the routes so you are bound to go and take the other path.

I may have misunderstood the question, but I hope this helps.

Please rate useful posts and remember to mark any solved questions as answered. Thank you.

[Damir Reic]

Hi,

our network -> provider1 router -> provider1 network (VPN) -> provider1 router -> our remote network

our network -> provider2 router -> provider2 network (VPN) -> provider2 router -> our remote network

This is how the network looks like. Since i didn't work with MPLS yet, can i from neighborship with this setup? Will i need to do some configuration on provider routers (i don't have access to them) ?

[Bilal Nawaz · Accepted solution]

Hello,

You will need to come to an agreement with the providers to enable the routing protocol so you can form a neighborship with them. Most providers do not have a problem with this. They will just enable the routing protocol - takes a bit of configuration from an MPLS point of view, but nothing that should concern you. All you are doing is enabling a routing protocol to learn routes dynamically.

Your topology would look like this...

The PE (provider edge) routers will be advertising you the networks. For example in the Remote network, you are advertising A.A.A.A to the PE routers (the service provider routers) and have formed a neighborship with them. They will then carry these routes in the MPLS VPN towards the other sites (Main site for example) on the left hand side. And the PE on the other side will advertise this to your router.

If connectivity were to fail, then the routes would stop being advertised anyway and you have a backup MPLS VPN which is also good. You can possibly loadbalance across both providers which is another benefit.

Hope this helps

Please rate useful posts and remember to mark any solved questions as answered. Thank you.

[Damir Reic]

Thank you!

Edit:

Load balancing is implemented "out of the box" with use of EIGRP?

[Kenneth Sharp]

Damir,

    Yes.  EIGRP supports equal-cost load balancing right out of the box.  Equal-cost meanins when the EIGRP metric is the same for different destinations. The EIGRP metric uses bandwidth and delay to calculate the metric by default.  Do a 'show interface g0/1' to see the values for bandwidth and delay on your device.

    For unequal-cost load balancing, add a  'variance ' command in your eigrp process.  This will allow eigrp to load balance across links with different metrics.  For example, a 'variance 2' command will allow eigrp to load balance across links with metrics that are up to double the smaller metric.  A 'variance 3' command will allow eigrp to load balance between links up to three times the smaller metric.

I hope this helps explain eigrp load balancing,

   - Ken

[Bilal Nawaz]

EIGRP puts up to four routes of equal cost in the routing table, which the router then load-balances. The type of load balancing (per packet or per destination) depends on the type of switching being done in the router. EIGRP, however, can also load-balance over unequal cost links.

Note: Using max-paths, you can configure EIGRP to use up to six routes of equal cost.

Please see here for more details:

http://www.cisco.com/en/US/tech/tk365/technologies_white_paper09186a0080094cb7.shtml#loadbalancing

Hope this helps

Please rate useful posts and remember to mark any solved questions as answered. Thank you.

[Damir Reic]

Got 1 more additional question, if i use my router on each side that is single point of failure. I can buy additional router and restore configuration from production router on it as a backup plan. Is there more elegant solution than this which would allow to auto kick in once router is down for any reason?

[Bilal Nawaz]

Hello Damir, good question, and good planning too. I missed out the resilience part of it all. This is what I might do if I was in your situation.
As a customer you could have Router per provider. Therefore you have a 'pair of routers' per site both would be active (one for Service provider 1 and another router for SP2) which will be providing the redundancy, they would be connected together and establish neighborship (EIGRP or OSPF, whichever routing protocol you decide). Then the routing protocol itself will take care of resilience and convergence.

This way you can also still load balance your traffic depending on routing protocol.

Hope this helps

Sent from Cisco Technical Support iPhone App

[Damir Reic]

Hello Bilal,

i don't understand what you are proposing here. Let's say i have 172.16.10.0/24 network on my side, and 172.16.20.0/24 on remote side. My default gateway is Fortigate firewall on 172.16.10.1. So if each provider has 1 router (with IP from 172.16.10.0/24), doubt fortigate firewall supports EIGRP (cisco proprietery) so i have to define 2 static routes (or OSPF if it supports it) on it to each of those routers which really isn't good solution (again single point of failure, but buying another fortigate firewall might even be acceptable (HA setup) even though it costs a lot since it's only internet gateway).

The ideal solution would be that i have 1 "central" router connecting each provider router and on Fortigate i define static route for remote network so router itself will handle load balancing and failover. But since i can't setup HA of 2 routers (at least not to my knowledge) only viable and afordable solution seems to have spare router with same configuration just in case one dies.

Any thoughts?

[Bilal Nawaz · Accepted solution]

Hello Damir,

Fortigate support OSPF, BGP and RIP for dynamic routing - gateway resilience can be dealt with by using VRRP for the internal networks.

http://docs.fortinet.com/fgt/handbook/fortigate-dynamic-routing-40-mr1.pdf

http://docs.fortinet.com/fgt/handbook/40mr2/fortigate-ha-40-mr2.pdf

What i was proposing was to have either two routers or FW's in your case that peer with the provider. A bit like this:

Enable VRRP for your gateway redundancy between firewalls, and enable OSPF between your provider. You will be advertising your subnets to them, so they can carry the OSPF subnets to the other side for advertisement, so its not necessary that you give them an IP in your own internal range, you can use different subnets like I have shown.

If provider 1 was to fail (Red), OSPF routes along this path would go out of the routing table of Fortigate 1. However, it would know how to route via the secondary link, because it would have routes to get there via OSPF the other firewall.

This way you have gateway resilience, provider resilience - and routing resilience. Green ovals representing OSPF neighborships from your point of view

Same can be achieved with Cisco routers.

Hope this helps.

Please rate useful posts & remember to mark any solved questions as answered. Thank you.

[Damir Reic]

Bilal,

thank you for all the info you've provide me sofar. I think i have all info now.

[Damir Reic]

Have more questions

On 172.16.10.0/24 side there is ASA5500 in HA setup on local network and then infront of it there is provider MPLS router. For other MPLS link there is user router inside local network and then infront of it provider router. I guess ASA does not have OSPF support and instead fortigate FW there will be Cisco 3560 L3 switch. So in this situation i can configure OSPF on 1 router and L3 switch, but not sure whould should i do with link where ASA is. On the link where there is user and provide router, do i need to call provider to enable OSPF on that router or no?

Also, is it possible to prefer 1 path (from two) just for 1 host inside 172.16.10.0/24 for a destination network or host on remote side (172.16.20.0/24) when OSPF is enabled.

Thanks

[Bilal Nawaz]

ASA's support OSPF too, not sure how they work in a HA state but i guess its just standard - I have configured OSPF on them before but not with active passive...

http://www.cisco.com/en/US/products/ps6120/products_configuration_example09186a00809a417a.shtml

http://www.cisco.com/en/US/docs/security/asa/asa90/configuration/guide/route_ospf.html

Is it possible to prefer 1 path from two?

Yes.

Possible to prefer 1 path from two for a specific host?

hmm...

I do need to think about this one. I guess you could advertise a /32 single host subnet to the OSPF domain, to one of the providers instead of both. When you do this - the more specific route will be chosen.

e.g. you advertise 172.16.10.100 which is your host as a /32 to provider 1. The more specific route is chosen as well as the path with less distance, this way you will be going via the provider you decided to advertise to. (guessing this is for testing? any specific reason for this?)

The reason why I suggest advertising another subnet (/32) is because if we try to manipulate OSPF the only way I know is via costs and other means which would subsequently effect all routes.

So you can have ASA's at one end and Fortigates at the other end, or have a mixture i guess, was this what you are planning? You can achieve same thing with routers or layer 3 switches.

Both doing OSPF which would work well I think...

Hope this helps

Please rate useful posts & remember to mark any solved questions as answered. Thank you.

[Damir Reic]

Well one MPLS link connects central and main remote location + 30 small offices other link connects only central and main remote location. Since atm static routes are used between main and remote location, 2nd MPLS link is under no load and is used for database replication only. After dynamic routing is implemented we would like to keep using 2nd link for database replication and in case it fails, database replication would take other path. Ofcourse upon link restore, database replication should revert back to 2nd link. If this can't be done, not a problem then we would rather load balance traffic between central and main remote location on those 2 links. Breaking OSPF and ability to loose auto path selection upon failure is not an option. OSPF will only be used between central and main remote location.

Fortigate 200B is main firewall on 172.16.10.0/24 network, but default gateway for all devices is actually Cisco 3560 L3 switch inside same network so i would have to configure OSPF between it, 2ASA in HA and 1 Cisco 1900 router (all owned by us). I am bit confused since provider is using their router (PE) after our networking gear so i am not sure if my plan where do i configure OSPF is correct or not.