Redundant link between two core sites with MPLS

[louis0001]

Hi,

we have 50+ sites connected via MPLS. (see diagram)

Out core sites are SITE A (primary) & SITE B (secondary). These core sites are connected via a ptp link.

Both sites connect to the MPLS cloud via their own routers and all sites can reach SITE A & SITE B.

Now, if I pull the cable on SITE A's MPLS router, nothing can reach SITE A as expected.

Is it possible to get all of the MPLS sites to reach SITE A via SITE B's MPLS router and PtP link (and vice versa if SITE B's MPLS router went down)

Basically, redundancy if one of the MPLS routers went down at one of the core sites?

[milan.kulik]

Hi, IMHO, you can easily configure static routes for site A prefixes on your MPLS router in site B. Using the site B Layer 3 switch as next-hop. And advertise these static route prefixes to the MPLS backbone with worse BGP attributes than the same prefixes advertised from site A. You can prepend the site B AS number 5 times to them, e.g. So as long as the same prefixes will be advertised from site A with better attributes, they will be used by other sites to reach site A. At the moment the MPLS router in site A will stop to advertise the prefixes, the same prefixes advertised from site B (worse attributes) will be used. Of course, you can configure site A the same way to advertise the prefixes of site B with worse attributes. You can test easily by creating some testing subnet in one site and configuring the routers to advertise it the way I suggested. Best regards, Milan

[louis0001]

Hi, thank you for the quick reply.

I'm unaware how to apply worse BGP attributes as I've never done this before. Could you provide an example please?

[milan.kulik]

Hi,

here is one example:

http://routerjockey.com/2011/02/28/bgp-essentials-the-art-of-path-manipulation/

Best regards,

Milan

[louis0001]

HI,

I've been having a little play on packet tracer with this using ospf. I'm not sure how to recreate the mpls network so I just created some dummy locations substituting ospf for mpls/bgp.

The 2 core routers (static routing and ptp link between them) send all traffic to the mpls router apart from traffic to themselves which traverse the ptp link. This is all done via static routing.

So, if my mpls routers advertise the other sites via bgp, where would I place ospf on the other routers? Would I be right in thinking that i would place ospf on the mpls routers and the core routers? I then place an ospf cost on both sides of the ptp link which forces all traffic to go the mpls route just in case ospf finds the odd route might be better through the ptp link.

How do I advertise the bgp routes to the ospf routes? eg the mpls router has 10.0.0.0/16 in its tables, do i just add ospf and place network 10.0.0.0/16 area 0 on the mpls routers?

So basically, it would be:

Core routers = ospf & static routes between the core routers & Mpls routers on same site

MPLS routers (on same site as above core routers) = ospf between mpls & core routers. Bgp to all other remote site mpls routers

MPLS routers (on all other sites) = just bgp between all other mpls routers.

[milan.kulik]

Hi,

I'm sorry I don't understand now.

So far yiou were talikng about BGP/MPLA and static in tyour network only.

Why do you need OSPF now?

Where is it running exactly?

Can you add it to your diagram?

Best regards,

Milan

[louis0001]

Hi, many thanks for your reply

in the diagram, our servers are at Site A & Site B. The layer 3 switches there use static routing to reach each MPLS router and the MPLS routers have all of the routes for the other mpls sites.

Only SITE A communicates with SITE B via the ptp link.

If I turn off the MPLS router at SITE A, no servers SITE A will be able to communicate with any of the MPLS sites as the static routes on the layer 3 switch at SITE A sends everything to the MPLS router at SITE A.

What I would like to do is if the MPLS router goes down at SITE A, the layer 3 switch at SITE A will send all traffic through the ptp link to the SITE B layer 3 switch which will then send everything out to the MPLS router at SITE B to reach the other MPLS sites.

The two layer 3 switches at both sites do not have mpls/bgp on them and currently have static routes only. I'd like to try and make these dynamic so that if one of the MPLS routers goes down, traffic is re-routed to the other MPLS router. That's why I was thinking of ospf to do the job. Hope this is clearer? 

[milan.kulik · Accepted solution]

Hi,

yes, it's clearer now.

So what I'd think about would be running OSPF between the MPLS router and L3 switch on each site.

You would redistribute BGP prefixes to OSPF then.

And also configure a default route on the L3 switch pointing to the other L3 switch over the ptp line.

Plus keep the static routes for the other core site prefixes pointing to the other L3 switch over the ptp line.

That way:

1) Under normal conditions the L3 switch in site A will use a static route over ptp line (better AD than OSFP) to reach site B and vice versa.

2) In a case of MPLS router A failure:

The outging traffic from Site A would be routed via the ptp line to the L3 switch in site B (due to the default route). The L3 switch in site B will know the routes to the remote MPLS sites (received via OSPF from MPLS router B).

The incoming traffic will be received on the MPLS router B (due to prefixes for site A advertised from B with worse BGP attributes - but no prefixes advertised to MPLS from A). The MPLS router in Site B will use its static routes to forward the traffic to the L3 switch in site B. And the L3 switch will use its static routes for site A prefixes to forward the traffic over the ptp line.

Analogic routing will be applied in a case of MPLS router failure in site B.

3) In a case of the ptp line going Down:

The static routes should disapper from the routing tables on both L3 switches.

But they should still get the routes to the other core site received from OSPF (redistributed from BGP).

4) So the only problematic case might be the ptp line failing but the L3 switch interface remaining Up.

You might need to get either some tracking (not sure if available on L3 switches) or dynamic routing involved to overcome this.

Either running OSPF between the L3 switches  or even runnig EIGRP to route the site A and B prefixes only possibly?

But I'm not sure if this wouldn't bring too much complexity to the design?

Best regards,

Milan

[louis0001]

Thank you very much for you reply. I was thinking of doing the following:

OSPF on the core switch & MPLS router on  both site A & B with static routes for site A to B and vice versa.

Then placing an OSPF cost on both sides of the ptp link to force all traffic to the MPLS routers in case ospf kicked in and sent some MPLS traffic from site A via the ptp link to site B and then out via site B's MPLS router.

So:

SITE A to SITE B traffic would go through the ptp link as normal (static routes)
SITE A to SITE B ptp would haveospf with an ip ospf cost to prevent ospf being used whilst the MPLS routers were up.
SITE A to other MPLS sites would go through SITE A MPLS router only (ospf)
SITE B to other MPLS sites would go through SITE B MPLS router only (ospf)

In event of MPLS router failure, traffic is redirected (ospf) via ptp link to opposite sites MPLS router and out that way.

I haven't yet thought about the ptp link failure although it's not as critical as the MPLS router failure in our case or a core switch failure 

[milan.kulik]

Hi,

I'm not sure if understand your idea 100%.

I still suppose BGP prefixes would be redistributed to OSPF from the MPLS router A and B?

I'm afraid running OSFP over the ptp line together with static routes for site A to B and vice versa might bring troubles:

In a case the ptp line would fail but the interface would remain Up, the static routes would still be used possibly (even while no OSPF prefixes would be received).

So if you want to handle this case, you would need dynamic (OSPF?) routes only received over the ptp line.

But tuning OSPF metrics might be quite complicated.

You would need on site A:

- prefer the site B prefixes received over the ptp line directly over the same prefixes redistributed to OSPF from MPLS router A.

- prefer the remote MPLS site prefixes redistributed to OSPF from router A over the same prefixes redistributed from router B (and received over the ptp) line.

And symmetric metric settings for site B would be necessary.

That would be very complicated!

So if you don't see the ptp line failure critical, I'd keep two separate OSPF clouds per site and use only static routes over the ptp line.

Or you might think about using EIGRP over the ptp line instead of static routes to advertise only site A prefixes to site B (and the default route) and vice versa?

Best regards,

Milan

[louis0001]

Thank you very much for you reply. I was thinking of doing the following:

OSPF on the core switch & MPLS router on  both site A & B with static routes for site A to B and vice versa.

Then placing an OSPF cost on both sides of the ptp link to force all traffic to the MPLS routers in case ospf kicked in and sent some MPLS traffic from site A via the ptp link to site B and then out via site B's MPLS router.

So:

SITE A to SITE B traffic would go through the ptp link
SITE A to other MPLS sites would go through SITE A MPLS router only
SITE B to other MPLS sites would go through SITE B MPLS router only

In event of MPLS router failure, traffic is redirected via ptp link to opposite sites MPLS router and out that way.

I haven't yet thought about the ptp link failure although it's not as critical as the MPLS router failure in our case or a core switch failure