Combining two MPLS circuits - A routing question.

[fahim]

In the attached diagram I have tried to make our apathy simple. I'll try to explain the diagram here and then relate my question in hope that malady becomes clearer :)

Consider this, we had one MPLS provider all over our offices and life was simple. Suddenly someone somewhere made a decision to reduce dependence on a single service provider and brought in a second MPLS Service provider leaving us techies to dump our heads in order to architect around the new scenario of connecting offices on separate MPLS clouds.

Hence, soon we'll have two MPLS circuit providers, termed in the diagram as BT and ETISALAT, or simply, provider A and B. The two routers attached to the MPLS cloud are not under our administration but would reside in our premises; we'll call it our Main converging Site A.

Both the MPLS circuits terminate in our datacenter of Site A which has a pair of Cisco 6509E with Sup 720 working at the distribution layer.

On the BT MPLS cloud and Etisalat MPLS cloud are different sets of offices and some of the times, the need of an office connected to Etisalat MPLS is to directly talk to another office on BT without having to do anything with our office or enter our internal LAN.

So what we decide is, first to optimise by placing a WAN optimiser (could be Riverbed, Cisco, Bluecoat etc.. not yet decided). Wan optimisers do not yet have the capability route the traffic neither are meant to.

Now if someone attached to Provider A needs to go to someone on Provider B bypassing Site A, I need some sort of a policy based routing somewhere. Hence I am thinking of bringing in some UTM appliance, capable of undertaking PBR. Fortinet shown in the diagram is for illustrative purposes as it yet does not do PBR either. The idea of placing a UTM there is, that way, I'll also be able to check/filter malware coming into my network from my other offices (just in case).

The question is, am I missing something here? Would this concept work? Has anyone been there done this before? What if I terminate the two links directly on my 6509s and do some MPLS related stuff?

Pls advise!!

[Mohamed Sobair]

Hi,

This concept will work, but will definetely causes overhead and a single point of failure since the terminating device is your internal Fortinet box.

I would choose designing (peer Architecture) model of connecting BT MPlS router and Etisalat MPLS router. This will require one of the following:

1- (Option 10 A)Back to back VRF connectivity between PEs: the Simplest and widely deployed Scenario, requires PEs to have inter-VRF connectivity and doesnt require an EBGP session to carry the Labeled VPNv4 routes. thus Enabling Etisalat talking to BT with normal MP-BGP redistributed routes. although this option is simple and widely deployed but it doesnt Scale enough on Certain requirment.

2- (Option 10 B): requires both ASBR PE routers to have VPNv4 session to carry labeled VPNv4 routes from one AS to another AS.

3- (Option 10 C): relies on option 10 B, requires VPNv4 session between Route-reflectors, and would therfore needs EBGP-multihop configuration. The nexthop of the learned VPNv4 routes should remain unchanged. Its the most Scalable option but it's not widely implemented like Option 10B.

I would therfore request both providers to have one of theses Options to have interconnectivity between the sites attached to both providers without utilizing or hitting my internal Network Inorder to prevent single point of failure as well. The ideal Design would be having direct inter-connectivity between both router , this would also be easier to provision.

HTH

Mohamed

[fahim]

Thanks Mohamed.

Given that tho routers in my premises are of 1800 series, I wonder if VPNv4 sessions can be brought right upto the last mile. Technically, what would I need to ask from the two ISPs in particular in order to deploy Option 10b?

Second; how would I connect these two routers to my network in order to tap the traffic that is meant to come into my network from the two MPLS circuits?

[Yasir Ashfaque]

Well for me, this design is very simple, but you will have to sacrifice for bandwidth and a little change in design.

Because only scalable design which will make it work is HUB and SPOKE, u can't make it like that some of your offices directly talk from Provider A to Provider B, ( for that you will have to include your Service provider for that and tell you, troubleshooting in those scenarios become very tough, cause sitting in Head Office and troubleshooting two different offices which are directly communicating and not going through your circuit will make you pain in * :P

So all u need to do is make Head office a HUB, and all of your other office will be SPOKES, in this way you wont have to worry which SPOKE is in which ISP's Network, Your both ISPs will be importing your HUb's Core subnet in SPOKE's VRF and all offices communication would be going through your Core Network, in this way management and troubleshooting will be very easy and you will have command to find issue very quickly rather calling your ISP guys again and again.

[jayanthan]

For me the design and the concept seems simple but i feel the following points make it more easy.

BO - Branch Office

HO - Head Office

CE - Customer Equipments of Service providers - SPs routers at HO site

1) We need to interconnect both SPs CE routers. - ICRL link - need the support of both SPs

2) This scenario will be utilized by a number of branch pairs. The branch use this design will be connected to one of the SP only.

3) So the Traffic from HO to that perticular BO always need to use one of the SPs CE router in the HO premises. So a PBR is not needed here as either traffic from the MPLS (destined to the particular BO) or the traffic from HO LAN (destined to the particular BO) always need to exit via one of these two CE routers.

4) Each CE routers has all BO's LAN subnets in its routing table either routed via its WAN or via the ICRL link.

5) If a branch is connected to both SPs, then it should select the correct SPs WAN link to send the traffic to a BO.

[fahim]

I am sorry Yasir, but I am not able to picture your suggestion to it's completeness. From I gather is that our current design is indeed hub and spoke with our HO acting as hub. How would I configure my Spoke's to participate as VRFs?

I do agree with the aspect that we'll have to somehow import our subnet's to respective router's of ISPs but routers being not under our administrative previlege, what could be the easiest way to do that?

Lastly, how do I bring in and stop the traffic which is meant for HO office and let the other go through and through.

Pls advise!

[Laurent Aubert]

Hi,

I would use a CPE with VRF-Lite feature. It means you configure it like a PE but without MPLS and MP-BGP.

So you configure two VRFs on the CPE (UTM box in your drawing), one for each MPLS cloud. You will need to learn the subnets of the remote sites from SP routers and redistribute them into BGP (VRF address-family). Now you can locally import/export routes with or without filtering between both VRFs and send to SP routers the subnets of the other MPLS cloud.

Last point is to add the links connected to the 6500 in the VRF connected to the MPLS cloud from where the remote site need to have access to your local ressources.

To secure the exchange, IOS support CBAC with VRF context.

HTH

Laurent.

[Yasir Ashfaque]

Yeh, its a fine solution too

[Mohamed Sobair]

Hi Fahim,

sorry for the delay reply,

Ok, both of the bellow could be valid solution for your criteria:

1- back to back VRF, as I mentioned its the simplest and widely deployed Option , the provider needs to create 2 VRFs and controls routing by applying normal Import and export policies.

2- Option B, I would suggest direct link between BT and Etisalat since the all transit links has to be MPLS enabled, then creats an EBGP VPNV4 session between both routers thus having both Sites connected without going through the Fortinet.

Note: The Simplest way is to have VRF-lite and control Routing based on the routing policies applied. This could be achieved also with your current design, and doesnt require MPLS enabled in the transit link betweeen BT and Etisalat " The Fortinet doesnt have to be MPLS enabled".

HTH

Mohamed