Routing loop concern

[Vishnu Reddy]

Hello all,

We have an MPLS setup with all remote sites connected to site X and site X is directly connected to site Y(datacenter). Note: site Y is not connected to MPLS cloud till now. All the traffic is passing from site X to site Y for accessing servers at datacenter. site X and site Y are connected to each other through Metro E. Now we have new circuit thats being provisioned for site Y(datacenter) that is going to be connected to MPLS cloud directly. We do bgp peering with ISP for MPLS connectivity and Internal routing protocol is Eigrp. We are doing mutual redistribution at each location for propogating routes at each site.  Now the concern is when site Y is connected to MPLS and when I will do the route redistribution on site Y with MetroE connected to site X will there be a routing loop occurance or will there be a best path selection based on the metrics calculated and will choose the optimal path.

Need suggestions. Thanks in advance.

[Jon Marshall]

Are you using the same AS number for each site or do they use a unique AS number per site ?

Do you want site X to still use the dedicated link to get to Y or do you want it to use the MPLS network ?

There are a number of issues that could occur depending on the AS number(s) in use at your sites.

The one issue you definitely have is that X and y are exchanging internal EIGRP over the existing link. If you want X to use MPLS to get to Y that could be an issue because the EIGRP routes for Y within X via MPLS will be EIGRP external so the direct link is preferred.

Note the above assumes X has a L3 switch behind the MPLS router where all the local vlans/IP subnets are routed.

If X only has the MPLS router and clients use that as the default gateway then the opposite could happen ie. traffic from X to Y will use MPLS even if you want to use the direct link but that is dependant on the question about AS numbers.

So can you clarify that and also how you expect traffic to flow between X and Y.

Jon

[Vishnu Reddy]

Thanks for reply.

They have unique private ASN no. for each remote sites. We want to retain the site x to site y metroE connection while the mpls circuit for site Y is provisioned for mpls.

site X(campus) has layer 3 switch with different vlans that access site Y(datacenter) through MetroE.

site X has one 7k as the core and site Y has 2 7ks for the core for layer 3 routing. site X is connected to each 7k at site Y through MetroE.

Also site X and site Y will have different private AS.

That's ok if the site X still wants to go to site Y through MetroE but I am worried whether there would be any routing loops with this senario.

Offcourse MPLS provider will have one public ASN that will be peering with each remote sites to acheive this.

Thanks in advance.

[Jon Marshall]

From the L3 switch perspective in site X and Y, assuming they are exchanging EIGRP they will continue to use the dedicated links because these should be EIGRP internal whereas the routes for the same subnets received by BGP will be redistributed into EIGRP and be EIGRP external.

The MPLS routers at each site though will see the best path via MPLS ie. BGP AD 20 is better than EIGRP 90 but it won't matter because traffic going to these routers will be for remote sites.

In terms of the remote sites you may want to filter X's subnets from the EIGRP to BGP redistribution at Y and Y's subnets from X's redistribution.

Otherwise you may find that remote sites use X to get to Y and Y to get to X.

However if you want X and Y to back each other up in case of an MPLS failure at either site then you can advertise out each others sites subnets (as well as it's own obviously) but use either MED or AS prepending eg.

X advertises it's own subnets and Ys and uses either MED or prepending on Y's subnets

Y does the same in reverse.

Jon

[Vishnu Reddy]

Thanks for the reply Jon.

From this what i can derive is there would be possibility of suboptimal routing if I am not filtering subnets from X at Y and Y at X, but no routing loop.

Secondly for backup for X and Y in case mpls goes down at either one of them, Should I be doing BGP peering for private ASN x with private ASN y on Metro E with either AS Path prepending or Metric but isn't this supposed to work by defauly anyway based on eigrp metrics?

Appreicate your response.

[Jon Marshall]

See my last post which I just added and you need to go through.

If you still have questions then by all means comes back.

Jon

[Vishnu Reddy]

Can we implement SOO Site-Of-Origin. This is actually a BGP Extended Community

1. In this senario, can we implement SOO(site of origin) site X interface connecting to site x PE is configured with a SoO $SOO1
2. site Y PE interface connecting to site Y is configured with a SoO $SOO2, where $SOO2 is unique from $SOO1
3. General following two rules are applied on site X PE /site Y PE to decide if a EIGRP route needs to be accepted or dropped
4. "1. If the route is sent or received on the interface has the same SoO value as configured on the interface the route is discarded.
5. 2. If the route sent or receive on the interface has an SoO value but it does not match the one configured for the interface, the value is preserved."
6. Due to the above rules, any stale EIGRP route is not advertised

I am not sure this will work but just throwing some ideas.

[Jon Marshall]

No as that is a PE configured thing and in addition the example you give above is not relevant as the PE never receives EIGRP routes.

I appreciate there is a lot of information in the previous posts so i'll boil it down to the two things you need to consider -

1) you have to do this one.

On site X and site Y MPLS routers when you redistribute EIGRP into BGP you must only allow the local subnets so use a route map and only permit those subnets.

You must not allow the remote sites (ie. not X or Y) routes to be redistributed into BGP at either site X or site Y.

2) If you want X and Y to back each other up then you still need to do the above but in addition X will not only redistribute it's EIGRP routes into BGP but also Ys.

Y will do the same for X.

If you just redistribute then remote sites could end up being routed to Y for X's subnets and to X for Y's subnets. It depends.

So you need to use BGP attributes as covered in an earlier post to make sure routing always goes direct to the site, unless of course it's MPLS connection is down.

That's it.

Jon

[Vishnu Reddy]

I am kind of new to this senario. As far as I understand is at

1. site X

redistribute eigrp into bgp - Allow only local subnet related to site X and all remote sites excluding site Y.

    site Y

redistribute eigrp into bgp - Allow only local subnet related to site Y and all remote sites excluding site X.

2. To avoid Y for X's subnets and to X for Y's subnets senario..

Where would i tweak the BGP metrics? CE routers at X and Y or at the providers PE routers?

Really appreciate for all the help so far..

[Jon Marshall]

Okay no problem, it can be a little daunting when you haven't done it before :-)

1) No this isn't right.

When I refer to remote sites I mean sites other than X or Y.

So you must not allow either X or Y to redistribute any remote site networks into BGP.

There is no need because the remote site networks should never be advertised from X or Y and if you do allow it it will cause you problems.

So assuming you want X and Y to back each other up -

X redistributes it's own subnets from EIGRP into BGP and also Y's but no remote sites.

Y redistributes it's own subnets from EIGRP into BGP and also X's but again no remote sites.

If X and Y were not connected via a dedicated link this would not be an issue but because they are and they are exchanging EIGRP routes via that link they each receive EIGRP routes for the remote sites networks from each other as well as from their own MPLS router which is why you need to filter.

2) You can do the configuration on your CE routers.

MED or AS prepending are the tools you use.

So at X you advertise it's own subnets without any modification. You advertise Y's subnets but change one of the above BGP attributes.

At Y you advertise it's own subnets without any modification and X's subnets you need to modify the BGP attribute again.

This basically means that traffic will go direct to X or Y as long as their MPLS links are up.

Either should work although it may be worth having a quick chat with your provider and tell them what you are doing to make sure they are not doing anything that would override your settings.

Apart from that internally the L3 switches in X and Y should see their own MPLS routers as the best path to all remote sites because from your description the L3 switch is only one hop away from the MPLS router in X and Y whereas to go via the dedicated link would mean more hops.

I'm assuming it means more hops.

Quick way to check is do a traceroute from  X L3 switch to it's MPLS router and then a traceroute from X L3 switch to Y MPLS router and hopefully there should be more hops going to Y.

Do the same from the Y L3 switch.

If it is the same number of hops then you may have to modify the EIGRP metrics.

All of the above about redistribution only applies to EIGRP to BGP.

There is nothing to do about the BGP to EIGRP redistribution.

Concentrate on the redistribution issue because that could seriously affect your network in terms of it working properly.

Regarding the second issue have a chat with your provider about the MED and AS prepending just to clear it with them.

It's not a given that traffic would go in the wrong site, it might, but it wouldn't be the end of the world if you got it wrong temporarily because traffic is doing exactly that to and from the DC at the moment ie. it goes into X and across to Y.

By all means come back if you need more help but like I say focus on the redistribution issue because that really does need addressing.

Jon

[Vishnu Reddy]

Thanks for all the help. You have explained to me as one would do to a layman. I am clear on the redistribution part as of now based on your comments. If any concerns I will come back to you. This is scheduled for extended maintanence for next weekend. I will update you once I accomplish this task. I am trying to set up the appointment to bring up the mpls level3 circuit by Tuesday at site Y.

Appreicate for your help.

[Jon Marshall]

You have explained to me as one would do to a layman.

Hope you didn't take any offence, I was just trying to explain as clearly as I could.

No problem with the help.

Jon

[Jon Marshall]

Just to emphasise the point.

There could well be a routing loop if you don't filter the subnets as in my last post because when the MPLS connection at X comes back up it is now advertising the remote site networks to the PE which will then advertise them to other sites including Y.

So Y could send traffic to a remote network to X, X would forward it through to Y and Y back to X.

I haven't tested this but as far as I can see it could happen although happy to be corrected otherwise.

Simple answer.

To avoid all problems just make sure X and Y do not redistribute remote sites networks learnt from each other via EIGRP into BGP on their own MPLS router.

Jon

[Vishnu Reddy]

Thanks again...

This is what configured at site X

router bgp 65XXX
 bgp log-neighbor-changes
 redistribute connected
 redistribute static
 redistribute eigrp 1
 neighbor 192.168.2xx.1 remote-as XX49
 neighbor 192.168.2xx.1 default-originate
 neighbor 192.168.2xx.1 soft-reconfiguration inbound

router eigrp 1
 network 10.1.2X.0 0.0.0.255
 network 10.1.2y.0 0.0.0.255
 redistribute bgp 65XXX metric 16 100 255 128 1500

I should be filtering routes for all the remote site networks at site Y and let site X take care of mutual redistribution so that way I can avoid routing loop in one way. If mutual redistribution is done at both sites then definitely there will be routing loop.

[Jon Marshall]

Not sure what you mean by filtering at just Y.

If you filter at Y in terms of redistribution of EIGRP into BGP then that will not solve the problem because X is still redistributing remote site networks into BGP so you will get the issue I have covered.

If you mean filter as in redistribution of EIGRP into BGP and also stopping those routes going down the dedicated link then no you don't want that otherwise you break redundancy.

X and Y are both advertising the remote site networks to each other via the dedicated link.

If you stop Y advertising them back out via BGP that doesn't stop X doing it.

You need to use a route map with your redistribution on X and Y MPLS routers and only redistribute X and Y subnets.

If you want X and Y to back each other up you also need to modify the BGP attributes as I covered earlier otherwise traffic for X could come in via Y and traffic for Y via X.

Perhaps I am not understanding what you mean by filtering the routes ?

Can you clarify and also can you clarify exactly what redundancy is needed between X and Y.

Finally are the only external EIGRP routes in X and Y those from BGP ie. do you have any other external EIGRP routes from other sources ie. not the MPLS routers.

Your BGP configuration is going to need some modification depending on exactly what you want.

If there is anything you don't understand then please ask for clarification as you need to understand how it will all work before implementing.

Jon