EBGP w/ 3 ISP's

[eknell]

I am fixing to peer with a 3rd carrier, and was hoping to get some design suggestions.

Currently, I do BGP load-sharing across an OC-12 and a Gig-E link. I weight all recieved routes from both eBGP peers to help equalize my outbound traffic down each connection.

I will be peering with a 3rd peer via another Gig-E link, and am only wanting to use my OC-12 for inbound and outbound traffic, if both of my Gig-E links fail.

This is what I was thinking, and please correct me if I am wrong:

I planned on load-sharing my two Gig-E links and weighting my recived routes from these 2 peers higher than the routes recieved over my OC-12, which should control outbound traffic and only use the OC-12 as a last resort. As far as inbound traffic over the OC-12, I was going to poison my routes via BGP prepending towards that peer.

Anyhow, any suggestions would be greatly appreciated! Thanks!

[Giuseppe Larosa]

Hello Ethan,

you should be fine with your setup but be aware that your influence on inbound traffic is limited and you have to accept the load ratio you get:

even if you perform AS path prepend towards the ISP with OC-12 link you will see non-zero return traffic because there will be someone on internet still using that ISP to reach you.

Hope to help

Giuseppe

[eknell]

Giuseppe,

Yes, I understand there maybe "some" inbound traffic over that OC-12, I just want to make sure that it is virtually un-used.

I just hope my router can handle another 150K-200K prefixes from another carrier. Have you seen or heard of anybody peering with 3 or more carriers and having any issues with thier router? It is a 7609 running SUP720-3BXL's...

Thanks for your help Giuseppe!

[Giuseppe Larosa]

Hello Ethan,

about the OC-12 in the modern Internet where the average AS path is lower then in the past AS path prepending is effective so this link will be used just a little inbound.

with a Sup 720 3BXL you should be fine.

we have two 6509 with sup 720 3BXL receiving a full table from ours two border routers (two 12000 with PRP2)

Others have reported to have used sup720 BXL with 3 ISP and 3 full tables.

Hope to help

Giuseppe

[Joseph W. Doherty]

You might also consider PfR. I think the 7600 is supported as a border router in the later releases. If it is, PfR could, outbound, dynamically load balance across your two gig links, or those and the OC12 in proportion to link capacity. For inbound, believe it can also dynamically prepend AS hops or send community strings to peers (so they can adjust their metrics).

PS:

As to working with 3 Internet tables, I too believe your XL should handle them, but since PfR watches end-to-end flow performance, in theory, outbound BGP routes shouldn't be necessary.

[Giuseppe Larosa]

Hello Joseph,

I thought PfR could be a solution for optimizing some destinations because it is flow based.

If the network is a service provider traffic can be more spread then that of a single enterprise.

However, I guess you can specify for what destinations PfR should work.

Hope to help

Giuseppe

[Joseph W. Doherty]

PfR does track flows (also link usage); it uses its own copy of Netflow tables. Given a flow's performance information, it can override a flow's natural (routing table) best path and redirect it to another path. Criteria for path selection is defined as part of the PfR policy. You can also select the flows that will be controlled by PfR. This criteria could include network addresses and/or type of traffic.

Unclear about the point you're making SP vs. enterprise. Perhaps you're thinking PfR only controls traffic on "outside" interfaces. If so, I would agree it would not be suitable for router that allows transit traffic between "outside" interfaces. From the original post, didn't think that would be an issue in this case.

[edit]

PS:

The 7600 PfR implementation isn't, I believe, full featured.

[Danilo Dy]

You did not mention how many routers you use to peer with 3 service providers but you mentioned "weight", I assume you only have one router to peer with 3 service providers.

In my case, I have 3 routers peers with 3 service providers (full BGP). I load-share the links to 2 service providers and make it preferred for outgoing (as 80% of my traffic is outgoing) by increasing the local-preference of all received routes from this 2 service providers from the default of 100 to say 150. For the 3rd service provider I keep the local-preference for received routes to the default of 100 and prepend my AS towards this 3rd service provider.

However, as what the other poster says, you cannot completely eliminate incoming from 3rd service provider (unless you put an ACL that blocks all incoming traffic), because the BGP administrator of the 3rd service provider can make his router choose his direct link to you as the best path (so is other service providers behind him). This is the interesting part of BGP, although it gives you total control for your outgoing traffic, it doesn't gives you total control for incoming traffic. I have a theory that if BGP gives us total control for both incoming and outgoing traffic, the internetworld will be chaotic, BGP administrators will be busy round the clock.

[eknell]

Medan,

That is correct...I only have one 7609 that will be peering with the 3 providers. I wish I had another one, but another 7609 with redundant 720's is just not in the budget right now.

What you said about load-sharing the two providers and path prepending the third was exactly what my initial thoughts were on this implementation.

The reason why I weight my routes, as opposed to adjusting the local-pref values is because I am really only running iBGP on one other router in my local AS, and I am not really concerned with propagating those values to any other router...but we are definitely on the same page as far as outgoing traffic is concerned.

Thanks for your help!