CCIE-Type Routing Question

[lamav]

Please view the attached drawing before you read on and you may want to leave it open to refer to it.

Anyway, here is the scoop.

The OSPF area represents our Australia network.

Routers SING and JAPAN are OSPF ASBRs that redistribute OSPF-learned routes into BGP.

Both the SING and JAPAN routers form iBGP neighborships with both FMC and ECC in the United States.

So, FMC learns OSPF Australia routes through iBGP from both SING and JAPAN.

And ECC learns OSPF Australia routes through iBGP from both SING and JAPAN, too.

Right now, both FMC and ECC routers favor the SING router as the "next hop."

The reason is that all things being equal, the SING router carries a better IGP metric into its BGP advertisements than the JAPAN router does.

So, if one does a "sh ip bgp 19.210.108.16" (this is an Australia network) in the FMC and ECC routers, they will both show that they have learned 2 routes through iBGP, one from SING and one from JAPAN, but the SING router carries an IGP (OSPF) metric of, say, 330, while the JAPAN router advertises an IGP metric of 445. Therefore, the SING route is favored, so the FMC and ECC routers place the SING-learned iBGP routes in their BGP table.

<i>ecc#sh ip bgp 19.210.108.16

BGP routing table entry for 19.210.108.16/29, version 31730939

Paths: (2 available, best #1, table Default-IP-Routing-Table)

Advertised to update-groups:

2 3

Local

19.213.1.130 (metric 1466112) from 19.213.1.130 (19.213.1.130)

Origin incomplete,

[Giuseppe Larosa]

Hello Jon,

playing with IGP metrics should always be the last resort.

Besides this, I think it doesn't work in this case without also changing the BGP attributes of the route:

the current situation of the displayed route shows that the best route has the lowest BGP metric and it is chosen even if the EIGRP metric to BGP next hop is higher then that of the other route.

This happens for the sequence of criteria used by BGP for path selection.

So changing EIGRP metric is useless in this specific case without other actions on BGP.

Hope to help

Giuseppe

[marikakis]

If I am wrong, then the scenario is simple enough to be handled with the local preference, as is the usual in such cases. Ask the engineer why local preference does not solve its issue and starts thinking of wild things.

Ok, the more I read this, the more I am thinking I might be right. Engineer tries to attract some traffic to JAPAN from ECC, but doesn't want all traffic in the AS to use the direct link, so he starts playing with the EIGRP metrics, instead of playing a bit with BGP on SING, because packets that reach SING have to go out SING as I said before. A little work on SING can stop traffic from exiting via SING, as I described previously (SING2 might also need some work to avoid a loop).

[Jon Marshall]

Giuseppe

"playing with IGP metrics should always be the last resort"

I totally agree. Was there something i wrote that suggested otherwise because i thought i was suggesting the same as you ie. use local preference.

Jon

[marikakis]

Most people don't play with metrics for fun because they lead to confusing and unscalable setups. It looks like this guy is in a last resort situation (load balancing internally issues are not uncommon) and might actually make it in the end. All this guy needs for load balancing to work is to reset the MEDs to zero. Then best path selection algorithm will go further down, and the lowest IGP metric to the BGP next hop from each router in the AS will be used to balance the traffic, so is close to a solution that will distribute traffic over the links to Australia. When someone suggests the weird, try not think they don't know what they are doing.

[lamav]

Giuseppe:

"Besides this, I think it doesn't work in this case without also changing the BGP attributes of the route:

the current situation of the displayed route shows that the best route has the lowest BGP metric and it is chosen even if the EIGRP metric to BGP next hop is higher then that of the other route."

I think you are confusing the issue. The client is not trying to influence the BGP selection process. The client is satisfied with leaving the SING router as the BGP next-hop for ECC and FMC to reach Australia. What the client is trying to change is the eigrp route from the FMC and ECC routers to the SING router, and that has nothing to do with the Australia IGP metrics that the SING router is advertising.

[Jon Marshall]

I'm confused now :-).

My understanding was that FMC & ECC have IBGP routes to the Australian networks which have been redistributed from OSPF.

So what i think Giuseppe is saying is that it doesn't matter what you do to the EIGRP metrics, FMC & ECC will choose the best path from the IBGP routes. The best path is SING and that is the one they will use.

Now i thought the question was how to make FMC & ECC see JAPAN as the best route and the answer is local pref. This is by far the cleanest solution to the problem and i would have thought the most logical for anybody else coming along later to understand.

Having FMC & ECC see SING as the best route but sending the traffic via JAPAN so that JAPAN sends the traffic is just overcomplicating things. If you were the next network administrator coming along that would not be an obvious solution to the initial problem.

The other point i was making is if the client wants to mess around with metrics why bother with BGP at all. Just run OSPF on all routers and then influence the OSPF metrics directly.

Maybe i'm missing something here and as Maria pointed out it's not that i'm assuming the engineer proposing the EIGRP solution doesn't know what he is doing. He may have a very good reason for what he is proposing. It's just not obvious at the moment.

Jon

[lamav]

"My understanding was that FMC & ECC have IBGP routes to the Australian networks which have been redistributed from OSPF."

Correct.

"So what i think Giuseppe is saying is that it doesn't matter what you do to the EIGRP metrics, FMC & ECC will choose the best path from the IBGP routes. The best path is SING and that is the one they will use."

Correct, although I think Giuseppe was saying something else. I think he is under the impression that the purpose the client has of changing the eigrp delay metric on the ECC and JAPAN routers is to affect the selection of the BGP next-hop, which of course is completely wrong. They are not trying to do that at all. The IGP metric they want to change is the EIGRP delay so that ECC and FMC take a route through JAPAN on their way to SING, which of course they never reach because JAPAN is going to send the traffic directly over to Australia itself. If that's not what he means, then he is correct, too. Sorry, Giuseppe, your foreign English throws me off sometimes. :-)

"Now i thought the question was how to make FMC & ECC see JAPAN as the best route and the answer is local pref. This is by far the cleanest solution to the problem and i would have thought the most logical for anybody else coming along later to understand."

Correct, once again. And I agree 100%.

"Having FMC & ECC see SING as the best route but sending the traffic via JAPAN so that JAPAN sends the traffic is just overcomplicating things. If you were the next network administrator coming along that would not be an obvious solution to the initial problem."

Correct and I agree. I would not take this approach. I would take the more direct approach of making JAPAN the next-hop router by manilpulating the BGP attributes (local pref, etc).

"The other point i was making is if the client wants to mess around with metrics why bother with BGP at all. Just run OSPF on all routers and then influence the OSPF metrics directly."

I agree and understood your point.

"Maybe i'm missing something here and as Maria pointed out it's not that i'm assuming the engineer proposing the EIGRP solution doesn't know what he is doing. He may have a very good reason for what he is proposing. It's just not obvious at the moment."

You are not missing anything, given what I have told you, which is all I know atthis moment.

You are not confused but right on target.

Victor

[marikakis]

Victor,

I am not the type of person that insists on something just so my own solution is the selected one (and so I get the points in this case)! I am more practical than this, and I only care if an issue is resolved, no matter who does it! I was actually very busy last night, but still decided to respond, because I thought I might help.

You have not still responded in my previous post about what using SING as "bait" and "detour" mean. If those were the words of the client, then the client might be trying to tell you something here. This sounds to me as if some traffic to Australia is the 'mouse', SING is the 'cheese' and detour is the path via SING2 to JAPAN. The mouse goes to the cheese, and instead of trapping and killing it, you reroute the mouse to go out of the house (AS) through the JAPAN door to live in Australia! This metaphor is so cute, I think I am going to cry in the end if I am wrong :-)

This issue is either very simple and can be solved with local preference us others have suggested from the beginning or is not.

If it is not, I have already provided guidelines. It is common for one type of traffic change to affect other links. This network was using the SING path to go to Australia. When they switched the traffic to go via JAPAN (probably with local preference), they might have noticed the direct link between ECC and JAPAN got overloaded. It is very hard doing traffic engineering using local preference, because it causes a lot of traffic to go in one direction only. If IGP metric to BGP next-hop is used (after reseting the MEDs to zero or make them equal), you could have some type of closest exit routing (and you can overcome altogether the exit via SING, by turning traffic to go via SING2 to JAPAN).

Kind Regards,

M.

[marikakis]

Note also, that setting local preference will cause traffic to Australia through SING router (which may have other sources of traffic for Australia via other of its interfaces), will choose to go to Australia via JAPAN, and will not use the direct link it has with Australia (which might not be a desired thing to happen). Unless OSPF steps in and fixes this.

Jon, maybe my phrase "don't know what they are doing" sounded somewhat rude, but written speech has those type of issues. I know how helpful all of you are. I meant that when someone tries to do weird things, those things might point to the problem at hand. Sometimes clients provide little information or if they are engineers assume you will understand what they are getting at by using jargon. And such issues are a little bit of a chess backtracking procedure: if this is their move now, what was the situation in the chessboard before they made this last move?

Also, when I said previously that client does "wild things", in the same sentence I said to ask why local preference does not resolve its issue, so there I assume client probably already did that and saw something bad happening, so tries to seek consultancy, because from this point the case is not trivial.

[marikakis]

One last probability is the following: There is not a problem with the bandwidth utilization of any links in the AS after setting local preference, but some traffic that reaches SING via some of its interfaces (e.g. from SING2 to SING in the case SING2 does not have full routing, I am not even sure if SING2 runs BGP at all from the diagram) keeps exiting via SING. OSPF might be the reason for this and changing the administrative distance for OSPF or BGP on SING might solve this. This is a more normal scenario than the one I was thinking up to now. It also explains a bit why the client is trying to attract traffic towards JAPAN by changing EIGRP metrics. Some type of traffic might keep exiting via SING even after setting local preference.

[marikakis]

If this is the case, then the mouse story changes a bit and becomes: mouse comes from SING2 (SING2 might just have some default towards SING) to cheese (i.e. SING) and needs to be diverted towards FMC and run a circle to exit via the JAPAN door.

[lamav]

Mari:

Easy there, young lady! I appreciate your efforts. I didnt respond to you because I was in the middle of an in-depth discussion with Giuseppe and Jon, which can actually help you understand exactly whats going on if you read it.

Im not even going to attempt to unravel your mouse, cheese and house analogy. Sorry, but it is way too complicated for a simpleton, like me. :-)

I think we may have beat this topic to death by now. I will talk to the client to try to unravel his logic and see if he is making any sense.

Thanks, folks!

Victor

[marikakis]

Victor,

I am almost 32 now and not sure if this is very young! :-) I have read the posts again and I won't mention the load balancing scenario again, although it is not an unrealistic requirement for the future if the JAPAN link to Australia starts experiencing similar issues as the SING link to Australia. In such a case, resetting the MEDs and using closest exit routing using IGP metrics might be a solution, so keep that in mind for future reference. In the ISP environment I was working in the past such issues were very common, so be prepared because this network is not very much different in logic as potential segment of an ISP network.

Typically, an engineer who has BGP configured in various countries and in more than say 2 routers, has to know BGP better than an engineer who only has 1-2 BGP connections in a single router towards its upstream(s). Local preference is the first thing an engineer that works with BGP learns to manipulate traffic. Such an engineer might get frustrated if the consultant that is giving them advice suggests the typical solution, because it might have been the very first thing that crossed their minds as well and they called for help to hear something more.

From my experience, there are particular phrases in client information that are key to see the solution, even if non-technical terms are used, so a consultant has to notice all the details to avoid frustrating the client and be prepared for various potential issues and solutions to give to the client directly or be prepared to ask the client the right questions. In this case I might went too far with my imagination, but you asked for more opinions in the first place, and I wanted to inform about potential issues. Clients sometimes might know their networks so well, that they don't bother to mention important details, because in their heads they are taking them for granted, and speak as if you knew their networks from day one and you can figure what they need now without expicit stating. I might have overestimated the client's knowledge in this case, but I have seen underestimation of others leading to worse results (i.e. we would not call our consultants anymore, because we knew they would suggest something we did 10 steps before).

In the end, it is always good to hear that problems are less hard than expected and typical solutions can be used. There is a well know saying that suggests to keep things simple, and I try to follow that. Whatever I might imagine, I don't put to practice for no good reason.

Kind Regards,

Maria

[marikakis]

If you look back in this conversation, you will see all I was trying to do was to through some ideas before client responds, so you are more prepared and client keeps calling you in the future. I could just wait for the client to respond and wash the dishes instead like a good lady! :-)

[marikakis]

Victor,

In another case you opened, I see that you are now making the MEDs equal. It would be nice to hear what happened in this case and why for future reference when you are done with this and you have more time, not only for all of us, but also because a conversation (although admittedly long) that has the specific title you chose in the first place is expected to be considered interesting for many people. :-)

Kind Regards,

Maria