Route redistribution from BGP to OSPF and rfc1583

[actyler1001]

Hi everyone, I'm experiencing a very frustrating problem with route redistribution from eBGP into OSPF. Consider the below diagram. The goal is to route traffic as directly as possible to the Azure networks 10.24.0.0/16 and 10.25.0.0/16. This was all going well until I connected a non-Area 0 (R5) router to R2 and/or R3. See referenced links A and B in the diagram.

Once R5 brought up an adjacency with R2 and/or R3, R1 and R4 decided the best path to these /16 networks was via R5. An ospf path cost much larger than any direct path. I've attempted to redistribute using External type 1 and type 2, no difference. All ospf areas are "normal".

If I take the link A and B down, all goes back to normal and routing makes sense again. R5 in this case chooses R1 > R2 > 10.25.0.0/16. Also R5 chooses R1 > R3 > 10.24.0.0/16.

R5 is only a member of Area 40. R1, R2, R3, and R4 are all ABRs. R2 and R3, ASBRs.

At first I thought this was a bug. I've managed to make it far up the escalation chain and was finally given the response that this behavior with external routes within ospf is normal while using the default rfc2328 operating mode. I was told that in order to deal with this I would have to enable the older rfc1583 operating mode. On every router in our organization. About 50.

I feel like rolling back to this older rfc is the wrong move. There has to be another way to deal with this, but for the life of me I have yet to come up with a solution. Other than simply NOT connecting my non-Area 0 routers directly to R2 and R3.

[actyler1001]

Thank you @paul driver, unfortunately R5 in this scenario isn't a Cisco device.  I'm attempting to find a comparable command in the separate OSPF interface.  At a glance, "max-metric" isn't available, I've asked our vendor.

In the meantime, I wanted to try and talk through your logic.  Rfc2328 seems to state that in this scenario regardless of the metric, the backbone area will be avoided.  I've already demonstrated that Area 0 routers will choose a route with a metric of over 2000 when they have route options where the metric 90'sh available.  How will changing the 2000 metric to over 6k make any difference?

RFC 2328 - OSPF Version 2 (ietf.org)

Section: 16.4.1. External path preferences

"When multiple intra-AS paths are available to ASBRs/forwarding addresses, the following rules indicate which paths are preferred. These rules apply when the same ASBR is reachable through multiple areas, or when trying to decide which of several AS-external-LSAs should be preferred.

o Intra-area paths using non-backbone areas are always the
most preferred.

o The other paths, intra-area backbone paths and inter-
area paths, are of equal preference."

Links A/B disabled.  Area 0 routers choose a metric of 90

O E1 10.24.0.0/16 [110/90] via 10.10.160.2, AZ1WUSx1  (AZ1WUSx1 is a tunnel interface directly to R3)

Links A/B active.  Area 0 routers happily choose a metric of over 2300 while metric 90 option is ignored.

O E1 10.24.0.0/16 [110/2304] via 10.10.148.230, 40x1 (40x1 is a tunnel interface directly to Area 40 router [R5].)

Ps..  I did try making area 40 an NSSA.  Instead of receiving an E1 route it turned into a Type 7 N1.  Area 0 had the same problem.  I was considering going with a stub area so that Area 40 never received the redistribution at all.  however, I'd just be faced with the problem of how to redistribute routes into OSPF for stubs since they ignore Type 5 LSAs completely.  We don't only redistribute for these Azure networks, but a few others in our environment.  Seems like this just moves the problem.  I don't have the option of leveraging the default route, it points to the internet at router and is static.

[Joseph W. Doherty]

Have you considered the possibility of making some or all the R5 <> R1..4 links, area zero?

One concern to doing such would be to avoid using R5 as an area zero transit, but interface costing might preclude that.

[actyler1001]

Thanks @Joseph W. Doherty, We have about 50 sites/Areas in our environment.  Area 40 in this diagram is just one of those sites, but we follow the same process/setup for all.  The original design was to establish about 7 of our most powerful routers with the most stable connections as Area 0/transit firewalls.  This Area 40 firewall is a tiny device and in some cases can have an internet connection that goes up and down.

So to optimize routing performance, to reliably predict traffic flow under failure conditions, and to insulate Area 0 from unnecessary reconvergence events, we chose to limit Area 0 routers and tightly manage the costs between each.  Not to mention a uniform firewall policy across Area 0 firewalls to permit interim hop behavior when/if direct paths fail.

About the only downside with the design so far is that non-Area 0 firewalls have to transit Area 0 in order to speak with one another.  A design consideration that was evaluated and accepted early on.  The new downside is this darn rfc E1/E2 routing craziness.  For the life of me I cannot imagine why a new rfc would cause OSPF to ignore cost, but only for external routes.

Lol, so long story short, yes I have considered making more firewalls members of Area 0.  For reasons above I don't consider this an option for R5.

Regards,

Adam Tyler

[Joseph W. Doherty]

Ok, how about the converse?  I.e., can you create another non area zero area sharing the area zero links?  E.g. using subinterfaces.

[actyler1001]

@Joseph W. Doherty This is an interesting idea.  So on R2 and R3 you are saying to create a fake network either via sub-interface or Loopback interface and get it into the OSPF process using a non-Area 0 designation.  Then what?  The redistribution command doesn't appear to take an area specification.  Maybe I don't follow your suggestion entirely.

[Joseph W. Doherty]

On some, or even all, of your area zero links, you setup a parallel non zero area for ASBR transit.  Basically, just as area 40 is being preferred over the backbone (area zero), this parallel area would be preferred to reach the ASBRs, but can appear to have better paths, to the ASBRs, vs. all the other non zero areas.

[actyler1001]

@Joseph W. Doherty Hmm.  Most (Almost all) router to router links you refer to are IPSEC VPN tunnel interfaces associated with little /30 transit networks.  So point-to-point adjacencies all around.  It seems in order to accomplish what you describe; we'd need to bring up an additional tunnel-interface between area 0 routers which would require potentially more WAN IP space with a different /30 network for the non-Area0 adjacency.  Sort of mind twisting, but interesting....  Might work.  Not sure I like it better than just enabling rfc1583 compatibility.

[Joseph W. Doherty]

From what you describe, yup additional tunnel interfaces.

BTW, don't think you need a full mesh.  If just, like in your diagram, you have two ASBRs, logically they should have all the other area zero routers as neighbors (on this "special" transit area).  For example, in your diagram R1 and R4 wouldn't need to logically link.

Regarding WAN IP space, since you're using tunnels, private IP space should work, and/or you might be able to use /31s or have one subnet, per ASBR DMVPN hub.

Other possibilities to reduce WAN IP space would be to use (Cisco only?) unnumbered p2p.  ([Cisco only?] OSPF prefix-suppression, to further reduce demand on some resources.)

[paul driver]

Hello
Just to confirm R5 is peering with all the other rtrs via area 40 , so by ospf design the backbone is always preferred over non BB areas, but this is not happening correct - most possibly due to the ospf RFC compatibility you are using?

Can you please post output of the following:

Sh ip ospf | in RFC

[actyler1001]

@paul driver Yes, R5 has OSPF adjacencies to other routers only via area 40.  Routing works perfectly respecting cost for any O, or IA route.  The trouble comes in when we route to E1/E2 destinations.  Because we've standardized on rfc2328, Area 0 is avoided for E1/E2.

We have multi-vendor routing devices in our environment, so that command you posted works different depending on the platform.  I can say however that we've confirmed that all devices are currently running in rfc2328 compliant mode.

[actyler1001 · Accepted solution]

I'm considering the adjustment of the design to something like this.  That way the routers responsible for redistribution only participate in Area 0.  This really does seem like a lot of pointless work though.

I mean, how bad of a crime is it for us to just enable rfc1583 compatibility on all of our devices and call it a day?  Network engineer heresy or have I found a real use case and should just give up?

[Joseph W. Doherty]

---

@actyler1001 wrote:

I mean, how bad of a crime is it for us to just enable rfc1583 compatibility on all of our devices and call it a day?  Network engineer heresy or have I found a real use case and should just give up?

An excellent question.  When I first read your OP, my first thought was WHY this RFC change.  For the last 60 years, an informal rule has been, maintain backward compatibility unless you have a really, really (really) good reason to not do so.

Consider, just like the Cisco reference you posted, besides changing behavior, mixing and matching devices running using RFC1583 and RFC2328 can cause routing loops.

You see the issue of upgrading 50 routers.  But what to do so, presumes all 50 routers can run the same RFC standard.  (Perhaps true now, but I'm when support for RFC2328 initially came out, even if you wanted to use it, you would have had to upgrade all your router software too.)

I point out the last, to highlight why these kinds of changes are not desired.

That was then, this is now, so should you downgrade?

Well, RFC2328 is the newer standard, and going backwards, often leads to future problem.  Like, all of a sudden, some vendors don't even support the "old" standard.  (As an aside, check into those complaining about 10 Mbps Ethernet no longer being supported, especially the 10/half.)

Personally, from a very brief review, unclear to me why RFC2328 made this change, but it did.  Perhaps, this shouldn't have been done, but it was.  Since it was, I wouldn't recommend 25+ years after RFC2328 (4/98) was published, going back to an even earlier standard.

BTW, unsure what you propose, in your topology change, would do the trick.  Currently, R1 goes to R2 via R5 to avoid using the backbone path.  How would moving ASBR on hop away, in area zero, preclude R1 still using R5 to R2 to VRF?  (I recall some where in this thread, mention of just using default route?  If you can use that, rather than redistribute directly into OSPF, might be the simplest solution.)

Also, BTW, what I proposed was the solution I came up with years ago for a somewhat similar issue, of OSPF intra-area routing being preferred over inter-area routing.

Had ABR<>router<>ABR, where ABRs were injecting a summary address for all the networks in the non-zero area.  Problem was, if non-zero router lost link to one of the ABRs, and traffic for it hit the ABR w/o a link to any specific router, ABR would NOT send traffic to other ABR, it would send traffic to another router, in that area, which would send traffic to the other ABR which would send traffic to the router we wanted the traffic to go to.  Since the two WAN ABRs were side by side, and the WAN was very "wide area" (the Americas), this wasn't ideal.  "Fix" was to also include each WAN area in a link between ABRs.  Then ABRs would send data, for non-zero areas, between themselves, across the non-zero area link between them.

[actyler1001]

@Joseph W. Doherty"BTW, unsure what you propose, in your topology change, would do the trick.  Currently, R1 goes to R2 via R5 to avoid using the backbone path.  How would moving ASBR on hop away, in area zero, preclude R1 still using R5 to R2 to VRF? "

The hope was that because the /16s were only advertised from Area 0 with the new vrf topology, we would skip the "external routes avoiding area 0" thing.  Obviously would have to test, but your stance is that once it hit the ABR that was a member of all areas I would run into the same problem?  It wouldn't be an ASBR anymore, that appears to be the root issue based on the rfc.

Regarding your comments about steering clear of the old rfc compatibility mode.  I had many of the same concerns.  Today that compatibility command is available on everything in our environment.  Tomorrow, maybe not.  It's why I've put so much Energy into coming up with a different solution.  I keep coming back to that compatibility command being a part of the current rfc2328.  I mean, all devices that are compatible with the current rfc will support this compatibility command.  Until a new rfc version of OSPF is released.  And then, not all routers will support it overnight.  So it's highly likely there are a lot of years of runway here.  Devil's advocate, I don't want to enable this setting.

[Joseph W. Doherty]

"It wouldn't be an ASBR anymore, that appears to be the root issue based on the rfc."

Again, I haven't develved deeply into this, but unclear (to me) whether the issue is what area ASBRs are in or how you reach them.

[actyler1001]

@Joseph W. Doherty Based on my research of rfc2328, the root problem I am facing is that the ASBR routers are accessible via "multiple intra-AS paths".  Because my ASBRs are members of Area 0 and Area 40 this rfc condition is met.  The theory is that if the ASBR was moved back into only Area 0 using this vrf hack, this condition would no longer be true.  See rfc 2328, section 16.4.1.

ietf.org/rfc/rfc2328.txt

"16.4.1. External path preferences

When multiple intra-AS paths are available to
ASBRs/forwarding addresses, the following rules indicate
which paths are preferred. These rules apply when the same
ASBR is reachable through multiple areas, or when trying to
decide which of several AS-external-LSAs should be
preferred. In the former case the paths all terminate at the
same ASBR, while in the latter the paths terminate at
separate ASBRs/forwarding addresses. In either case, each
path is represented by a separate routing table entry as
defined in Section 11.

This section only applies when RFC1583Compatibility is set
to "disabled".

The path preference rules, stated from highest to lowest
preference, are as follows. Note that as a result of these
rules, there may still be multiple paths of the highest
preference. In this case, the path to use must be determined
based on cost, as described in Section 16.4.

o Intra-area paths using non-backbone areas are always the
most preferred.

o The other paths, intra-area backbone paths and inter-
area paths, are of equal preference.

"