Route redistribution from BGP to OSPF and rfc1583

[actyler1001]

Hi everyone, I'm experiencing a very frustrating problem with route redistribution from eBGP into OSPF. Consider the below diagram. The goal is to route traffic as directly as possible to the Azure networks 10.24.0.0/16 and 10.25.0.0/16. This was all going well until I connected a non-Area 0 (R5) router to R2 and/or R3. See referenced links A and B in the diagram.

Once R5 brought up an adjacency with R2 and/or R3, R1 and R4 decided the best path to these /16 networks was via R5. An ospf path cost much larger than any direct path. I've attempted to redistribute using External type 1 and type 2, no difference. All ospf areas are "normal".

If I take the link A and B down, all goes back to normal and routing makes sense again. R5 in this case chooses R1 > R2 > 10.25.0.0/16. Also R5 chooses R1 > R3 > 10.24.0.0/16.

R5 is only a member of Area 40. R1, R2, R3, and R4 are all ABRs. R2 and R3, ASBRs.

At first I thought this was a bug. I've managed to make it far up the escalation chain and was finally given the response that this behavior with external routes within ospf is normal while using the default rfc2328 operating mode. I was told that in order to deal with this I would have to enable the older rfc1583 operating mode. On every router in our organization. About 50.

I feel like rolling back to this older rfc is the wrong move. There has to be another way to deal with this, but for the life of me I have yet to come up with a solution. Other than simply NOT connecting my non-Area 0 routers directly to R2 and R3.

[MHM Cisco World]

Change AD of ibgp to be 100. 

This make Routers prefer it than ospf. 

[actyler1001]

Hi @MHM Cisco World, I was doing some digging and it appears the default admin distance for iBGP is 200.  You are recommending I change this on the routers responsible for redistribution into OSPF?  To 100 which is 10 less than the OSPF standard admin distance..

It looks like I made a mistake in my original problem description.  I'm using eBPG, not iBGP.  So the BGP peerings for R2 and R3 are to different AS numbers.  I just used the private range for the AS string.  So the rib on R2 and R3 show an AD of 20 for 10.24.0.0 and 10.25.0.0 as expected with eBGP.

Regards,

Adam Tyler

[MHM Cisco World]

When you do 

Show ip ospf database external 

What is ip of forward address

Check metric to this IP

[actyler1001]

From the perspective of R1, it looks like this (See below).  10.24.50.4 is the router ID of R3 in the diagram.  However, the RIB currently is directing all traffic for 10.24.0.0/16 to the Area 40 router.  This appears to be due to the rfc1583 deprecation and now with rfc2328 Intra-area routes are preferred regardless of cost and the backbone area is avoided completely.  Very strange.

Here is a good reference...

OSPF Routing Loop/Sub-Optimal Routing between Cisco IOS and NXOS for External Routes Configuration Example

LS age: 1262
Options: 0x2 (-|-|-|-|-|-|E|-)
LS Type: AS-external-LSA
Link State ID: 10.24.0.0 (External Network Number)
Advertising Router: 10.24.50.4
LS Seq Number: 80001577
Checksum: 0x61e1
Length: 36
Network Mask: /16
Metric Type: 1
TOS: 0
Metric: 10
Forward Address: 0.0.0.0
External Route Tag: 0

[MHM Cisco World]

But as I see the R1/R4 connect to R2/3 via intra-area.

Here forward address is 0.0.0.0 so the path will use RID which is  10.24.50.4

Can you traceroute from R1:R4 to this IP and see path is it pass through R5 or not? 

[actyler1001]

Traceroute from R1 to anything 10.24.0.0 takes the cost 80 direct path to R3, but only if links A/B are disabled.  If links A/B are enabled, R1 tries to access 10.24.0.0/16 through R5 which is the root of the problem I've attempted to describe in original post.

Here is an example of the route that makes it into the RIB on R1 when links A/B are disabled.  Notice the cost of 90 and the AD of 110.  Our reference bw is 40Gbps, the direct cost is only 80, but 10 gets added along the way during cumulative Dykstra calc.

O E1 10.24.0.0/16 [110/90] via 10.10.160.2, AZ1WUSx1  (AZ1WUSx1 is a tunnel interface directly to R3)

Now check out what happens when I enable the link A/B to the non-Area 0 firewall.  The R1 routing table does this.  Chooses a path with a much higher cost (2304 instead of 90) in order to avoid the "backbone area", due to the rfc change I refence above.

O E1 10.24.0.0/16 [110/2304] via 10.10.148.230, 40x1 (40x1 is a tunnel interface directly to Area 40 router [R5].)

[MHM Cisco World]

I understand what you mean here but

R4 AND R1 have intra area to R2/R3

The issue the RID is same as external!!

So change RID of R2/R3 and check. 

[actyler1001]

@MHM Cisco World Thanks for sticking with me.

I'm not sure what you mean by "R4 AND R1 have intra area to R2/R3".  All of the links (with the exclusion of the R5 links) are classified Area 0 which is considered the backbone area.  So I guess I agree that R1, R2, R3, and R4 all have intra-area connectivity to one another over Area 0.  However Area 0 is avoided apparently based on the rfc change from 1583 to 2328.

R1, R2, R3, and R4 also all have an Area 40 link to R5 when this problem is present.  We've had to keep links A and B down hard to keep routing from breaking.

Regarding your statement about the RID (I think you are referring to Router ID here), the RID is currently set as a unique 32-bit address on every router.  Are you referencing that the fact that the RID IP of R2 and R3 being in the same subnet as the redistributed routes as a problem?  Like I need to create a lo interface on R2 and R3 outside the /16 range?

You may have to dumb it down for a bit, apologies.  Appreciate your help.

[paul driver]

Hello @actyler1001 @MHM Cisco World 
Have you tried maxing out R5 LSAs metric as i suggested?

[MHM Cisco World]

No friend 

I will try it in R5 and update you

MHM

[actyler1001]

I changed the RID on the OSPF process of R3 to 10.10.124.1.  I created a loopback interface and passively injected into OSPF Area 0 (10.10.124.0/30).

External LSA on R1 looks like this now, but the problem persists.

LS age: 620
Options: 0x2 (-|-|-|-|-|-|E|-)
LS Type: AS-external-LSA
Link State ID: 10.24.0.0 (External Network Number)
Advertising Router: 10.10.124.1
LS Seq Number: 80000002
Checksum: 0x068e
Length: 36
Network Mask: /16
Metric Type: 1
TOS: 0
Metric: 10
Forward Address: 0.0.0.0
External Route Tag: 0

[MHM Cisco World]

the R2 and R3 is connect to R4 and R1 via cross link, 
this cross link is in area 0 or in area 40 ? 
I build lab to test this config and check solution 

[actyler1001]

@MHM Cisco World , I admit there is some detail missing from the diagram.  See below, in this diagram if it isn't marked Area 40, it is set as Area 0 at each end.

[paul driver]

Hello
The most simplistic way is to change the maximum metric of R5 (65535 + the cost of the destination link to azure networks) so the other rtrs will see their direct connection to the R2/R3 as preferred path.

R5
router opsf x
max-metric router-lsa summary-lsa