eBGP learned routes preventing redistribution.

[jwegleitner]

To make a long story short, we have two datacenters that have a high speed private link.  Each datacenter is being treated as a hub to the WAN.  The problem is, when one WAN router learns about eBGP routes from the other, it ignores the EIGRP routes I'm trying to redistribute into BGP.  The idea is that remote sites would prefer going to DC1 for specific routes and DC2 for other specific routes.

In the example provided, R3 advertises the EIGRP routes to the BGP cloud.  R4 receives the BGP routes and stops advertising the routes I was redistributing.  If I reset BGP on R3, the reverse happens.

The end goal:

Remote sites will take the path through R3 for all routes local to the left side.  This is done with a route-map and as-prepend.

Remote sites will take the path through R4 for all routes local to the right side.  Same method as previous point.

If R3 fails, R4 will back it up and vice versa

If the link between R1 and R2 fails, Left can get to Right using the WAN links.

I'm sorry if this is unclear, I'm having trouble describing everything without writing a novel.  I will try to answer any questions to clarify.

Some show commands:

R3 preferring BGP routes learned via BGP over the EIGRP routes I am trying to redistribute into BGP.

R3# show ip route
Codes: C - connected, S - static, I - IGRP, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default, U - per-user static route, o - ODR
       P - periodic downloaded static route

Gateway of last resort is not set

     172.16.0.0/32 is subnetted, 2 subnets
B       172.16.50.7 [20/0] via 192.168.1.1, 00:33:49
B       172.16.49.7 [20/0] via 192.168.1.1, 00:33:49
     172.30.0.0/30 is subnetted, 5 subnets
B       172.30.247.12 [20/0] via 192.168.1.1, 00:33:49
B       172.30.246.8 [20/0] via 192.168.1.1, 00:33:49
C       172.30.247.20 is directly connected, FastEthernet0/1
B       172.30.240.36 [20/0] via 192.168.1.1, 00:33:49
B       172.30.240.40 [20/0] via 192.168.1.1, 00:33:49
     10.0.0.0/16 is subnetted, 3 subnets
B       10.6.0.0 [20/0] via 192.168.1.1, 00:33:50
B       10.4.0.0 [20/0] via 192.168.1.1, 00:33:50
B       10.5.0.0 [20/0] via 192.168.1.1, 00:33:50
     192.168.1.0/30 is subnetted, 1 subnets
C       192.168.1.0 is directly connected, FastEthernet0/0
     192.168.2.0/30 is subnetted, 1 subnets
B       192.168.2.0 [20/0] via 192.168.1.1, 00:33:51
     192.168.3.0/30 is subnetted, 1 subnets
B       192.168.3.0 [20/0] via 192.168.1.1, 00:33:51

R4 Advertising routes learned via EIGRP into BGP.  R4 prefers EIGRP routes over BGP routes.

R4#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, + - replicated route

Gateway of last resort is not set

      10.0.0.0/16 is subnetted, 3 subnets
D        10.4.0.0 [90/156160] via 172.30.240.37, 00:47:11, GigabitEthernet0/0
D        10.5.0.0 [90/161536] via 172.30.240.37, 00:47:11, GigabitEthernet0/0
B        10.6.0.0 [20/0] via 192.168.2.1, 00:42:12
      172.16.0.0/32 is subnetted, 2 subnets
D        172.16.49.7
           [90/158976] via 172.30.240.37, 00:47:11, GigabitEthernet0/0
D        172.16.50.7
           [90/156416] via 172.30.240.37, 00:47:11, GigabitEthernet0/0
      172.30.0.0/16 is variably subnetted, 6 subnets, 2 masks
C        172.30.240.36/30 is directly connected, GigabitEthernet0/0
L        172.30.240.38/32 is directly connected, GigabitEthernet0/0
D        172.30.240.40/30
           [90/28416] via 172.30.240.37, 00:47:12, GigabitEthernet0/0
D        172.30.246.8/30
           [90/30976] via 172.30.240.37, 00:47:12, GigabitEthernet0/0
D        172.30.247.12/30
           [90/33536] via 172.30.240.37, 00:47:12, GigabitEthernet0/0
D        172.30.247.20/30
           [90/33792] via 172.30.240.37, 00:47:12, GigabitEthernet0/0
      192.168.1.0/30 is subnetted, 1 subnets
B        192.168.1.0 [20/0] via 192.168.2.1, 00:42:13
      192.168.2.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.2.0/30 is directly connected, GigabitEthernet0/1
L        192.168.2.2/32 is directly connected, GigabitEthernet0/1
      192.168.3.0/30 is subnetted, 1 subnets
B        192.168.3.0 [20/0] via 192.168.2.1, 00:42:14

[Jon Marshall]

Do you want to use the MPLS connection as backup for traffic between the DCs if the direct link fails ?

Jon

[jwegleitner]

Yes, that is one of the goals.  If that wasn't the case, I could simple filter those routes from the MPLS provider.

[Jon Marshall · Accepted solution]

Okay, did a quick lab and you can actually get both routers using EIGRP and advertising but with the configuration you have it is not reliable.

To try and explain what I think is happening I'll use an example.

Key thing to understand is a route can only be redistributed if it is in the IP routing table.

So R3 and R4 are both up and using their EIGRP routes which are being redistributed into BGP.

R3 loses it's connection to the internal switch and so loses it's EIGRP routes. It then accepts the BGP routes from it's PE peer and installs them in the IP routing table which is what you want.

R3's connection then comes back up and it receives the EIGRP routes again but they cannot be installed into the routing table because of the AD of the BGP routes already there.

So R3 cannot then redistribute EIGRP and so cannot advertise those routes to it's PE peer.

If R4 then lost it's internal connection the same would happen in reverse ie. R3 would then not receive any routes via BGP  because R4 cannot advertise them so R3 can then redistribute EIGRP into BGP and advertise them and R4 receives them.

Again what you want until R4's connection comes back up and it cannot then advertise the networks to it's PE device because of the above.

Hope I haven't just explained something you already fully understood.

This I think is what is happening.

The solution that comes to mind is to make the AD of the EIGRP routes always better than the AD of the BGP routes on R3 and R4 so it will always use the EIGRP routes if they are available and therefore will always be able to do the redistribution and advertisement of the BGP routes to the BGP peer.

It's late where I am and I'm a bit tired after today so at the moment my brain is not quite working as well as it should :-) so I'm not entirely sure which is better at the moment ie. increase BGP AD or decrease EIGRP AD.

Maybe someone else reading this could add to it.

It would need testing anyway to make sure there were no unexpected consequences but I think that would fix your problem.

Jon

[jwegleitner]

Thanks for the detailed explanation.  Your description of WHY routing is behaving the way it is makes perfect sense to me.  I started down the path you mention by increasing the AD of BGP.  This does present the reverse issue within the EIGRP domain.  This is easy to resolve with filtering.  I then tagged R3 routes with 3, tagged R4 routes with 4.  I then filtered tag 4 from coming into R3.  I filtered tag 3 from coming into R4.  I will do more testing, but these seems to have resolved the stability issues.

[Jon Marshall]

No problem, glad to help.

Not sure I follow when you say it presents a reverse issue with EIGRP but like I say I'm not thinking too straight at the moment so as long as you are happy with it that's fine by me :-)

Jon

[Jon Marshall]

Last post of the evening.

Just worked out what you meant I think.

You meant because you favoured EIGRP over BGP on R3 and R4 that meant it also affected remote sites ie. R3 and R4 see each other as the best paths to remote sites rather than via the MPLS network.

If so sorry I should have thought about that :-)

Jon