OSPF route preferred over BGP route

[rick505d3]

Hi,

In the network below, the objective is redundant reachability to prefix 1.1.1.1 (say public cloud hosting) routers csr2 and csr3 (internal network).

In the order of operations, BGP was configured and verified first, followed by OSPF configuration. The working state is like this - csr2 and csr3 can reach 1.1.1. directly via BGP:

csr2#show ip bgp 1.1.1.1
BGP routing table entry for 1.1.1.1/32, version 2
Paths: (1 available, best #1, table default)
Not advertised to any peer
Refresh Epoch 1
1, (received & used)
10.0.12.1 from 10.0.12.1 (1.1.1.1)
Origin IGP, metric 0, localpref 100, valid, external, best
rx pathid: 0, tx pathid: 0x0

csr2#show ip ospf database ext 1.1.1.1

            OSPF Router with ID (2.2.2.2) (Process ID 1)

                Type-5 AS External Link States

  LS age: 82
  Options: (No TOS-capability, DC, Upward)
  LS Type: AS External Link
  Link State ID: 1.1.1.1 (External Network Number )
  Advertising Router: 2.2.2.2
  LS Seq Number: 80000001
  Checksum: 0x9704
  Length: 36
  Network Mask: /32
        Metric Type: 2 (Larger than any link state path)
        MTID: 0
        Metric: 1
        Forward Address: 0.0.0.0
        External Route Tag: 12

  LS age: 45
  Options: (No TOS-capability, DC, Upward)
  LS Type: AS External Link
  Link State ID: 1.1.1.1 (External Network Number )
  Advertising Router: 3.3.3.3
  LS Seq Number: 80000001
  Checksum: 0x8B0B
  Length: 36
  Network Mask: /32
        Metric Type: 2 (Larger than any link state path)
        MTID: 0
        Metric: 1
        Forward Address: 0.0.0.0
        External Route Tag: 13

csr3#show ip bgp 1.1.1.1
BGP routing table entry for 1.1.1.1/32, version 2
Paths: (1 available, best #1, table default)
  Not advertised to any peer
  Refresh Epoch 2
  1, (received & used)
    10.0.13.1 from 10.0.13.1 (1.1.1.1)
      Origin IGP, metric 0, localpref 100, valid, external, best
      rx pathid: 0, tx pathid: 0x0

csr3#show ip ospf data ext 1.1.1.1

            OSPF Router with ID (3.3.3.3) (Process ID 1)

                Type-5 AS External Link States

  LS age: 214
  Options: (No TOS-capability, DC, Upward)
  LS Type: AS External Link
  Link State ID: 1.1.1.1 (External Network Number )
  Advertising Router: 2.2.2.2
  LS Seq Number: 80000001
  Checksum: 0x9704
  Length: 36
  Network Mask: /32
        Metric Type: 2 (Larger than any link state path)
        MTID: 0
        Metric: 1
        Forward Address: 0.0.0.0
        External Route Tag: 12

  LS age: 176
  Options: (No TOS-capability, DC, Upward)
  LS Type: AS External Link
  Link State ID: 1.1.1.1 (External Network Number )
  Advertising Router: 3.3.3.3
  LS Seq Number: 80000001
  Checksum: 0x8B0B
  Length: 36
  Network Mask: /32
        Metric Type: 2 (Larger than any link state path)
        MTID: 0
        Metric: 1
        Forward Address: 0.0.0.0
        External Route Tag: 13

Now, shut/no shut the BGP neigh 10.0.12.1 on csr2 - or shut/noshut the interface long enough for bgp to go down. The state now looks like this:

csr2#show ip bgp 1.1.1.1
BGP routing table entry for 1.1.1.1/32, version 7
Paths: (2 available, best #2, table default)
  Not advertised to any peer
  Refresh Epoch 2
  1, (received & used)
    10.0.12.1 from 10.0.12.1 (1.1.1.1)
      Origin IGP, metric 0, localpref 100, valid, external
      rx pathid: 0, tx pathid: 0
  Refresh Epoch 1
  Local
    10.0.23.3 from 0.0.0.0 (2.2.2.2)
      Origin incomplete, metric 1, localpref 100, weight 32768, valid, sourced, best
      rx pathid: 0, tx pathid: 0x0


csr2#show ip ospf data ext 1.1.1.1

            OSPF Router with ID (2.2.2.2) (Process ID 1)

                Type-5 AS External Link States

  LS age: 358
  Options: (No TOS-capability, DC, Upward)
  LS Type: AS External Link
  Link State ID: 1.1.1.1 (External Network Number )
  Advertising Router: 3.3.3.3
  LS Seq Number: 80000001
  Checksum: 0x8B0B
  Length: 36
  Network Mask: /32
        Metric Type: 2 (Larger than any link state path)
        MTID: 0
        Metric: 1
        Forward Address: 0.0.0.0
        External Route Tag: 13

csr2#show ip route 1.1.1.1
Routing entry for 1.1.1.1/32
  Known via "ospf 1", distance 110, metric 1
  Tag 13, type extern 2, forward metric 1
  Redistributing via bgp 2
  Advertised by bgp 2 match internal external 1 & 2
  Last update from 10.0.23.3 on GigabitEthernet3, 00:28:14 ago
  Routing Descriptor Blocks:
  * 10.0.23.3, from 3.3.3.3, 00:28:14 ago, via GigabitEthernet3
      Route metric is 1, traffic share count is 1
      Route tag 13

csr3#show ip bgp 1.1.1.1
BGP routing table entry for 1.1.1.1/32, version 2
Paths: (1 available, best #1, table default)
  Not advertised to any peer
  Refresh Epoch 3
  1, (received & used)
    10.0.13.1 from 10.0.13.1 (1.1.1.1)
      Origin IGP, metric 0, localpref 100, valid, external, best
      rx pathid: 0, tx pathid: 0x0

csr3#show ip ospf data ext 1.1.1.1

            OSPF Router with ID (3.3.3.3) (Process ID 1)

                Type-5 AS External Link States

  LS age: 375
  Options: (No TOS-capability, DC, Upward)
  LS Type: AS External Link
  Link State ID: 1.1.1.1 (External Network Number )
  Advertising Router: 3.3.3.3
  LS Seq Number: 80000001
  Checksum: 0x8B0B
  Length: 36
  Network Mask: /32
        Metric Type: 2 (Larger than any link state path)
        MTID: 0
        Metric: 1
        Forward Address: 0.0.0.0
        External Route Tag: 13

csr3#show ip route 1.1.1.1
Routing entry for 1.1.1.1/32
  Known via "bgp 2", distance 20, metric 0
  Tag 1, type external
  Redistributing via ospf 1
  Advertised by ospf 1 subnets tag 13
  Last update from 10.0.13.1 00:36:12 ago
  Routing Descriptor Blocks:
  * 10.0.13.1, from 10.0.13.1, 00:36:12 ago
      Route metric is 0, traffic share count is 1
      AS Hops 1
      Route tag 1
      MPLS label: none

Here, if I shut/noshut csr2:G3 (to drop/re-estab ospf adj), then csr2 will route to 1.1.1.1 via bgp again.

What's causing this behavior and how to fix this?

Thanks,

Rick.

[rick505d3 · Accepted solution]

Found the problem. Must not redistribute 1.1.1.1 from ospf back into BGP on csr2 or csr3. I think it must a timing thing as it works initially but after a bgp failure event, ospf redistributed route which is already in the bgp database isn't getting kicked out by ebgp learnt same route.

Filtering can be done based on tag or prefix match - done using deny on prefix below for csr2. Similar on csr3 as well.

router bgp 2
 redistribute ospf 1 match internal external 1 external 2 route-map rm2-ospf-into-bgp

route-map rm2-ospf-into-bgp deny 10
 match ip address prefix-list rm2-ospf-into-bgp-1
route-map rm2-ospf-into-bgp permit 20

ip prefix-list rm2-ospf-into-bgp-1 seq 5 permit 1.1.1.1/32

Regards,

Rick.

[paul driver · Accepted solution]

Hello

It a race condition
BGP advertises a ebgp route with a admin distance of 20 and no weight, however a redistributed igp route into bgp is advertised with a admin distance of 20 plus a default weight of 32768,

So when you shut/no shut the ebgp links from bgp1 between either 2 or 3, then 1.1.1.1 will be redistributed from bgp-into ospf and then from ospf back into bgp and in bgp best path selection weight has precedence this is what you see.

localpref 100, weight 32768, valid, sourced, best

To negate this, set the default weight on rtr 2/3 to be higher than 32768
neighbor xxxx weight 40000