OSPF Equal-Cost Load Balancing not working as expected

[tlxbx]

• I have a situation where ASR-903 has OSPF adj. with 2 ASR1k. ASR1k are in HSRP pair.
• ASR1k is re-distributing BGP into OSPF so ASR-903 learns the same subnets from Both ASR1k. OSPF & BGP configs are EXACTLY same on both. 
• However the issue is ASR-903 routing table only includes routes learned from ASR1k-A but not B.
• OSPF database on ASR903 shows the subnet is learned from Both. 
• When I issue 

  sh ip ospf database external 

  I noticed one has Forward Address: 0.0.0.0 while other has Forward Address: 172.27.200.106. Not sure why this is but the one with 0.0.0.0 is added in RIB. 

So why it is not doing the ECMP and suppressing one subnet?

One of the actual example below from the Router. 

ASR903-RTR#sh ip ospf neighbor

Neighbor ID Pri State Dead Time Address Interface
10.255.1.9 1 2WAY/DROTHER 00:00:54 172.27.137.2 Port-channel1
10.255.1.10 1 FULL/BDR 00:00:58 172.27.137.3 Port-channel1

***************************************************************************************

ASR903-RTR#sh ip route ospf | i 172.27.144.
O E1 172.27.144.0/23 [110/2] via 172.27.137.2, 2w1d, Port-channel1

***************************************************************************************
ASR903-RTR#sh ip ospf database | i 172.27.144.
172.27.144.0 10.255.1.9 1028 0x80001CDE 0x00854C 64512
172.27.144.0 10.255.1.10 1660 0x80001CDB 0x003A9E 64512
***************************************************************************************

ASR903-RTR#sh ip ospf database external 172.27.144.0

OSPF Router with ID (172.27.137.4) (Process ID 1)

Type-5 AS External Link States

LS age: 320
Options: (No TOS-capability, DC, Upward)
LS Type: AS External Link
Link State ID: 172.27.144.0 (External Network Number )
Advertising Router: 10.255.1.9
LS Seq Number: 80001CFE
Checksum: 0x456C
Length: 36
Network Mask: /23
Metric Type: 1 (Comparable directly to link state metric)
MTID: 0
Metric: 1
Forward Address: 0.0.0.0
External Route Tag: 64512

LS age: 1330
Options: (No TOS-capability, DC, Upward)
LS Type: AS External Link
Link State ID: 172.27.144.0 (External Network Number )
Advertising Router: 10.255.1.10
LS Seq Number: 80001CFB
Checksum: 0xF9BE
Length: 36
Network Mask: /23
Metric Type: 1 (Comparable directly to link state metric)
MTID: 0
Metric: 1
Forward Address: 172.27.200.106
External Route Tag: 64512

[tlxbx]

Correct. asr-a goes to nx-a & asr-b goes to nx-b. 

Still doesn't explain why asr-903 is not adding the subnet in its RIB with a non-zero forwarding address as the non-zero address is being learned from OSPF. So from the doc. I read it should be added to the routing table. 

[Harold Ritter]

Hi @tlxbx ,

Please refer to the following document to find out why you are not using both external routes on the ASR903.

https://www.cisco.com/c/en/us/support/docs/ip/open-shortest-path-first-ospf/25493-type5-lsa.html

More specifically this section:

You can see above that the type 5 LSA generated by Router 3 now has the forwarding address set to 172.16.3.8, while the type 5 LSA generated by Router 4 still has a forwarding address of 0.0.0.0.

In this case, the LSA to be installed in the Router 5 routing table is determined by comparing the Router 5 metric to the ASBR (Router 4) that generated the LSA with a forwarding address of 0.0.0.0 to the Router 5 metric to reach the forwarding address of 172.16.3.8, which was set for the LSA generated by the ASBR (Router 3).

To see the metrics to the ASBRs, issue the

show ip ospf border 

command in Router 5. The output is shown below.

Regards,

[tlxbx]

Both metrics looks the same

i 10.255.1.9 [1] via 172.27.137.2, Port-channel1, ASBR, Area 0, SPF 91
i 10.255.1.10 [1] via 172.27.137.3, Port-channel1, ASBR, Area 0, SPF 91

I think @MHM Cisco World is on point. This has to do smth with BGP. We had 7606 prior to ASR903 and looking into old output I see it only had 1 subnet as well rather than ECMP. 

[Harold Ritter]

Hi @tlxbx ,

You will load-balance between the two ASBRs only if you make sure that both external routes have a forwarding address or both of them don't have a forwarding address.

Regards,

[MHM Cisco World]

You mention that asr-a learn prefix from nx-b not nx-a,

Do see next hope of prefix

Show ip route < next hope> 

From where it learn 

Ospf

Static

Bgp

[tlxbx]

asr-a learns via BGP from both nexus a&b but uses nx-b as the best path. I am not sure why as I mentioned above all metrics are the same & nx-a rid is lower than nx-b. Also nx-a uptime is 16 min higher than nx-b so most likely its BGP was established sooner so IDK why it chooses nx-b path as best. 

ASR1001-A#sh ip bgp 172.27.144.0
BGP routing table entry for 172.27.144.0/23, version 685
BGP Bestpath: med
Paths: (2 available, best #2, table default)
Advertised to update-groups:
4
Refresh Epoch 1
64512, (received & used)
172.27.200.110 from 172.27.200.110 (10.0.100.130) --> NX-A(lower RID)
Origin incomplete, metric 0, localpref 100, valid, external
Extended Community: RT:64512:50000
rx pathid: 0, tx pathid: 0
Refresh Epoch 1


64512, (received & used)
172.27.200.114 from 172.27.200.114 (10.0.100.131) -->NX-B(Higher RID)
Origin incomplete, metric 0, localpref 100, valid, external, best
Extended Community: RT:64512:50000
rx pathid: 0, tx pathid: 0x0

[paul driver]

Hello
The FA in a external LSA is an adverted address in ospf address that is the next hop towards that specific network- usually this is set to all zeros meaning there is a direct ospf connection to the ospf rtr that is advertising the route.

If the FA is a non zero address then this is stating there is an alternative path towards this network via "this ASBR" which can be be taken but it would be classed as a longer path as such less preferred path and this FA address is usually take from a ospf loopback address if one is configured if not then its taken from the oldest ospf active interface.

sh ip inter brief
sh ip ospf inter brief