Hello @jacky.cheuk ,
Thank you for the outputs. We are getting somewhere!
sw01 is not directly connected to area 4 where the 172.24.2.0/24 network lives, so it computes the best path to it through an ABR in area 0 that provides the least total cost to that network - which consists of the cost to reach the ABR inside area 0, plus the distance of that ABR to reach 172.24.2.0/24 which is advertised in the LSA3 from the ABR. Looking at sw01, it prefers the path throught the ABR 192.168.13.4, and the next hop toward that ABR goes through r3:
sw01# show ip route 172.24.2.0 255.255.255.0
Routing entry for 172.24.2.0/24
Known via "ospf 64", distance 110, metric 358, type inter area
Last update from 10.42.252.145 on TenGigabitEthernet1/2/1, 00:01:57 ago
Routing Descriptor Blocks:
* 10.42.252.145, from 192.168.13.4, 00:01:57 ago, via TenGigabitEthernet1/2/1
Route metric is 358, traffic share count is 1
sw01# show ip ospf border-routers
i 192.168.13.4 [106] via 10.42.252.145, TenGigabitEthernet1/2/1, ABR/ASBR, Area 0, SPF 7357
sw01# show ip ospf database summary 172.24.2.0
Routing Bit Set on this LSA in topology Base with MTID 0
LS age: 118
Options: (No TOS-capability, DC, Upward)
LS Type: Summary Links(Network)
Link State ID: 172.24.2.0 (summary Network Number)
Advertising Router: 192.168.13.4
LS Seq Number: 80000001
Checksum: 0xEF0F
Length: 28
Network Mask: /24
MTID: 0 Metric: 252
Note that the distance from sw01 to the ABR 192.168.13.4 is 106, and the ABR's distance to the destination network is 252, and 106+252=358 - exactly the sw01's total cost.
So in order to reach 172.24.2.0/24, sw01 picks r3 as the next hop, so let's see what's happening on r3:
rt03# show ip route 172.24.2.0 255.255.255.0
Routing entry for 172.24.2.0/24
Known via "ospf 100", distance 110, metric 1001, type intra area
Last update from 192.168.39.182 on Tunnel1620, 4w5d ago
Routing Descriptor Blocks:
* 192.168.39.182, from 10.42.249.23, 4w5d ago, via Tunnel1620
Route metric is 1001, traffic share count is 1
rt03# show ip ospf border-routers
...
i 192.168.13.4 [105] via 192.168.2.2, Tunnel1601, ABR/ASBR, Area 0, SPF 5232
...
rt03# show ip ospf database summary 172.24.2.0
OSPF Router with ID (10.42.249.80) (Process ID 100)
Summary Net Link States (Area 0)
LS age: 235
Options: (No TOS-capability, DC, Upward)
LS Type: Summary Links(Network)
Link State ID: 172.24.2.0 (summary Network Number)
Advertising Router: 10.42.249.80
LS Seq Number: 800005A4
Checksum: 0x1250
Length: 28
Network Mask: /24
MTID: 0 Metric: 1001
LS age: 578
Options: (No TOS-capability, DC, Upward)
LS Type: Summary Links(Network)
Link State ID: 172.24.2.0 (summary Network Number)
Advertising Router: 192.168.13.4
LS Seq Number: 80000001
Checksum: 0xEF0F
Length: 28
Network Mask: /24
MTID: 0 Metric: 252
...
r3 is well aware of the existence of ABR 192.168.13.4 and its LSA3. But the gotcha is that r3 is also an ABR with area 4 whicih is the home area for 172.24.2.0/24, and as you know, in OSPF, intra-area paths are always preferred to inter-area paths, no matter the cost. Therefore, even if it was a cheaper path for r3 to use the path through 192.168.13.4, it must not - the OSPF rules do not allow that. It has to ignore the cheaper inter-area path and stick with the intra-area path which happens to be more costly.
You may ask how is it possible, then, that sw01 does not derive its cost from the real cost of r3.
The answer lies in the nature of link-state routing. sw01, not being a member of area 4, does not see its topology. sw01 can only see the topology of those areas it is a member of. About networks in other areas, all it knows is what ABRs advertise them into area 0, how far the ABRs are from those networks, and how far sw01 is from those ABRs.
So when sw01 runs the shortest-path-first algorithm on this knowledge, obviously, it sees that the ABR 192.168.13.4 provides the least combined cost to 172.24.2.0/24. The fact that r3 also advertises 172.24.2.0/24 in its own LSA3 does not matter much to sw01 - the combined cost is too high for sw01 to be interested. The best path is through the ABR 192.168.13.4, and so sw01 decides to use that path.
But the path always starts with the next hop, and the next from from sw01 toward the ABR 192.168.13.4 still happens to be r3. And now notice what happens: Because r3 is a member of area 4 where 172.24.2.0/24 is located, r3 does not route based on area 0's knowledge which is what sw01 did - it routes based on the area 4 knowledge, and that's where the paths start diverging, along with their cost.
At the bottom of this disparity is the fact that because of OSPF rules (prefer intra-area paths to inter-area paths), sw01 and r3 used different link-state databases to compute the best path. sw01 used the topology from area 0, r3 used the topology from area 4.
This phenomenon in OSPF may occur at any time where an inter-area path through an ABR traverses through another transit ABR, because that transit ABR will be forced by OSPF rules to
- prefer intra-area paths if the destination network lives in another area the transit ABR is connected to
- only use area 0 for inter-area traffic if the destination network lives in yet another area that the transit ABR is not connected to
I know this is quite mind-boggling and will require some time to digest. If this explanation does not quite cut it, we can pull additional outputs from your routers to demonstrate how exactly this happened.
Best regards,
Peter
