OSPF 'capability vrf-lite' command effect on SD-WAN IOS XE

[junglesman]

Hello,

I'd like to get more information about the effect of the 'capability vrf-lite' command on Cisco SD-WAN Edge devices. I'm using IOS-XE Version 17.3.3. I tried some things in the lab to test what happens if the command is there and when it is not but I see no change. The DN-bit is set anyway on Type-5 LSAs, no matter if this command is there or not.
And the DN-bit will be checked anyway, and the prefix will not be redistributed back into OMP + AD will be set to 252 (as it should be according to this: https://www.cisco.com/c/en/us/support/docs/routers/sd-wan/215029-cisco-ios-xe-sd-wan-installs-ospf-exter.html)

So my point is that there seems to be no difference with or without this command and the command is not even listed on the following Cisco page:
https://www.cisco.com/c/en/us/td/docs/routers/sdwan/command/iosxe/qualified-cli-command-reference-guide/m-ospf-commands.html

The main purpose was to disable loop prevention checks but it is not doing that anymore. The old IOS did not set the DN Bit when the command was configured and ignored the DN Bit when it was there and installed the routes. 

Of course, it still makes sense (or let's say it is a must) to configure 'capability vrf-lite' on the receiving LAN router if that also has VRFs with OSPF so it can install the routes into their table but it looks like it makes no difference on the SD-WAN Edge device.

-------------------------------------

router ospf 1111 vrf 1111
capability vrf-lite

[...]
router ospf 2222 vrf 2222
capability vrf-lite

[...]

Prefix 4.4.4.4 is learned from OMP in VPN 1111 and redistributed into OSPF. In the OSPF LSA database it is visible, that the Down bit was set:

ed-01#sh ip ospf database external 4.4.4.4

OSPF Router with ID (3.3.3.3) (Process ID 1111)

Type-5 AS External Link States

LS age: 1300
Options: (No TOS-capability, DC, Downward) <<<<<<<<< DN Bit is set
LS Type: AS External Link
Link State ID: 4.4.4.4 (External Network Number )
Advertising Router: 3.3.3.3
LS Seq Number: 80000002
Checksum: 0x789F
Length: 36
Network Mask: /32
Metric Type: 2 (Larger than any link state path)
MTID: 0
Metric: 16777214
Forward Address: 0.0.0.0
External Route Tag: 0

When route is leaked in the LAN and learned back in the other VRF via OSPF the Down bit is visible:

ed-01#sh ip route vrf 2222 4.4.4.4

Routing Table: 2222 
Routing entry for 4.4.4.4/32
Known via "ospf 2222", distance 252, metric 16777214, type extern 2, forward metric 2 
Redistributing via omp <<<<<<<<<<<<<< But this actually does not happen
Last update from 192.168.2.2 on GigabitEthernet2, 00:03:09 ago
Routing Descriptor Blocks:
* 192.168.2.2, from 3.3.3.3, 00:03:09 ago, via GigabitEthernet2
SDWAN Down

ed-01#sh sdwan omp routes 4.4.4.4/32
VPN PREFIX
----------------------
1111 4.4.4.4/32
<<<<<<<<<<<<<<<<<<<<<<<<<< Not present in VPN 2222, on remote sites either

-------------------------------------

The capability vrf-lite had another effect on MPLS environment because due to this the OSPF router will stop being connected to the MPLS Superbackbone: http://wiki.kemot-net.com/mpls-inter-as-option-a  (causing a problem when OSPF is between ASBRs) but I don't think it has to be considered when we have an SD-WAN Fabric.

[Kanan Huseynli]

Hi,

do you allow router to redistribute OSPF external into OMP? AFAIK, it is now allowed by default.

[junglesman]

Hi,

Right, I forgot that but even after I configured 'advertise ospf external' this route is not redistributed back into OMP. Most probably because of the DN Bit. So still no "loop" which is a good thing, but I still have no idea what the "capability vrf-lite" does then on SDWAN IOS.

But thanks anyway!

-------------

ed-01#show sdwan running-config | s omp

omp

 address-family ipv4 vrf 2222
 advertise ospf external   << 

[JOHNALLAN2098]

Did you ever solve this issue? 

I am struggling to get OSPF external routes into OMP (again, I suspect it’s due to the Dow but that’s present)

thanks 

[Kanan Huseynli]

In general, DN bit prevents router to install OSPF route into RIB, basically LSA with DN bit should be ignored, unless capability-vrf is configured. This logic happens before redistribution. So, you must not see route with OSPF.

It is clearly written in tech doc:

"Both routers set DN-bit to external LSA type 5 and that should prevent these routes from being installed into the RIB as OSPF routes and hence redistributed back to the OMP, essentially preventing the loop. This is the same mechanism described in RFC 4576 and RFC 4577. "

For me, it is interesting even how your router installs route as OSPF in the VRF 2222. The second issue (which is you mentioned) is why it is not redistributing. Try to clear VRF RIB (soft refresh) and OMP also. Based on output it should be redistributing via OMP.

This was my first assumption, however it looks like it is different. Let me give what I read from SD-WAN CVD (use below link, OSPF section):

"For loop prevention, routes are redistributed from OMP to OSPF as an external OSPF route and the DN bit is set. This prevents other routers from redistributing the route. For the SD-WAN router that receives the OMP to OSPF redistributed route, the OSPF route with the DN bit set is received and assigned an Administrative Distance (AD) of 251 on a vEdge router and 252 on an IOS XE SD-WAN router (AD is one more than the AD on the OMP routes). If OMP disappears, the redistributed route can then be installed in the routing table."

https://www.cisco.com/c/en/us/td/docs/solutions/CVD/SDWAN/cisco-sdwan-design-guide.html#Layer3Redundancy

Based on above, my first understanding is DN prevents router to redistribute route, but route is installed anyway. However, what does "OMP disappears" means, I didnt get. Is it meaning loosing all OMP peers OR loosing exact route from OMP.

Just question: in your lab , is OMP peers reachible when router installs route as OSPF external in VRF 2222?

regards,

[junglesman]

Hi,

Yes, OMP is up, but this route is not learned in VPN 2222 so it means that if the route is missing from OMP, the prefix is installed in the routing table as an OSPF route with AD 252 because of the DN Bit (my route also has AD 252) so the route will be installed from OSPF until it is not learned from OMP but when OMP route appears again it will choose that instead (OMP AD is 251). See https://www.cisco.com/c/en/us/support/docs/routers/sd-wan/215029-cisco-ios-xe-sd-wan-installs-ospf-exter.html

This is all fine and it is described correctly in the documentations but no documentation mentions 'capability vrf-lite' and the behavior is the same as above without this command as well. This is what I don't get. 

The only actual effect is that 'show ip ospf' shows "Connected to MPLS VPN Superbackbone" without the command and it shows the domain-id:

With capability vrf-lite:

ed-01#sh ip ospf 2222
Routing Process "ospf 2222" with ID 3.3.3.4
Start time: 00:10:18.114, Time elapsed: 22w1d
Supports only single TOS(TOS0) routes
Supports opaque LSA
Supports Link-local Signaling (LLS)
Supports area transit capability
Supports NSSA (compatible with RFC 3101)
Supports Database Exchange Summary List Optimization (RFC 5243)
Event-log disabled
It is an autonomous system boundary router
Redistributing External Routes from,
omp, includes subnets in redistribution
Router is not originating router-LSAs with maximum metric
Initial SPF schedule delay 200 msecs
Minimum hold time between two consecutive SPFs 1000 msecs
Maximum wait time between two consecutive SPFs 10000 msecs
Incremental-SPF disabled
Initial LSA throttle delay 50 msecs
Minimum hold time for LSA throttle 200 msecs
Maximum wait time for LSA throttle 5000 msecs
Minimum LSA arrival 100 msecs
LSA group pacing timer 240 secs
Interface flood pacing timer 33 msecs
Retransmission pacing timer 66 msecs
EXCHANGE/LOADING adjacency limit: initial 300, process maximum 300
Number of external LSA 59. Checksum Sum 0x1D6E9B
Number of opaque AS LSA 0. Checksum Sum 0x000000
Number of DCbitless external and opaque AS LSA 0
Number of DoNotAge external and opaque AS LSA 0
Number of areas in this router is 1. 1 normal 0 stub 0 nssa
Number of areas transit capable is 0
External flood list length 0
IETF NSF helper support enabled
Cisco NSF helper support enabled
Reference bandwidth unit is 100 mbps
Area BACKBONE(0)
Number of interfaces in this area is 1
Area has no authentication
SPF algorithm last executed 00:00:19.468 ago
SPF algorithm executed 8 times
Area ranges are
Number of LSA 4. Checksum Sum 0x019576
Number of opaque link LSA 0. Checksum Sum 0x000000
Number of DCbitless LSA 0
Number of indication LSA 0
Number of DoNotAge LSA 0
Flood list length 0

---------------------------------------------------------

Without:

ed-01#sh ip ospf 2222
Routing Process "ospf 2222" with ID 3.3.3.4
Domain ID type 0x0005, value 0.0.8.174
Start time: 00:10:18.114, Time elapsed: 22w1d
Supports only single TOS(TOS0) routes
Supports opaque LSA
Supports Link-local Signaling (LLS)
Supports area transit capability
Supports NSSA (compatible with RFC 3101)
Supports Database Exchange Summary List Optimization (RFC 5243)
Connected to MPLS VPN Superbackbone, VRF 2222
Event-log disabled
It is an area border and autonomous system boundary router
Redistributing External Routes from,
omp, includes subnets in redistribution
Router is not originating router-LSAs with maximum metric
Initial SPF schedule delay 200 msecs
Minimum hold time between two consecutive SPFs 1000 msecs
Maximum wait time between two consecutive SPFs 10000 msecs
Incremental-SPF disabled
Initial LSA throttle delay 50 msecs
Minimum hold time for LSA throttle 200 msecs
Maximum wait time for LSA throttle 5000 msecs
Minimum LSA arrival 100 msecs
LSA group pacing timer 240 secs
Interface flood pacing timer 33 msecs
Retransmission pacing timer 66 msecs
EXCHANGE/LOADING adjacency limit: initial 300, process maximum 300
Number of external LSA 59. Checksum Sum 0x1CF6D8
Number of opaque AS LSA 0. Checksum Sum 0x000000
Number of DCbitless external and opaque AS LSA 0
Number of DoNotAge external and opaque AS LSA 0
Number of areas in this router is 1. 1 normal 0 stub 0 nssa
Number of areas transit capable is 0
External flood list length 0
IETF NSF helper support enabled
Cisco NSF helper support enabled
Reference bandwidth unit is 100 mbps
Area BACKBONE(0)
Number of interfaces in this area is 1
Area has no authentication
SPF algorithm last executed 00:00:46.338 ago
SPF algorithm executed 4 times
Area ranges are
Number of LSA 4. Checksum Sum 0x018B76
Number of opaque link LSA 0. Checksum Sum 0x000000
Number of DCbitless LSA 0
Number of indication LSA 0
Number of DoNotAge LSA 0
Flood list length 0

[Kanan Huseynli]

Hi,

how do you do config capability-vrf lite? I havent seen it in OSPF template.

regards,

[junglesman]

Hi,

With CLI Add-On Template (in case of a Feature device template) or with CLI Device template. Yes, it's probably not in the OSPF Feature template but possible to set on Cisco WAN Edges with IOS XE. 

[Kanan Huseynli]

May be, it does not have any impact in reality.Better to open case (like question), if you have smartnet.

If I find time , I'll do test lab.