Hi,
I am working with this scenario:
I have Router A that has 2 VRFS. In one of them, we'll call it VRF RED, I receive two networks from a remote Router with whom Router A has a vpnv4 iBGP peering:
show route vrf RED
Fri Apr 11 22:05:12.454 CEST
Codes: C - connected, S - static, R - RIP, B - BGP, (>) - Diversion path
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP
i - ISIS, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, su - IS-IS summary null, * - candidate default
U - per-user static route, o - ODR, L - local, G - DAGR, l - LISP
A - access/subscriber, a - Application route
M - mobile route, r - RPL, t - Traffic Engineering, (!) - FRR Backup path
Gateway of last resort is 172.16.0.235 to network 0.0.0.0
B* 0.0.0.0/0 [200/0] via 172.16.0.235 (nexthop in vrf default), 02:09:24
B 192.168.0.176/28 [200/0] via 172.16.0.46 (nexthop in vrf default), 02:06:42
B 192.168.3.128/28 [200/0] via 172.16.0.46 (nexthop in vrf default), 02:06:42
In VRF BLUE I have this:
show route vrf BLUE
Fri Apr 11 22:06:47.695 CEST
Codes: C - connected, S - static, R - RIP, B - BGP, (>) - Diversion path
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP
i - ISIS, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, su - IS-IS summary null, * - candidate default
U - per-user static route, o - ODR, L - local, G - DAGR, l - LISP
A - access/subscriber, a - Application route
M - mobile route, r - RPL, t - Traffic Engineering, (!) - FRR Backup path
Gateway of last resort is 172.16.0.235 to network 0.0.0.0
B* 0.0.0.0/0 [200/0] via 172.16.0.235 (nexthop in vrf default), 02:10:59
B 192.168.0.176/28 [200/0] via 172.16.0.46 (nexthop in vrf default), 02:08:17
B 192.168.3.128/28 [200/0] via 172.16.0.46 (nexthop in vrf default), 02:08:17
B 198.51.100.0/29 [200/0] via 172.16.0.48 (nexthop in vrf default), 00:21:15
In order to make vrf RED aware of the 198.51.100.0/29 I configured a static route:
router static
vrf RED
address-family ipv4 unicast
198.51.100.0/29 vrf BLUE
This seems to be successful, looking at the routing table for the VRF RED now:
show route vrf RED
Fri Apr 11 22:05:12.454 CEST
Codes: C - connected, S - static, R - RIP, B - BGP, (>) - Diversion path
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP
i - ISIS, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, su - IS-IS summary null, * - candidate default
U - per-user static route, o - ODR, L - local, G - DAGR, l - LISP
A - access/subscriber, a - Application route
M - mobile route, r - RPL, t - Traffic Engineering, (!) - FRR Backup path
Gateway of last resort is 172.16.0.235 to network 0.0.0.0
B* 0.0.0.0/0 [200/0] via 172.16.0.235 (nexthop in vrf default), 02:09:24
B 192.168.0.176/28 [200/0] via 172.16.0.46 (nexthop in vrf default), 02:06:42
B 192.168.3.128/28 [200/0] via 172.16.0.46 (nexthop in vrf default), 02:06:42
S 198.51.100.0/29 [1/0] via 0.0.0.0 (nexthop in vrf BLUE), 00:12:59
We do have a route to 198.51.100.0/29. Looking at Router C now (the one that originates the 2 /28 networks in vrf RED)
show ip route vrf RED
Routing Table: RED
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2, m - OMP
n - NAT, Ni - NAT inside, No - NAT outside, Nd - NAT DIA
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
H - NHRP, G - NHRP registered, g - NHRP registration summary
o - ODR, P - periodic downloaded static route, l - LISP
a - application route
+ - replicated route, % - next hop override, p - overrides from PfR
Gateway of last resort is 192.168.0.141 to network 0.0.0.0
B* 0.0.0.0/0 [200/0] via 192.168.0.141, 02:12:31
S 192.168.0.160/28 [1/0] via 192.168.0.177
C 192.168.0.176/28 is directly connected, BDI221
L 192.168.0.186/32 is directly connected, BDI221
192.168.3.0/24 is variably subnetted, 2 subnets, 2 masks
C 192.168.3.128/28 is directly connected, BDI233
L 192.168.3.130/32 is directly connected, BDI233
B 192.168.96.0/20 [200/0] via 192.168.0.141, 02:12:31
198.51.100.0/28 is subnetted, 1 subnets
B 198.51.100.0 [200/0] via 192.168.0.141, 00:00:06
It learns the network as well.
However despite the control-plane looking sound, reachability is not present. It seems that if I use a static route on Router A to make the vrf RED aware that it has to look up the network in vrf BLUE it will only work if within that VRF a route from that network is locally present.
If I configure a loopback interface inside vrf BLUE on A with an IP address that belongs to the 198.51.100.0/28 network and try to ping 192.168.0.186 (from the 192.168.0.176/28 network) the ping will work, but if the router is not locally present in the VRF on A but is learnt from a remote PE, the LSP breaks. Looking at the CEF within the RED vrf:
show cef vrf RED 198.51.100.0/29
Fri Apr 11 21:52:27.354 CEST
198.51.100.0/29, version 148, internal 0x1000001 0x30 (ptr 0x786fad9c) [1], 0x400 (0x786bd0e8), 0x0 (0x0)
Updated Apr 11 21:52:12.846
Prefix Len 29, traffic index 0, precedence n/a, priority 3
gateway array (0x785544c8) reference count 1, flags 0x0, source rib (7), 0 backups
[2 type 3 flags 0x8401 (0x785f4258) ext 0x0 (0x0)]
LW-LDI[type=3, refc=1, ptr=0x786bd0e8, sh-ldi=0x785f4258]
gateway array update type-time 1 Apr 11 21:52:12.847
LDI Update time Apr 11 21:52:12.847
LW-LDI-TS Apr 11 21:52:12.847
via 0.0.0.0/32, 0 dependencies, weight 0, class 0 [flags 0x0]
path-idx 0 NHID 0x0 [0x783f1258 0x0]
next hop VRF - 'BLUE', table - 0xe0000015
next hop 0.0.0.0/32
Load distribution: 0 (refcount 2)
Hash OK Interface Address
0 Y recursive Lookup in table
There's no mention of labels which would explain why using the static route approach works if the route is locally present on the router within that VRF.
Looking at the CEF in vrf BLUE:
show cef vrf BLUE 198.51.100.0/29
Fri Apr 11 21:52:45.412 CEST
198.51.100.0/29, version 351, internal 0x5000001 0x30 (ptr 0x786fa1e4) [1], 0x0 (0x0), 0x208 (0x79fd2010)
Updated Apr 11 21:45:32.714
Prefix Len 29, traffic index 0, precedence n/a, priority 3
gateway array (0x785543d8) reference count 1, flags 0x2038, source rib (7), 0 backups
[1 type 1 flags 0x48441 (0x7a00cad8) ext 0x0 (0x0)]
LW-LDI[type=0, refc=0, ptr=0x0, sh-ldi=0x0]
gateway array update type-time 1 Apr 11 21:45:32.714
LDI Update time Apr 11 21:45:32.714
via 172.16.0.48/32, 5 dependencies, recursive [flags 0x6000]
path-idx 0 NHID 0x0 [0x7a0dac28 0x0]
recursion-via-/32
next hop VRF - 'default', table - 0xe0000000
next hop 172.16.0.48/32 via 24046/0/21
next hop 192.168.100.136/32 Hu0/1/0/1 labels imposed {24011 24354}
Load distribution: 0 (refcount 1)
Hash OK Interface Address
0 Y recursive 24046/0
It looks like if the router tries to recursively look up a route using a static route with a VRF as next-hop, unless the route is locally there, the router is not capable of recursing into a labeled vpnv4 prefix inside another VRF and impose a label stack to reach it. However I am not quite sure my reasoning is correct. Could someone kindly help out?
Thank you.
