LISP Route in "Route_rejected" status?

[bfbcnet]

Hi,

First time post. Apologies in advance if this a too in depth a question for here, but you don't know unless you try.

The scenario is that we are on the journey of moving to SD-Access. We have a normal fabric with a standard VN setup, up and working fine with client devices being able to get out the internet, etc. We are also wanting the try using the Guest VN, Multi-Site Remote border, (or whatever it is called this week) functionality, to have some of our wired devices brake out in the DMZ at our site vs where our more corportate traffic carrying VN breaks out. This feature is not super documented and has gone through an amount of work flows changes over DNA versions. We are running DNA-C (that name appears to have changed as well) 2.3.5.

Due to lack of documentation I have used the guide here: Multisite Remote Border, to set it all up. Clients on this VN though are not being able to get outside of the fabric. The I can ping the internet from Remote Border, from the loopback interface with the same id as the VN handoff for the VN. We are getting a default route coming into the remote border via BGP. The Client devices can ping their default gateway on their edge node. The IP's of edge devices are showing up as registered on the Remote Border that is acting as the Control Plane for this VN as well.

What appears to be going wrong is that the default route from the Remote border is not getting to the edge node. Or more it is but it is in a status of rejected as below out put from the edge node:

-----------------------

XXXX-FEN-01#show lisp instance-id 4100 ipv4 map-cache detail
LISP IPv4 Mapping Cache for LISP 0 EID-table vrf PUBLIC_WIRED_VN (IID 4100), 2 entries

0.0.0.0/0, uptime: 00:03:39, expires: 00:11:20, via map-reply, unknown-eid-forward
Sources: map-reply, static-send-map-request
State: unknown-eid-forward, last modified: 00:03:39, map-source: X.X.1.66
Exempt, Packets out: 8(4608 bytes), counters are not accurate (~ 00:25:21 ago)
Configured as EID address space
action: send-map-request + Encapsulating to proxy ETR
PETR Uptime State Pri/Wgt Encap-IID Domain-ID/MH-ID Metric
X.X.1.66 00:30:32 route-rejec 10/10 - 2292703398/57510 0

-----------------------------------

I am stuck troubleshooting further, as I don't know what that status means and therefore where to go next. I of course have googled that lisp status but have not had any matching results. Is anyone in the community able to shed any light on this status?

It does not help that what guides are available, are for earlier version of the 2.x DNA train or before and this setup / workflow has gone through a lot of changes.

I have also read through the DNA 2.3.5 user guide, but multi-site remote border setups do not seem to be covered, so whether this it is not supported anymore? Anyway if is anyone in the community able to shed any light on this status, or point me at where it might be documented, that would be super helpful.

[andy!doesnt!like!uucp]

hi
from above output , X.X.1.66 is assumed to be PETR for your site for specified VN. did u check legacy route 0.0.0.0 presence in target VRF on PETR?

[jalejand]

Route-reject means that you are not meeting the reachability criteria for that RLOC.

Borders use: ipv4 locator reachability exclude-default
Edges use: ipv4 locator reachability minimum-mask-lenght 32

This tracks RLOC reachability using the underlay (global) RIB.
The first one requires any route except a 0.0.0.0/0 route to make it reachable
The second one requires a /32 route to make it reachable

Either way, I suggest using /32 and not summaries to achieve faster convergence in any scenario.

[andy!doesnt!like!uucp]

it's not useful explanation. what has to be checked on the EN? what has be checked on BN?
from your output u advice to check PETR's RLOC has /32 entry in the underlay RIB on the EN or any less specific except 0.0.0.0/0? 

ultimately isnt it something to be configured on both EN & BN by DNAC?

[jalejand]

Well the RLOC is X.X.1.66 in XXXX-FEN-01, so based on what I said, he must check the reachability criteria on FEN01 to x.x.1.66. Is it using a 0.0.0.0/0 route to reach that RLOC in the global RIB? If so, modify the underlay to announce a /32 to x.x.1.66

[andy!doesnt!like!uucp]

@bfbcnet
try this sho ip cef X.X.1.66 in global or VRF context
@jalejand let me to ask u one more time: whilst diagnosis is still matter of CLI commands, isnt configuration of underlay, router lisp & router bgp DNACs job just to reduce manual work so carefully emerged by Cisco SDA?

[jalejand]

If the entire underlay was designed via DNAC (lan automation) then yeah, the ISIS underlay that is configured will have no summarization or way to block /32 advertisements. However, if this is multi-site remote border, that often implies that a different type of underlay already exists in between FE and MSRB, which DNAC could not control or manage. These underlay reachability conditions (/32, exclude default and more) unfortunately cannot be relaxed as it might cause more harm than benefit (blackholing and more traffic loss scenarios).

[andy!doesnt!like!uucp]

so... there is something that DNAC cant manage in SDA, it's a pity.
thank you for explanations

[bfbcnet]

Thanks for the responses. So just to answer a couple of things raised.

The Loopback address of the edge node can ping the Remote border node and the same vice versa for the Remote border’s loopback address. Also the Loopback with the same ID as the VLAN assigned for the VN can reach an internet address.

------------------------------
REMOTE-BORDER#ping X.X.1.97 source loopback 0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to X.X.1.97, timeout is 2 seconds:
Packet sent with a source address of X.X.1.66
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 2/2/2 ms

REMOTE-BORDER #ping vrf PUBLIC_WIRED_VN 8.8.8.8 source lo2XX
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 8.8.8.8, timeout is 2 seconds:
Packet sent with a source address of X.X.10.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 10/11/13 ms

----------------

EDGE-NODE#ping X.X.1.66 source loopback 0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to X.X.1.66, timeout is 2 seconds:
Packet sent with a source address of X.X.1.97
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 2/2/2 ms
--------------------------------

On the remote border it has a default route fed in by BGP for the VN, over the L3 Handoff that was setup via DNAC, as can be seen below. This appears to be in the LISP table on the remote border.

-----------------------------------------
REMOTE-BORDER#show ip route vrf PUBLIC_WIRED_VN

-Snip-

Gateway of last resort is X.X.9.2 to network 0.0.0.0

B* 0.0.0.0/0 [20/0] via X.X.9.2, 00:00:29
-Snip-

REMOTE-BORDER#show lisp instance-id 4100 ipv4 database
LISP ETR IPv4 Mapping Database for LISP 0 EID-table vrf PUBLIC_WIRED_VN (IID 4100), LSBs: 0x1
Entries total 53, no-route 0, inactive 0, do-not-register 0

0.0.0.0/0, locator-set DEFAULT_ETR_LOCATOR, default-ETR
Uptime: 00:11:35, Last-change: 00:10:18
Domain-ID: local
Metric: 0
Service-Insertion: N/A
Locator Pri/Wgt Source State
X.X.1.66 10/10 cfg-intf site-self, reachable
----------------------

For the Edge Node it’s break out is via it’s fabric border to get via an underlay network to the Remote border.. This border has a specific vs a the default route for the ‘Remote Border’.s Loopback address. This has made it's way to the GRT on the EDGE NODE

----------------------
SITE-BORDER#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
-Snip-
B X.X.1.66/32 [20/0] via X.X.253.162, 2d23h

----------------------

EDGE-NODE#sho ip cef X.X.1.66
X.X.1.66/32
nexthop X.X.254.2 TwentyFiveGigE1/1/1
nexthop X.X.254.5 TwentyFiveGigE1/1/2
----------------------

So, I have met the conditions for not falling foul of the ipv4 locator reachability controls? Or am I misunderstanding where these /32 routes need to be in the various routing tables (Edge, site border, remote border, GRT table or VN Table), to stop this restriction kicking in? I am sorry, the LISP stuff is still new territory from me and I did not dig into it much in advance as was under the maybe mistaken assumption, that DNAC would be doing the heavy lifting on setting it up.

[jalejand · Accepted solution]

In your output:

XXXX-FEN-01#show lisp instance-id 4100 ipv4 map-cache detail
LISP IPv4 Mapping Cache for LISP 0 EID-table vrf PUBLIC_WIRED_VN (IID 4100), 2 entries

0.0.0.0/0, uptime: 00:03:39, expires: 00:11:20, via map-reply, unknown-eid-forward
Sources: map-reply, static-send-map-request
State: unknown-eid-forward, last modified: 00:03:39, map-source: X.X.1.66
Exempt, Packets out: 8(4608 bytes), counters are not accurate (~ 00:25:21 ago)
Configured as EID address space
action: send-map-request + Encapsulating to proxy ETR
PETR Uptime State Pri/Wgt Encap-IID Domain-ID/MH-ID Metric
X.X.1.66 00:30:32 route-rejec 10/10 - 2292703398/57510 0

XXXX-FEN-01 has a route-reject state for X.X.1.66, meaning you need to check the RIB (not CEF as it can be influeced by an SGT or recursive/tracking state) for that prefix.

You must have an /32 route for that X.X.1.66, what is the "show ip route X.X.1.66"  output?

[bfbcnet]

Hi,

Again that for all the comments. Mystery solved. The Border Node had a /32 for X.X.1.66 (via BGP L3 handoff), but the Fabric Edge Node (FEN-01) did not. This was due to the BGP routes on the Border not fully being redistributed into the Underlay routing protocol for the fabric (IS-IS). So I logged in into the border and manually put in a redistribution of BGP into the IS-IS. Yes that was just to prove out what is going on. I will push via template going forwards, instead of CLI. I kind of assumed DNAC would look after that. Put it down being overly cautious with the config I am pushing as not to step on DNAC's toes. It has be the most difficult part if this for me to grasp, what DNAC does and what then I need to do to fill in the gaps.

So again thanks for the pointers. Are there any other LISP gotas, like the needing /32's for certain routes, documented somewhere that I should be aware of?

Next is seeing whether the Multi-Site Remote Border setup is compatible with IP Transit setups...

[andy!doesnt!like!uucp]

a lot of. specifically pay attention to what is configured on the L2-handoff BNs switched interfaces. when or before you deploy new VLAN on the L2-handoff interface in the BN configuration. in my case there were several VLANs allowed. & when i deployed new VLAN to interface it didnt show up among allowed those.