Re: BGP, NX-OS, "is backup urib route"

mfarrenkopf · ‎07-16-2019

I work in a large environment. We use MPLS and multiple L3VPN instances. Our data centers are Nexus 7700s.

As part of our standard config, when we redistribute from iBGP to EIGRP, we use an external EIGRP AD of 250.

I have seen, off and on, where a route is in BGP and EIGRP, the BGP ought to be chosen, yet it says the route is a backup route and the EIGRP route -- with an AD of 250 -- takes precedence. Here are a couple of output snippets.

B-Core# show ip bgp vrf MAIN 10.137.0.0/16
BGP routing table information for VRF MAIN, address family IPv4 Unicast
BGP routing table entry for 10.137.0.0/16, version 64922
Paths: (1 available, best #1)
Flags: (0x80080012) on xmit-list, is in urib, is backup urib route, is in HW
vpn: version 223786, (0x100002) on xmit-list

Advertised path-id 1, VPN AF advertised path-id 1
Path type: internal, path is valid, is best path, in rib
Imported from 1200:12:10.137.0.0/16
AS-Path: NONE, path sourced internal to AS
172.21.255.149 (metric 201) from 172.21.255.1 (172.21.255.1)
Origin incomplete, MED 0, localpref 100, weight 1000
Received label 178
Extcommunity:
RT:1300:12
Originator: 172.21.255.149 Cluster list: 172.21.255.1
Connector Attribute: count=1
type 1 len 12 value 1200:12:172.21.255.149

B-Core# show ip route 10.137.0.0/16 vrf MAIN
IP Route Table for VRF "MAIN"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

10.137.0.0/16, ubest/mbest: 2/0
*via 10.81.251.1, Eth1/10.12, [250/51456], 33w0d, eigrp-MAIN, external, tag 65001
*via 10.86.251.52, Po2.12, [250/51456], 28w5d, eigrp-MAIN, external, tag 65001
via 172.21.255.149%default, [200/0], 1w6d, bgp-65001, internal, tag 65001 (mpls-vpn)

Why is NX-OS installing the EIGRP route over the BGP route?

I'm happy to add additional outputs as needed.

Giuseppe Larosa · ‎07-16-2019

Hello mfarrenkopf,

at first sight I think it might be related to the fact that the MP iBGP next-hop is actually in the Global routing table.

This is normal and standard IOS does not care about this.

>>

B-Core# show ip route 10.137.0.0/16 vrf MAIN
IP Route Table for VRF "MAIN"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

10.137.0.0/16, ubest/mbest: 2/0
*via 10.81.251.1, Eth1/10.12, [250/51456], 33w0d, eigrp-MAIN, external, tag 65001
*via 10.86.251.52, Po2.12, [250/51456], 28w5d, eigrp-MAIN, external, tag 65001
via 172.21.255.149%default, [200/0], 1w6d, bgp-65001, internal, tag 65001 (mpls-vpn)

Here %default means the next-hop comes from VRF default = Global routing table.

The EIGRP next-hops belong to VRF MAIN.

In addition EIGRP routes are older then the iBGP route.

Edit:

the MP iBGP route has origin incomplete meaning it is originated from a redistribution into BGP in another PE node.

Hope to help

Giuseppe

mfarrenkopf · ‎07-16-2019

Hi Giuseppe,

I appreciate the quick response. Thank you.

Yes, understood. However, I have other routes from other sources that are installing just fine. For example, 10.129/16 shows in EIGRP as VPNv4 sourced and the BGP route is installed.

B-Core# show ip eigrp top 10.129.0.0/16 vrf MAIN

IP-EIGRP (AS 12): Topology entry for 10.129.0.0/16
State is Passive, Query origin flag is 1, 1 Successor(s), FD is 1
Routing Descriptor Blocks:
172.21.200.25, from VPNv4 Sourced, Send flag is 0x0
Composite metric is (1/0), Route is Internal (VPNv4 Sourced)
Vector metric:
Minimum bandwidth is 0 Kbit
Total delay is 0 microseconds
Reliability is 0/255
Load is 0/255
Minimum MTU is 1500
Hop count is 0
Internal tag is 0

B-Core# show ip route 10.129.0.0/16 vrf MAIN
IP Route Table for VRF "MAIN"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

10.129.0.0/16, ubest/mbest: 1/0
*via 172.21.255.118%default, [200/0], 10w5d, bgp-65001, internal, tag 65001 (mpls-vpn)

So fundamentally, the configuration is right -- or acts right for other routes. One interesting point is that 10.129/16 is redistributed by an IOS device, whereas 10.137/16 is redistributed by an NX-OS device. The 10.129/16 announcement has BGP costpath information, but the 10.137/16 announcement does not.

mfarrenkopf · ‎07-16-2019

Hi Giuseppe,

The problem is fixed. The issue was that 10.137/16 is a static route and did not have a pre-costpath extended community when advertised into BGP. Because of this, B-Core was selecting the EIGRP route it was hearing from its partner.

I added a set extcommunity COST:pre-bestpath:128:1 into the BGP announcement -- to match what I was seeing from IOS devices -- and that fixed the problem.

10-137-PE# show route-map PERMITALL-RM
route-map PERMITALL-RM, permit, sequence 10
Match clauses:
Set clauses:
extcommunity COST:pre-bestpath:128:1

B-Core# show ip route 10.137.0.0/16 vrf MAIN
IP Route Table for VRF "MAIN"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

10.137.0.0/16, ubest/mbest: 1/0
*via 172.21.255.149%default, [200/0], 00:08:44, bgp-65001, internal, tag 65001 (mpls-vpn)

I don't think the documentation is particularly clear on this. But I'm glad I figured it out, because it was causing a multicast loop for us through the MPLS MDT tunnel.

Thank you!!!

mfarrenkopf · ‎07-18-2019

Hi Giuseppe,

So this has been partially fixed, but I'm still seeing some non-deterministic behavior. There are some routes that I missed that don't have the pre-bestpath cost attribute. Sometimes clearing the route from the routing table and letting it re-choose a route works. Sometimes it doesn't. I also added the bestpath cost ignore statement into the BGP section on the 7Ks, which was standard configuration on our IOS devices but not on the 7Ks.

Is there a document that you know of that describes the conditions under which the 7K flags the route as a backup route? What conditions cause "is backup urib route" to appear? I stumbled onto some vague statements involving EIGRP and BGP interactions, but I'd really like a hard document that explains those interactions.

If you know of one, I'd appreciate getting a link to it. Thank you!