Sham-link and default vpnv4 route

zoran.erdec · ‎05-03-2024

Hello everyone,

I have a problem with sham link and default route and any help is appreciated. Let's assume that the topology is as follows:

The initial configuration is as follows:
CE1 has a static default route that it advertises into ospf with default-information originate metric-type 1
PE1 and PE2 are route reflectors and they advertise the default route in BGP with network 0.0.0.0
PE3 has sham links to PE 1 and PE 2.
PE1, PE2 and PE3 advertise a default route with default-information originate in OSPF.
In this initial configuration, traffic routing takes place as expected, i.e. on PE 3 traffic for the default route is routed via the sham link because the next-hop BGP IP address is from PE 1.
RIB-failure is visible in the BGP table for the default route.

#show ip route vrf CUSTOMER 0.0.0.0
Routing Table: CUSTOMER
Routing entry for 0.0.0.0/0, supernet
   Known via "ospf 3009", distance 110, metric 301, candidate default path
   Tag 3009, type external 1
   Redistributing via bgp 65001
   Last update from 10.199.253.2 00:17:17 ago
   Routing Descriptor Blocks:
   * 10.199.253.2 (default), from 10.190.30.1, 00:17:17 ago
       Route metric is 301, traffic share count is 1
       Route tag 3009
       MPLS label: 27859
       MPLS Flags: MPLS Required

#show ip cef vrf CUSTOMER 8.8.8.8 detail
0.0.0.0/0, epoch 3, flags rib defined all labels, default route
   NetFlow: Origin AS 0, Peer AS 0, Mask Bits 0
   recursive via 10.199.253.2 label 27859
     nexthop 10.193.100.9 Vlan2303

#show ip bgp vpnv4 vrf CUSTOMER 0.0.0.0
BGP routing table entry for 24250:3009:0.0.0.0/0, version 7893351
Paths: (5 available, best #5, table CUSTOMER, RIB-failure(17) - next-hop mismatch)
   Not advertised to any peer
   Refresh Epoch 1
   Local, imported path from 25110:3012:0.0.0.0/0 (global)
     10.199.253.14 (metric 21) (via default) from 10.0.255.1 (10.1.255.2)
       Origin IGP, metric 401, localpref 100, valid, internal
       Extended Community: RT:24250:3009 RT:25110:3012
         OSPF DOMAIN ID:0x0005:0x000000003009 OSPF RT:0.0.0.0:5:0
         OSPF ROUTER ID:10.223.10.1:0
       Originator: 10.199.253.14, Cluster list: 10.1.255.2
       mpls labels in/out nolabel/1135
       rx pathid: 0, tx pathid: 0
   Refresh Epoch 1
   Local
     10.199.253.2 (metric 11) (via default) from 10.0.255.1 (10.1.255.2)
       Origin IGP, metric 300, localpref 100, valid, internal
       Extended Community: RT:24250:3009
       Originator: 10.193.255.3, Cluster list: 10.1.255.2
       mpls labels in/out nolabel/27859
       rx pathid: 0, tx pathid: 0
   Refresh Epoch 1
   Local
     10.199.253.2 (metric 11) (via default) from 10.0.255.2 (10.1.255.103)
       Origin IGP, metric 300, localpref 100, valid, internal
       Extended Community: RT:24250:3009
       Originator: 10.193.255.3, Cluster list: 10.1.255.103
       mpls labels in/out nolabel/27859
       rx pathid: 0, tx pathid: 0
   Refresh Epoch 1
   Local
     10.199.253.1 (metric 14) (via default) from 10.199.253.1 (10.193.255.2)
       Origin IGP, metric 300, localpref 100, valid, internal
       Extended Community: RT:24250:3009
       mpls labels in/out nolabel/24003
       rx pathid: 0, tx pathid: 0
   Refresh Epoch 1
   Local
     10.199.253.2 (metric 11) (via default) from 10.199.253.2 (10.193.255.3)
       Origin IGP, metric 300, localpref 100, valid, internal, best
       Extended Community: RT:24250:3009
       mpls labels in/out nolabel/27859
       rx pathid: 0, tx pathid: 0x0

Now on PE 3 I announce the default route in BGP with the command network 0.0.0.0 and everything is still fine (although there is no more RIB-failure in the BGP table).

#show ip bgp vpnv4 vrf CUSTOMER 0.0.0.0
BGP routing table entry for 24250:3009:0.0.0.0/0, version 7893357
Paths: (6 available, best #1, table CUSTOMER)
  Advertised to update-groups:
     1         
  Refresh Epoch 1
  Local
    10.199.253.2 (metric 11) (via default) from 0.0.0.0 (10.193.253.8)
      Origin IGP, metric 301, localpref 100, weight 32768, valid, sourced, local, best
      Extended Community: RT:24250:3009 OSPF DOMAIN ID:0x0005:0x000000003009 
        OSPF RT:0.0.0.0:5:0 OSPF ROUTER ID:10.250.9.65:0
      mpls labels in/out 279/nolabel
      rx pathid: 0, tx pathid: 0x0
  Refresh Epoch 1
  Local, imported path from 25110:3012:0.0.0.0/0 (global)
    10.199.253.14 (metric 21) (via default) from 10.0.255.1 (10.1.255.2)
      Origin IGP, metric 401, localpref 100, valid, internal
      Extended Community: RT:24250:3009 RT:25110:3012 
        OSPF DOMAIN ID:0x0005:0x000000003009 OSPF RT:0.0.0.0:5:0 
        OSPF ROUTER ID:10.223.10.1:0
      Originator: 10.199.253.14, Cluster list: 10.1.255.2
      mpls labels in/out 279/1135
      rx pathid: 0, tx pathid: 0
  Refresh Epoch 1
  Local
    10.199.253.2 (metric 11) (via default) from 10.0.255.1 (10.1.255.2)
      Origin IGP, metric 300, localpref 100, valid, internal
      Extended Community: RT:24250:3009
      Originator: 10.193.255.3, Cluster list: 10.1.255.2
      mpls labels in/out 279/27859
      rx pathid: 0, tx pathid: 0
  Refresh Epoch 1
  Local
    10.199.253.2 (metric 11) (via default) from 10.0.255.2 (10.1.255.103)
      Origin IGP, metric 300, localpref 100, valid, internal
      Extended Community: RT:24250:3009
      Originator: 10.193.255.3, Cluster list: 10.1.255.103
      mpls labels in/out 279/27859
      rx pathid: 0, tx pathid: 0
  Refresh Epoch 1
  Local
    10.199.253.1 (metric 14) (via default) from 10.199.253.1 (10.193.255.2)
      Origin IGP, metric 300, localpref 100, valid, internal
      Extended Community: RT:24250:3009
      mpls labels in/out 279/24003
      rx pathid: 0, tx pathid: 0
  Refresh Epoch 1
  Local
    10.199.253.2 (metric 11) (via default) from 10.199.253.2 (10.193.255.3)
      Origin IGP, metric 300, localpref 100, valid, internal
      Extended Community: RT:24250:3009
      mpls labels in/out 279/27859
      rx pathid: 0, tx pathid: 0

The problem occurs when I do anything related to sham links on PE3, for example I delete one of the sham links or change the cost.
In this case I will delete the sham link to PE2.
PE3 now has no label:

#sh ip cef vrf CUSTOMER 8.8.8.8 de
0.0.0.0/0, epoch 3, flags default route
   NetFlow: Origin AS 0, Peer AS 0, Mask Bits 0
   local label info: other/279
   recursive via 10.199.253.2 unusable: no label

#show ip route vrf CUSTOMER 0.0.0.0

Routing Table: CUSTOMER
Routing entry for 0.0.0.0/0, supernet
   Known via "ospf 3009", distance 110, metric 301, candidate default path
   Tag 3009, type external 1
   Redistributing via bgp 65001
   Advertised by bgp 65001 match internal external 1 & 2
   Last update from 10.199.253.2 00:04:22 ago
   Routing Descriptor Blocks:
   * 10.199.253.2 (default), from 10.190.30.1, 00:04:22 ago
       Route metric is 301, traffic share count is 1
       Route tag 3009
       MPLS Flags: MPLS Required

#show mpls forwarding-table 10.199.253.2
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
193 Pop Label 10.199.253.2/32 165576195772 Vl2303 10.193.100.9

#show ip bgp vpnv4 vrf SDV_OS 0.0.0.0
BGP routing table entry for 24250:3009:0.0.0.0/0, version 7893357
Paths: (6 available, best #1, table SDV_OS)
   Advertised to update-groups:
      1
   Refresh Epoch 1
   Local
     10.199.253.2 (metric 11) (via default) from 0.0.0.0 (10.193.253.8)
       Origin IGP, metric 301, localpref 100, weight 32768, valid, sourced, local, best
       Extended Community: RT:24250:3009 OSPF DOMAIN ID:0x0005:0x000000003009
         OSPF RT:0.0.0.0:5:0 OSPF ROUTER ID:10.250.9.65:0
       mpls labels in/out 279/nolabel
       rx pathid: 0, tx pathid: 0x0
   Refresh Epoch 1
   Local, imported path from 25110:3012:0.0.0.0/0 (global)
     10.199.253.14 (metric 21) (via default) from 10.0.255.1 (10.1.255.2)
       Origin IGP, metric 401, localpref 100, valid, internal
       Extended Community: RT:24250:3009 RT:25110:3012
         OSPF DOMAIN ID:0x0005:0x000000003009 OSPF RT:0.0.0.0:5:0
         OSPF ROUTER ID:10.223.10.1:0
       Originator: 10.199.253.14, Cluster list: 10.1.255.2
       mpls labels in/out 279/1135
       rx pathid: 0, tx pathid: 0
   Refresh Epoch 1
   Local
     10.199.253.2 (metric 11) (via default) from 10.0.255.1 (10.1.255.2)
       Origin IGP, metric 300, localpref 100, valid, internal
       Extended Community: RT:24250:3009
       Originator: 10.193.255.3, Cluster list: 10.1.255.2
       mpls labels in/out 279/27859
       rx pathid: 0, tx pathid: 0
   Refresh Epoch 1
   Local
     10.199.253.2 (metric 11) (via default) from 10.0.255.2 (10.1.255.103)
       Origin IGP, metric 300, localpref 100, valid, internal
       Extended Community: RT:24250:3009
       Originator: 10.193.255.3, Cluster list: 10.1.255.103
       mpls labels in/out 279/27859
       rx pathid: 0, tx pathid: 0
   Refresh Epoch 1
   Local
     10.199.253.1 (metric 14) (via default) from 10.199.253.1 (10.193.255.2)
       Origin IGP, metric 300, localpref 100, valid, internal
       Extended Community: RT:24250:3009
       mpls labels in/out 279/24003
       rx pathid: 0, tx pathid: 0
   Refresh Epoch 1
   Local
     10.199.253.2 (metric 11) (via default) from 10.199.253.2 (10.193.255.3)
       Origin IGP, metric 300, localpref 100, valid, internal
       Extended Community: RT:24250:3009
       mpls labels in/out 279/27859
       rx pathid: 0, tx pathid: 0

Also, as I deleted sham links several times, once the next-hop for the default route was the IP address of the sham link, instead of the BGP address of PE1 or PE2 (I can't influence when this case will happen).
Thank you in advance for any help.

MHM Cisco World · ‎05-03-2024

CE advertise 0.0.0.0 as E1 and that make sham link not need.

Link solve backdoor if the lsa is 1'2 or 3 not lsa 5

MHM

zoran.erdec · ‎05-06-2024

Thank You for your reply.
Sham-link was implemented because CE devices have specific networks and I want to use MPLS links.
Can You explain why the sham-link would not also be used for the default E1 route?
I see it in the following way:
Let's assume that the cost for each link = 1
The pure OSPF path from CE1 to PE3 is, for example, the next-hop of static route - CE1 - PE2 - CE3 - CE2 - PE3 => 1 + 1 + 1 + 1 +1 = cost 5
OSPF path over the SHAM link from CE1 to PE3 are, for example, next-hop static route - C1 - PE2 - PE3 (via sham-link) => 1 + 1 + 1 = cost 3

That is why the default route via sham-link is better and I want to use a better link.

Related to the original question where PE 3 does not have a label for the default route.
Let's say the order of configuration on PE3 was as follows:
1. Defining sham-links
2. Default-information originates in OSPF
3. Network 0.0.0.0 in BGP
Then everything is fine, but if I change anything on the Sham-link after that, the PE3 can no longer use the default route (as I described in the initial post).

I can solve the problem by writing no network 0.0.0.0 and then network 0.0.0.0 on PE3 in the BGP process and then PE3 again has labels for the default route.
Of course, if I do something with sham-links again, PE3 won't have labels.

From the above, I conclude that this problem is affected by the order in which PE3 decides how to forward the packet, but I cannot determine what it is.

MHM Cisco World · ‎05-06-2024

Let me review issue and I will update you tomorrow