01-10-2021 07:46 AM
Hey folks,
Background
I am running a home lab and practicing on InterAS Option C with IOSXR, the OSes here are IOSXRv and IOSXR9kv, naming is used accordingly in the post.
Topology
Description
We are in AS 400. iosxrv-2 is the ASBR and BGP ipv4 labelled-unicast RR (it uses next-hop-self for the ipv4 labelled unicast af). iosxr-9k-2 is one of the internal peers in AS 400 peers, it receives remote loopbacks addresses from iosxrv-2 and runs a eBGP VPNv4 peering with a remote PE 6.6.6.6 in AS 100 (Inter AS Option C).
All goes well before establishing the eBGP VPNv4 peering, iosxr-9k-2 can trace to the remote PE fully labelled (traceroute 6.6.6.6 so 102.102.102.102)
RP/0/RP0/CPU0:iosxr-9k-2#trace 6.6.6.6 so 102.102.102.102 Sun Jan 10 15:35:05.516 UTC Type escape sequence to abort. Tracing the route to 6.6.6.6 1 189.148.139.1 [MPLS: Label 16008 Exp 0] 5 msec 3 msec 3 msec < -- label 2 152.218.199.2 [MPLS: Label 16003 Exp 0] 3 msec 3 msec 3 msec 3 177.191.130.1 [MPLS: Label 16000 Exp 0] 3 msec 3 msec 3 msec 4 139.208.128.1 3 msec * 3 msec
The problem starts when eBGP VPNv4 peering is configured, in fact probably due to the following:
the local PE stops binding a label for the remote PE address which makes the traffic to the remote PE unlabeled in the first hop.
This represents a problem given that L3 VPN packets need underlying transport label and that just isn't there.
RP/0/RP0/CPU0:iosxr-9k-2#trace 6.6.6.6 so 102.102.102.102 Sun Jan 10 15:35:55.780 UTC Type escape sequence to abort. Tracing the route to 6.6.6.6 1 189.148.139.1 3 msec 1 msec 2 msec < -- no label 2 152.218.199.2 [MPLS: Label 16003 Exp 0] 4 msec 3 msec 3 msec 3 177.191.130.1 [MPLS: Label 16000 Exp 0] 3 msec 3 msec 3 msec 4 139.208.128.1 3 msec * 4 msec
Configurations
iosxr-9k-2
router bgp 400 address-family ipv4 unicast allocate-label all ! address-family vpnv4 unicast ! address-family ipv6 unicast ! address-family vpnv6 unicast ! neighbor 2.2.2.2 < -- iosxrv-2 (ASBR RR) remote-as 400 update-source Loopback0 address-family ipv4 labeled-unicast ! neighbor 6.6.6.6 < -- remote PE remote-as 100 ebgp-multihop 255 update-source Loopback0 address-family vpnv4 unicast route-policy PASS in route-policy PASS out ! address-family vpnv6 unicast route-policy PASS in route-policy PASS out
iosxrv2
router bgp 400 address-family ipv4 unicast network 102.102.102.102/32 < -- iosxr-9k-2 (local PE) loopback address allocate-label all ! address-family vpnv4 unicast ! address-family vpnv6 unicast ! neighbor-group ibgp remote-as 400 update-source Loopback0 address-family ipv4 labeled-unicast route-reflector-client next-hop-self ! neighbor 152.218.199.2 < -- remote AS remote-as 100 address-family ipv4 labeled-unicast route-policy PASS in route-policy PASS out ! ! neighbor 102.102.102.102 < -- iosxr-9k-2 (local PE) use neighbor-group ibgp
Outputs
RP/0/RP0/CPU0:iosxr-9k-2#sh mpls forwarding prefix 6.6.6.6/32 Sun Jan 10 15:17:03.736 UTC Local Outgoing Prefix Outgoing Next Hop Bytes Label Label or ID Interface Switched ------ ----------- ------------------ ------------ --------------- ------------ 24003 Pop 6.6.6.6/32 Gi0/0/0/0.522 189.148.139.1 576
Output after removing vpnv4 neighbor
RP/0/RP0/CPU0:iosxr-9k-2#conf t Sun Jan 10 15:17:15.729 UTC no RP/0/RP0/CPU0:iosxr-9k-2(config)#router bgp 400 RP/0/RP0/CPU0:iosxr-9k-2(config-bgp)#no nei 6.6.6.6 RP/0/RP0/CPU0:iosxr-9k-2(config-bgp)#commit RP/0/RP0/CPU0:iosxr-9k-2(config-bgp)#end RP/0/RP0/CPU0:iosxr-9k-2#sh mpls forwarding prefix 6.6.6.6/32 Sun Jan 10 15:17:27.356 UTC Local Outgoing Prefix Outgoing Next Hop Bytes Label Label or ID Interface Switched ------ ----------- ------------------ ------------ --------------- ------------ 24003 16008 6.6.6.6/32 2.2.2.2 0 RP/0/RP0/CPU0:iosxr-9k-2#
I know between directly connected eBGP vpnv4/ipv4-labelled peers a static route to the neighbor would generate a /32 entry in rib and that would do the trick.
Question
How about in this case? The eBGP VPNv4 peering is remote and I am not sure how to ensure the traffic would travel labelled all the way.
It is the same thing on the other AS. Also the behavior is flaky, reconfiguring multiple times makes the next hop stay as 2.2.2.2 and the label stick. Not sure how to fix this, please advise.
Thanks, L.
Solved! Go to Solution.
01-11-2021 01:19 PM - edited 01-11-2021 03:19 PM
Hi Loris,
> I am struggling to visualize why your solution of moving network advertisement on the PE works
Correct. Both originating from the PE or the ASBR should be possible. It is just that I normally prefer to originate from the related device itself.
I think the issue you are seing is related to a bug. Do you see messages like the one below on the device, where you see the pop label.
RP/0/0/CPU0:Jan 11 20:34:04.890 : bgp[1052]: %ROUTING-BGP-4-LABEL_COLLISION : Label 24001 collision: prev: [T: 13 RD:0:0:0 PFX/NHID:0.0.0.0] curr: [T: 3 RD:0:0:0 PFX/NHID:192.168.100.4/32]
> Whilst in this case I see why "next-hop-unchanged" is useful when terminating VPNv4 peerings on RRs (to keep RRs out of the
> forwarding plane), I am not sure I understand why it is triggering such a behavior in this case.
The next-hop-unchanged is definitely required, but the behavior you are seeing is probably due to the same bug mentioned above.
Regards,
01-10-2021 09:28 AM - edited 01-11-2021 07:10 PM
...
01-10-2021 12:17 PM
Hi,
It is a good idea to have an RR in each domain for scaling purposes, but there is no strict requirement. The ASBR in this scenario is not an RR, as it propagates BGP-LU prefixes from eBGP to iBGP and vice versa. There is no iBGP to iBGP propagation.
Regards,
01-10-2021 12:10 PM
Hi Loris,
> How about in this case? The eBGP VPNv4 peering is remote and I am not sure how to ensure the traffic would travel labelled all the way.
Communication between the two PEs will use the BGP-LU LSP.
As far as the issue is concerned, try originating the BGP-LU advertisement from the PE itself, rather than from the ASBR.
remove the network statement on iosxrv2 102.102.102.102/32 and add it to iosxr-9k-2 instead.
Also if you have an SR domain in this scenario, make sure you set the prefix-sid on the when you originate the prefix in BGP on PE.
router bgp xxx
address-family ipv4 unicast
network x.x.x.x/32 route-policy setSID
!
route-policy setSID
set label-index <sr prefix-sid index>
end-policy
Regards,
01-11-2021 12:18 AM - edited 01-11-2021 12:36 AM
@Harold Ritter, thanks!
In this case I don't have SR but just BGP-LU for loopbacks exchange with the remote AS and for which iosxrv-2 is the RR and LDP for internal loopbacks inside the AS (given i am running BGP-LU between loopbacks and not using SR).
To your solution, leaving everything as is and moving network advertisement on the PE works:
RP/0/RP0/CPU0:iosxr-9k-2#sh run router bgp Mon Jan 11 07:55:11.627 UTC router bgp 400 address-family ipv4 unicast network 102.102.102.102/32 <-- network statement removed from ASBR on to the PE allocate-label all RP/0/RP0/CPU0:iosxr-9k-2#trace 16.16.16.16 so 102.102.102.102 Mon Jan 11 07:55:56.342 UTC Type escape sequence to abort. Tracing the route to 16.16.16.16 1 189.148.139.1 [MPLS: Label 16012 Exp 0] 5 msec 4 msec 4 msec 2 152.218.199.2 [MPLS: Label 16005 Exp 0] 6 msec 3 msec 3 msec 3 150.177.191.1 [MPLS: Label 16008 Exp 0] 3 msec 4 msec 3 msec 4 180.128.172.2 4 msec * 5 msec
For the sake of learning, I have tried migrating the AS400 to BGP-LU only the edge with external prefixes redistributed into ISIS and mapped using LDP. I have tried terminating the eBGP VPNv4 peering on both ASBR or internal PE and both work until I remove the "next-hop-unchanged" keyword. It looks like the label for the remote PE (in this case the remote PE and RR in AS100 is 16.16.16.16) gets popped when "next-hop-unchanged" is not used.
In the following example I have iosxrv-2 as ASRB + RR, advertising internal loopbacks to external ASes and redistributing external loopbacks into ISIS (LDP maps labels to external prefixes). The eBGP VPNv4 peering between local and remote PE is terminated on iosxr-9k-2 and tried to remove "next-hop-unchanged"
RP/0/RP0/CPU0:iosxr-9k-2(config)#router bgp 400 RP/0/RP0/CPU0:iosxr-9k-2(config-bgp)#nei 16.16.16.16 RP/0/RP0/CPU0:iosxr-9k-2(config-bgp-nbr)#address-family vpnv4 unicast RP/0/RP0/CPU0:iosxr-9k-2(config-bgp-nbr-af)#no next-hop-unchanged RP/0/RP0/CPU0:iosxr-9k-2(config-bgp-nbr-af)#address-family vpnv6 unicast RP/0/RP0/CPU0:iosxr-9k-2(config-bgp-nbr-af)#no next-hop-unchanged RP/0/RP0/CPU0:iosxr-9k-2(config-bgp-nbr-af)#commit RP/0/RP0/CPU0:iosxr-9k-2(config-bgp-nbr-af)#end RP/0/RP0/CPU0:iosxr-9k-2# RP/0/RP0/CPU0:iosxr-9k-2#sh mpls forwarding prefix 16.16.16.16/32 Mon Jan 11 06:36:37.460 UTC Local Outgoing Prefix Outgoing Next Hop Bytes Label Label or ID Interface Switched ------ ----------- ------------------ ------------ --------------- ------------ 24016 Pop 16.16.16.16/32 Gi0/0/0/0.522 189.148.139.1 281 <-- label pop
Adding back "next-hop-unchanged"
RP/0/RP0/CPU0:iosxr-9k-2(config)#router bgp 400 RP/0/RP0/CPU0:iosxr-9k-2(config-bgp)#nei 16.16.16.16 RP/0/RP0/CPU0:iosxr-9k-2(config-bgp-nbr)#address-family vpnv4 unicast RP/0/RP0/CPU0:iosxr-9k-2(config-bgp-nbr-af)#next-hop-unchanged RP/0/RP0/CPU0:iosxr-9k-2(config-bgp-nbr-af)#address-family vpnv6 unicast RP/0/RP0/CPU0:iosxr-9k-2(config-bgp-nbr-af)#next-hop-unchanged RP/0/RP0/CPU0:iosxr-9k-2(config-bgp-nbr-af)#commit RP/0/RP0/CPU0:iosxr-9k-2(config-bgp-nbr-af)#end RP/0/RP0/CPU0:iosxr-9k-2# RP/0/RP0/CPU0:iosxr-9k-2#sh mpls forwarding prefix 16.16.16.16/32 Mon Jan 11 06:40:28.529 UTC Local Outgoing Prefix Outgoing Next Hop Bytes Label Label or ID Interface Switched ------ ----------- ------------------ ------------ --------------- ------------ 24016 16016 16.16.16.16/32 Gi0/0/0/0.522 189.148.139.1 0 <-- label is back
Questions
BGG-LU on ASRB and PE + LDP for internal loopbacks only
BGP-LU on ASBR only + ISIS redistribution and LDP for external prefixes
Thanks in advance
L.
01-11-2021 01:19 PM - edited 01-11-2021 03:19 PM
Hi Loris,
> I am struggling to visualize why your solution of moving network advertisement on the PE works
Correct. Both originating from the PE or the ASBR should be possible. It is just that I normally prefer to originate from the related device itself.
I think the issue you are seing is related to a bug. Do you see messages like the one below on the device, where you see the pop label.
RP/0/0/CPU0:Jan 11 20:34:04.890 : bgp[1052]: %ROUTING-BGP-4-LABEL_COLLISION : Label 24001 collision: prev: [T: 13 RD:0:0:0 PFX/NHID:0.0.0.0] curr: [T: 3 RD:0:0:0 PFX/NHID:192.168.100.4/32]
> Whilst in this case I see why "next-hop-unchanged" is useful when terminating VPNv4 peerings on RRs (to keep RRs out of the
> forwarding plane), I am not sure I understand why it is triggering such a behavior in this case.
The next-hop-unchanged is definitely required, but the behavior you are seeing is probably due to the same bug mentioned above.
Regards,
01-12-2021 09:39 AM
Yess and yess! i saw those logs whenever I changed config specifically when removing next-hop-unchanged and/or removing ldp/bgp-lu support for either the PE or ASBR.
I am glad this might be related to a bug
Thanks very much for all your help here.
Cheers, L.
01-12-2021 10:45 AM
Hi Loris,
As you are probably running the latest XRv (6.1.2), your best bet at this point would be to start running XRv9k, as you mentioned in another post. This will allow you to run newer images, which will solve this specific bug and let you experiment with the latest features.
Regards,
01-12-2021 11:07 AM
01-10-2021 12:38 PM - edited 01-11-2021 07:09 PM
...
01-11-2021 09:57 AM - edited 01-11-2021 03:00 PM
...
01-11-2021 07:16 PM - edited 01-12-2021 09:38 AM
....
01-12-2021 09:36 AM
Hey thanks for posting here.
I have accepted @Harold Ritter's post as the solution given it was spot on.
Thanks for helping and to the next one.
Cheers, L.
01-12-2021 08:59 AM - edited 01-12-2021 09:38 AM
....
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide