iBGP and MPLS question

[BrettCurtis5442]

I'm new to this network.  They're running iBGP, ospf and MPLS as an enterprise on a regional network.  I've done VRFlite with MPBGP before between a CE and PE router, but there's a lot I have to learn about actual MPLS and label switching.  There's a lot I don't understand about how their network is actually working. 

This is a regional network with about 25 sites in a partial mesh.  It seems that almost every router is configured with both mpls LDP and vrfs.  So I guess we would describe those as all PE type routers.  All the main site routers are running iBGP, mpls ip, LDP, and OSPF, vrfs, and OSPF within those vrfs. 

The first thing I don't understand is that none of the iBGP routers seem to have any neighbors:

Router1#sh ip bgp summary
Router1#sh ip bgp

Nothing, no neighbors.  Everything seems to be happening within  vpnv4.  I don't understand it.

Router1#sh ip bgp vpnv4 all
BGP table version is 5050202, local router ID is 10.11.131.248
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 65500:200102
* i 0.0.0.0 10.11.128.238 30 100 0 65502 i
*>i 10.11.128.238 30 100 0 65502 i
* i 10.10.10.6/32 10.11.128.238 0 100 0 i
*>i 10.11.128.238 0 100 0 i
* i 10.11.128.192/28
10.11.128.238 0 100 0 i
*>i 10.11.128.238 0 100 0 i
* i 10.11.215.1/32 10.11.128.238 0 100 0 i
*>i 10.11.128.238 0 100 0 i
Route Distinguisher: 65500:200103
* i 10.10.8.28/30 10.11.130.119 0 100 0 i
*>i 10.11.130.119 0 100 0 i
* i 10.11.128.192/28
10.11.130.119 0 100 0 i
*>i 10.11.130.119 0 100 0 i
* i 10.11.131.52/32
10.11.130.119 0 100 0 i
*>i 10.11.130.119 0 100 0 i
* i 10.11.215.2/32 10.11.130.119 0 100 0 i
*>i 10.11.130.119 0 100 0 i
Route Distinguisher: 65500:200104
* i 10.208.4.160/28 10.10.10.14 0 100 0 i
*>i 10.10.10.14 0 100 0 i

Here is the config of what appears to be one of two core routers.  Can anyone explain how the iBGP neighbor relationships are working?

router bgp 65500
bgp log-neighbor-changes
no bgp default ipv4-unicast
neighbor RRC peer-group
neighbor RRC remote-as 65500
neighbor RRC update-source Loopback1
neighbor RRP peer-group
neighbor RRP remote-as 65500
neighbor RRP update-source Loopback1
neighbor 10.10.10.13 peer-group RRC
neighbor 10.10.10.14 peer-group RRC
neighbor 10.10.10.15 peer-group RRC
neighbor 10.11.2.193 peer-group RRC
neighbor 10.11.2.206 peer-group RRP
neighbor 10.11.128.238 peer-group RRC
neighbor 10.11.130.119 peer-group RRC
neighbor 10.11.131.40 peer-group RRC
neighbor 10.11.131.47 remote-as 65500
neighbor 10.11.131.47 update-source Loopback1
neighbor 10.11.131.252 peer-group RRC
neighbor 10.11.254.39 remote-as 65500
neighbor 10.11.254.39 update-source Loopback1
!
address-family ipv4
exit-address-family
!
address-family vpnv4
neighbor RRC send-community both
neighbor RRC route-reflector-client
neighbor RRP send-community both
neighbor 10.10.10.13 activate
neighbor 10.10.10.14 activate
neighbor 10.10.10.15 activate
neighbor 10.11.2.193 activate
neighbor 10.11.2.206 activate
neighbor 10.11.128.238 activate
neighbor 10.11.130.119 activate
neighbor 10.11.131.40 activate
neighbor 10.11.131.47 activate
neighbor 10.11.131.47 send-community both
neighbor 10.11.131.252 activate
neighbor 10.11.254.39 activate
neighbor 10.11.254.39 send-community both
exit-address-family

address-family ipv4 vrf VRF2001
network 10.12.14.194 mask 255.255.255.255
network 10.12.14.249 mask 255.255.255.255
network 10.12.14.250 mask 255.255.255.254
network 10.12.14.252 mask 255.255.255.255
network 10.10.2.0 mask 255.255.254.0
network 10.10.4.0 mask 255.255.254.0
network 10.11.12.0 mask 255.255.255.0
network 10.11.20.0 mask 255.255.255.0
network 10.11.21.0 mask 255.255.255.0
network 10.11.31.0 mask 255.255.255.0
network 10.11.128.176 mask 255.255.255.252
network 10.11.128.208 mask 255.255.255.240
network 10.11.131.246 mask 255.255.255.255
network 10.11.215.3 mask 255.255.255.255
network 10.11.222.0 mask 255.255.255.0
redistribute static
redistribute ospf 2300 match internal external 1 external 2
exit-address-family

[BrettCurtis5442]

I found part of the answer.  I guess I don't really understand this vpvn4 address family concept and establishing neighbors within vpnv4:

#sh bgp vpnv4 unicast all summary
BGP router identifier 10.11.131.248, local AS number 65500
BGP table version is 5050202, main routing table version 5050202
648 network entries using 165888 bytes of memory
1057 path entries using 126840 bytes of memory
92/80 BGP path/bestpath attribute entries using 24288 bytes of memory
72 BGP rrinfo entries using 2880 bytes of memory
1 BGP AS-PATH entries using 24 bytes of memory
18 BGP extended community entries using 648 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 320568 total bytes of memory
BGP activity 64467/63805 prefixes, 3799970/3798885 paths, scan interval 60 secs

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
10.10.10.13 4 65500 1313987 3609061 5050202 0 0 2y26w 10
10.10.10.14 4 65500 237671 422825 5050202 0 0 21w2d 11
10.10.10.15 4 65500 0 0 1 0 0 14:00:01 Idle
10.11.2.193 4 65500 0 0 1 0 0 13w6d Idle
10.11.2.206 4 65500 1466750 1508526 5050202 0 0 2y26w 98
10.11.128.238 4 65500 537369 1639763 5050202 0 0 48w2d 15
10.11.130.119 4 65500 390842 973118 5050202 0 0 35w1d 19
10.11.131.40 4 65500 237608 422825 5050202 0 0 21w2d 8
10.11.131.47 4 65500 158151 186922 5050202 0 0 13w6d 207
10.11.131.252 4 65500 378736 858601 5050202 0 0 34w0d 5
10.11.254.39 4 65500 158276 185953 5050202 0 0 13w6d 208

[Giuseppe Larosa]

Hello Brett,

your understanding is correct

show ip bgp summary provides an output only if there are neighbors defined within the address-family ipv4 unicast that is the equivalent of standard non multi protocol BGP.

the quick show to check the VPNv4 address family is

show ip bgp vpnv4 all summary

As you have found.

A VPNv4 prefix is different from a standard IPv4 prefix

IPv4 prefix : 32 bit address represented in dotted decimal format for human beings readability

VPNv4 prefix: a 96 bit prefix made by prepending a 64 bit value called Route distinguisher to a 32 bit IP address

you can represent a VPNv4 prefix like RD:IPv4 prefix

the RD can have two main formats BGP-ASN:32 bit value or PE-loop0-IPaddress:16 bit value

(actually two bytes of the 64 bit are not configurable and used to specify the type of RD).

The RD serves an important role in an MP BGP + MPLS backbone: it provides a way to discriminate between overlapping IP subnets belonging to different customers / different VRFs on the same PE node and on all other PE nodes in the network

Example:

65000:100:10.10.10.0/24 is seen different from 65000:535:10.10.10.0/24.

This allows for correct routing and forwarding in different VRFs.

The other key element of VPNv4 prefix is an additional MP BGP attribute that is an extended community called route target. One or more route targets can be associated to a VPNv4 prefix (Rd:IPv4).

Route targets are used by all PE nodes receiving VPNv4 advertisements to decide if they have to import a VPNv4 prefix in each of the locally defined VRFs

For example if prefix 65000:100:10.10.10.0/24 is advertised to the backbone = exported with route target 65000:201 only those PE nodes that have one (or more the one) VRFs with the command

route-target import 65000:201 will import the prefix in the local routing table of the VRF

importing means stripping the RD field and inserting the 10.10.10.0/24 IPv4 prefix in the dedicated routing table of the VRF as a BGP B route internal B [200/0] via next-hop the loopback address of the PE that exported / originated the VPNv4 prefix.

All this is about the control plane of L3 VPN.

The forwarding plane = sending packets within  a VPN between VRF sites uses the MPLS forwarding infrastructure.

LDP protocol creates automatically Label switched paths with destination all the remote PE loopbacks.

Actually the MP BGP VPNv4 prefix in addition to the route target contains a VPN label that represents the prefix in the local MPLS forwarding table of the sender PE node.

The sending of an MPLS L3 VPN packet is performed by inserting the IP packet inside an MPLS frame with two MPLS 4 byte headers:

the external label also called IGP label is actually the LDP built LSP to the PE loopback

the innner label or VPN label is provided by MP BGP in AF VPNv4 using a next-hop = PE loopback.

The core router in your example is acting as route reflector server in address family VPNv4 avoiding the need for full mesh of MP iBGP sessions.

Hope to help

Giuseppe

[BrettCurtis5442]

Thanks for that.  I'm trying to piece together all those concepts that I'm trying to learn all at the same time!  So I guess my question is why aren't they running iBGP at IPv4?  In this case, it's an isolated regional network.  The only external connectivity, that I've discovered so far, is a connection to a global provider within the corporate VRF.  Within that vrf, they have a eBGP neighbor with the provider PE router.  

I suppose they're running OSPF in the global stack.  That appears to have all the WAN links and loopbacks.  I guess they just don't need iBGP within IPv4?  When would you need it within IPv4, and how would that change my configuration?  Would just deleting the no bgp default ipv4-unicast establish those IPv4 neighbor relationships?

[Giuseppe Larosa]

Hello Brett,

if they are not running iBGP for address family ipv4 unicast it is because it is not needed.

OSPF and LDP are running in the global routing table GRT also named vrf default in some platforms.

Actually LDP does only the work to assign labels to all prefixes known from IGP OSPF.

the actually installed MPLS label follows the OSPF best path choice all other alternative labels from non best paths are not used. So LDP builds LSPs over the OSPF best paths on links with MPLS enabled (mpls ip).

Note: it is very important to avoid to have two equal cost paths one with MPLS enabled (mpls ip in interface mode) and one without. This can break MPLS L3 VPN connectivity.

>> When would you need it within IPv4, and how would that change my configuration? Would just deleting the no bgp default ipv4-unicast establish those IPv4 neighbor relationships?

The use of BGP address-family ipv4 unicast is needed in contexts like receiving a full Internet BGP table ( 760,000 prefixes now !!) that cannot be supported by an IGP.

In service provider networks all service related prefixes are advertised in BGP either in AF ipv4 unicast or in other address families for scalability reasons.

To activate the BGP AF ipv4 unicast you don't need to remove the command

no bgp default ipv4-unicast

(actually this command enables MP BGP I would not remove it for any reason)

you just need to create the AF ipv4 unicast and to activate neighbors within

Example

router bgp 65000

address-family ipv4 unicast

neighbor 172.16.20.1 activate

network ...

redistribute ...

!

Note:

under router bgp you define the neighbors their remote AS numbers and other parameters like using update-source loop0.

Under the specific address family you can activate a neighbor that is already defined in the router bgp context.

So in this example the neighbor 172.16.20.1 has to be defined in router bgp mode before you can activate it.

All the activations you perform in each address-family simply specifies what type of prefixes (NLRI) you want to exchange with the specific neighbor. Both neighbors have to agree on BGP session setup of what AFs they want to exchange prefixes (MP NLRI).

Under IPv4 address-family you can use the usual network or redistribute commands to inject routes that are in the GRT global routing table.

Under AF ipv4 vrf <vrf-name> with network and redistribute you can inject routes that are in vrf <vrf-name> routing table.

In AF VPNv4 you just need to activate and to send community extended or both

both = extended AND standard.

No network or redistribute commands are needed here.

Hope to help

Giuseppe

[BrettCurtis5442]

Got it.  This discussion is really filling in some of the blanks.  Thanks for your detailed answers!

[BrettCurtis5442]

I guess this may answer more of my question:

no bgp default ipv4-unicast - from the command reference:

(Optional) Disables the IPv4 unicast address family on all neighbors.

• Use the no bgp default ipv4-unicast command if you are using this neighbor for Multiprotocol Label Switching (MPLS) routes only.

[BrettCurtis5442]

One more question.  Here's another router in another region.  In this case, they activated neighbors under both ipv4 and vpnv4.  The routers in this region are connected to a pair of route reflectors at the service provider edge.  There are no routes in ipv4.  It would seem that in this case, there's no reason to activate neighbors at ipv4.  I wonder why they did?  What could the purpose be?  Could it be, if any reason at all, to just be able to see the neighbors at ipv4?  Not seeing any neighbors has definitely thrown a few folks in my group for a loop.  Interesting that in this case  you see the neighbors at both levels.

router bgp 65000
bgp router-id 10.96.16.3
no bgp default ipv4-unicast
bgp log-neighbor-changes
neighbor 10.96.16.1 remote-as 64622
neighbor 10.96.16.1 update-source Loopback10
neighbor 10.96.16.2 remote-as 64622
neighbor 10.96.16.2 update-source Loopback10
!
address-family ipv4
neighbor 10.96.16.1 activate
neighbor 10.96.16.2 activate
no auto-summary
no synchronization
exit-address-family
!
address-family vpnv4
neighbor 10.96.16.1 activate
neighbor 10.96.16.1 send-community extended
neighbor 10.96.16.2 activate
neighbor 10.96.16.2 send-community extended
exit-address-family

Router1#sh bgp vpnv4 unicast all summary
BGP router identifier 10.96.16.3, local AS number 65000
BGP table version is 2408791, main routing table version 2408791
8427 network entries using 1154499 bytes of memory
16844 path entries using 1145392 bytes of memory
487/439 BGP path/bestpath attribute entries using 77920 bytes of memory
20 BGP rrinfo entries using 480 bytes of memory
334 BGP AS-PATH entries using 16442 bytes of memory
25 BGP extended community entries using 936 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 2395669 total bytes of memory
BGP activity 119702/111275 prefixes, 572128/555284 paths, scan interval 15 secs

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
10.96.16.1 4 64622 4702532 4498880 2408791 0 0 3w5d 8387
10.96.16.2 4 64622 4735649 4503060 2408791 0 0 3w2d 8402

Router1#sh bgp summary
BGP router identifier 10.96.16.3, local AS number 64622
BGP table version is 1, main routing table version 1

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
10.96.16.1 4 64622 4702535 4498883 1 0 0 3w5d 0
10.96.16.2 4 64622 4735652 4503063 1 0 0 3w2d 0

Router1#sh bgp

[Giuseppe Larosa · Accepted solution]

Hello Brett,

we cannot know why they have activated also AF ipv4 unicast.

However, this demonstrates the multiprotocol BGP capability without removing

no bgp default ipv4-unicast

as described in my previous post in this thread you can activate neighbors under each desired  AF including ipv4 unicast.

Hope to help

Giuseppe

[BrettCurtis5442]

Thanks again.  It was useful to see activations in both.  I guess it gets back to the fact that there are just no useful routes to exchange in the bgp GRT for these regional networks.   This no neighbors business within ipv4 has thrown several of my team members for a loop as well.  Perhaps they activated at ipv4 just for clarity.  Like you said, who knows.  

[Harold Ritter · Accepted solution]

By the way Brett, you should consider using the new BGP syntax (i.e. show bgp <address-family> <sub address-family>). The "show ip bgp" is the IOS legacy format and sometimes lead to confusion.

Regards,

[BrettCurtis5442]

thank you, I was confused about that too.