Hi Peter,

hiersd · ‎12-26-2015

Hi,

I'm trying to find better documentation that describes the differences between VDCs in IOS and NX-OS. So far, this is the best I can find:

http://docwiki.cisco.com/wiki/Cisco_NX-OS/IOS_Layer-3_Virtualization_Comparison

The key question is the need for a route distinguisher for a VRF in BGP.

In NX-OS, BGP does not demand an RD for the VRF. My tests agree with the wiki doc for NX-OS.

In IOS (Well, IOSv running in virl, and IOS in GNS3), BGP *does* demand an RD for the VRF. For example, if I create this VRF:

vrf definition CUSTOMER_1
address-family ipv4
 exit-address-family

And then attempt to use the VRF in BGP per the wiki doc, I get the error shown below:

router bgp 8 
 address-family ipv4 vrf CUSTOMER_1
% VRF CUSTOMER_1 does not have an RD configured.

I'm not sure yet if virl/GNS3 makes a difference, but at least they agree with each other and fail with the same error message.

Can you successfully configure the IOS config shown on the wiki doc?

Are you aware of any documentation that explains the observed difference in behavior? Perhaps the "vrf context" in NX-OS does a bit more in the way of virtualization that "vrf definition" in IOS.

Thanks,

David

Peter Paluch · ‎12-26-2015

Hi David,

I have not used the NX-OS personally so I cannot speak from a first hand experience but there is, I believe, a principial explanation that could provide some insight to your case.

The RD is truly required for BGP, and only for BGP. Basically, the RD is a clever workaround for a specific BGP trait: If a BGP speaker receives an advertisement about the same network/netmask combination twice, it will consider the second advertisement to be a replacement (an update) of the previous advertisement. With VRFs that can potentially hold identical networks, this behavior would be disastrous. As you surely know, the sorting of received networks into VRFs is done using the RT (route target) value, which is technically a BGP extended community attribute. Assume that a router has two VRFs, one configured with RT=1:1, the second configured with RT=1:2, and both these VRFs hold the same network 10.0.0.0/24. If these two VRFs were advertised to a second BGP speaker and no RD was used, the other BGP speaker would see two updates about the 10.0.0.0/24 network coming in, one having RT=1:1, the other having RT=1:2. Whichever comes later will take over - and the result will be that on the second BGP speaker, the network 10.0.0.0/24 will be placed only in one of the two VRFs, depending on what RT value was attached to the update that came in later.

The RD helps here to "inflate" the addresses of all networks in a particular VRF with a locally unique value for the entire VRF so that network 10.0.0.0/24 from VRF X does not appear as an update of the same network from VRF Y when these networks are advertised in BGP.

I recall that some older IOSes (12.x) actually allowed you to omit the configuration of an RD value in a VRF and yet run BGP over these VRFs. What happened was that all VRFs with an unconfigured RD used an implicit RD value of 0:0, and it was actually possible with this behavior to show the possible problem with BGP getting confused as explained above. Newer IOS versions do not allow running BGP with VRFs that do not have their RD configured.

In IOS-XR, the configuration of the RD has actually been moved out from the definition of a VRF and moved to BGP process configuration. In addition, IOS-XR allows the RD to be assigned automatically to a VRF (instead of using rd value, you would use the rd auto command).

I would assume that the NX-OS behaves according to one of these possibilities - either it uses some implicit and identical value for all VRFs with unconfigured RD (say, 0:0), or it generates and assigns RDs automatically. There should actually be a command allowing you to check the VRF, something along the lines of show vrf, that should (in a verbose mode) show you the default RD for this VRF. By observing this value for various VRFs without an explicit RD, you should be able to deduce quickly what is the NX-OS doing.

In any case, BGP cannot operate without an RD. If there is no RD configured for a VRF, then it's either an implicit constant, or an automatically generated unique value. I do not personally see any other option.

My two cents...

Best regards,
Peter

hiersd · ‎12-26-2015

Hi Peter,

Thanks for the reply.

I agree that RDs are required to handle overlapping addresses in a single BGP process/table. If you spin up a new BGP process, table or some other construct for each VRF, however, the overlapping address problem goes away.

It is possible the the NX-OS does something clever under the hood.

There is no RD that I can find; "show vrf detail" reports "VRF RD: Not configured" on the 7k, and we have a pile of VRFs working just fine.

Thanks,

David

Masoud Pourshabanian · ‎12-26-2015

Hello,

There must be a way that BGP can differentiate routes of different VRF inside the BGP table.

Try to check BGP table

show ip bgp vpnv4 all

Check the output of this command also

show ip vrf detail [vrf name]

Peter Paluch · ‎12-27-2015

Hi David,

If you spin up a new BGP process, table or some other construct for each VRF, however, the overlapping address problem goes away.

This is actually not possible, at least not on IOS-based devices. You can start only a single BGP process, and only additionally, you can tell it to operate within selected VRFs (by the address-family { ipv4 | ipv6 } vrf vrf-name) as well as above all VRFs at once (by the address-family { vpnv4 | vpnv6 } ). The problem of overlapping addresses with either VPNv4 or VPNv6 address families therefore principially cannot go away without RD, as you cannot have multiple BGP processes on a device.

There is no RD that I can find; "show vrf detail" reports "VRF RD: Not configured" on the 7k, and we have a pile of VRFs working just fine.

If you are using VRFs locally and do not synchronize their contents across multiple devices using BGP and VPNv4/VPNv6 address families then you truly do not need any RD and everything will be working okay. In other words, if you are running the VRFs in a VRF-lite scenario then you can have them all without an RD configured and all will still be fine. The RD is required for the VPNv4/VPNv6 address family in BGP to operate properly, so the question is: Are you running BGP and VPNv4/VPNv6 address families?

Best regards,
Peter

hiersd · ‎12-27-2015

I'm just trying to enter the IOS commands on this page:

http://docwiki.cisco.com/wiki/Cisco_NX-OS/IOS_Layer-3_Virtualization_Comparison

The NX-OS works fine, but IOS does not let you create the address-family in BGP without configuring an RD in the VRF. Well, it doesn't let me do it, anyway. Maybe you can give it a whirl if you get a chance?

Sure, you can't type "router bgp ASN" more than once, but I continue to suspect that Cisco is doing something clever with additional constructs under the hood like creating separate BGP tables or processes for each VRF. Its not like they'd have to tell a couple of schmoes like us if they did, I suppose.

Fig 14-1 comes close, but still leaves a bit of wiggle room:

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/sw/5_x/nx-os/unicast/configuration/guide/l3_cli_nxos/l3_virtual.html

David

VRF/BGP differences between IOS and NX-OS: is RD needed for BGP?