multiple VRF across BGP

[cgarringer]

I have a situation where I am using a vrf to isolate subnets from the default system.    It has been working well internally and over MAN connections using eigrp.   Now I have to do this over the MPLS system we have.     We are the end-user of the MPLS system, and the provider distributes our routes using BGP.    I have the vrf defined remotely and have seen the Cisco docs on route-targets.    I have set this up (I think) but I do not get any routing info across.    Are route-targets and BGP address families the way to go on this, or is there a better route?     I need to keep the 2 groups isolated.   Attached is a file with the routing setup.

Thanks for all input

[Peter Paluch]

Hello,

If I understand you correctly, you are advertising networks from your datacenter towards the remote site, but the remote site is not getting the routes. Is that correct?

One issue I see is that both your locations, the datacenter and the remote, are using the same AS number. I am not sure if your provider is replacing the private ASN you are using when advertising the networks from the datacenter to the remote site and vice versa. If he does not then your locations mutually reject their routes, as they consider them to be looped (they find their own ASN in the update coming from across the VPN).

You may want to add these commands to allow the incoming updates to have the same ASN as your own BGP:

Datacenter:

router bgp 64602
 address-family vpnv4
  neighbor 198.18.37.213 allowas-in

Remote:

router bpg 64602
 address-family vpnv4
  neighbor 198.18.129.5 allowas-in

You may need to issue the clear ip bgp * soft command on both routers to readvertise the networks.

In addition, I do not see the address-family ipv4 vrf ADP stanza in the BGP configuration on the Remote site, but I assume that it has just been edited out.

If this modification to your configuration does not help then please include the output of the show ip bgp vpnv4 all neighbor X.X.X.X advertised-routes and show ip bgp vpnv4 all neighbor X.X.X.X received-routes from both your routers where X.X.X.X is the IP address of the defined BGP neighbor.

Best regards,

Peter

[Marwan ALshawi]

Just to add to Peter's post

if your issue as Peter found which is the receiving BGP router seeing its own ASN in the AS path

the above command will fix However be careful if you have a back door link between your Data centers this command might lead to routing loop and better to ask the ISP to over ride your ASN with their own ASN

hope this help

[Peter Paluch]

Marwan,

Absolutely agree. The provider should ideally be using the as-override neighbor option, or use different ASNs for different locations of this single customer.

Best regards,

Peter

[cgarringer]

I added the suggested commands and did not get anything.      I ran the show commands and the output is shown below.    The remote site had 0 prefixes.    The data center was most interesting, it said the either the neighbor or address family did not exist.   I ran sh ip bgp neighbors and the neighbor and family was there, I included the output in the attached file.

[Peter Paluch]

Hello,

I believe I understand what is happening.

Your routers have a BGP peering with the PE routers of your provider. While you are running plain BGP towards your provider, the PE router is already running BGP towards you in a VRF created on the provider's PE router. In other words, even though you believe you are speaking with the global routing table on your ISP's router, in fact you are already enclosed in your provider's VRF with your global routes. Now you are trying to send VPNv4 prefixes to the VRF on the PE route, essentially trying to create a VRF inside a VRF from the provider's viewpoint, and you expect that the PE router will be fine with it.

Sadly, it won't. This scenario is not possible and not supported. If you have a close look on the the show ip bgp neighbor output you have provided yourself, there is an information saying:

BGP neighbor is 198.18.37.213,  remote AS 4323, external link

  BGP version 4, remote router ID 198.18.37.213

  BGP state = Established, up for 1d12h

  Last read 00:00:05, last write 00:00:21, hold time is 90, keepalive interval is 30 seconds

  Neighbor capabilities:

    Route refresh: advertised and received(old & new)

    Address family IPv4 Unicast: advertised and received

    Address family VPNv4 Unicast: advertised

You are trying to speak VPNv4 to your PE but it does not accept that (you are just advertising the capability to speak VRF rotes but you are not receiving a confirmation for such dialog from the PE router). As a result, you are unable to carry your VRF routes over the provider's network to your other location.

I am afraid there is no easy solution to your requirement here. You are currently keeping your networks separate - some are in your global routing table, some are in VRF. However, to the provider backbone, packets destined both to the global and VRF destinations would be indistinguishable - the provider's backbone would need to know both the global and VRF routes, thereby losing their isolation.

One possibility is to establish a GRE tunnel between your CE routers in the data center and the remote location which is assigned to the corresponding VRF (while having the endpoints described by global routing table entries). Then, run a routing protocol (any protocol you wish) over this tunnel within the particular VRF.

Another possibility would be to deal with your provider to allow using MPLS between your CE and his PE, and essentialy configuring what is called a Carrier Supporting Carrier described here:

http://www.cisco.com/en/US/docs/ios/mpls/configuration/guide/mp_carrier_bgp_ps6441_TSD_Products_Configuration_Guide_Chapter.html

The MPLS is used here to prevent the PE routers from learning the routes in your own VRF - instead, only a clever labeling is used that will always direct the packet to the appropriate PE or CE router.

I am aware that neither of these options is ideal but unfortunately, as I explained, it can not be done just as directly.

Best regards,

Peter

[cgarringer]

Thanks, that keeps me from banging my head on this any further.     I will examine the options you suggested and look into ACL's and leaking routes into the VRF at the data center.   Again, thanks for the help.

[cgarringer]

Actually neither the data center or the remote site is getting routes on the vrf ADP.    The default vrf routes have always come across fine, so I assume the vendor is using the as-override option.    I will try the suggestions immediately and will let you know what happens.   Thanks for the quick responses.