BGP route sharing between global routing instance and connected VRFs

[John Hinckley]

Hello Everyone,

I'm trying to work out a BGP design that talks to 3 different ISPs and 2 VRFs on a single L3 switch.  I've been working this out for days in my CML lab with some minor success but I'm starting to wonder if what I'm trying to do isn't even possible.  

Here is the scenario:  I have a L3 switch that connects to 3 different ISPs and has 2 VRFs configured for customer space.  The switch has IPs in both VRFs.  The switch is BGP peering with all 3 ISPs in the global instance and can reach all 3 without issue.  Both customers need to receive a default route and their subnets advertised out to ISPs accordingly. 

Customer 1 VRF:

Should receive a redistributed default route from ISP 1 and ISP 2 for redundancy but should favor the route from ISP 2.  Customer 1 should advertise its subnets from the global instance to both ISPs and favor the advertisements to ISP 2.

Customer 2 VRF:

Should receive a redistributed default route from ISP 3 and ISP 1 for redundancy but should favor the route from ISP 3.  Customer 2 should advertise its subnets from the global instance to both ISPs and favor the advertisements to ISP 3.  

I attached a very high-level diagram to help visualize what I'm trying to do.  Assume L3 connectivity between the switch to the ISPs and the VRF hosts.  I would also be fine using an interior protocol like OSPF or EIGRP on the VRF side if that makes things cleaner. Lastly, to clarify, the L3 switch will receive all 3 default routes in the global BGP instance and then redistribute them to the VRFs according to preference.   

Please let me know if I'm ice-skating uphill with this design and if so, is there a viable way to get this done on one device? I have to use BGP on the edge, everything else is flexible.  

TIA,

-John

[MHM Cisco World]

from first view there is no issue at all if you run BGP
just you want to take care about route-target.

[John Hinckley]

Thanks for responding.  So use iBGP for the VRFs and eBGP for the ISPs? Can you provide a small, config snippet for the flow from one VRF to an ISP?  Feel free to makeup interface names. Thx.

[Harold Ritter · Accepted solution]

Hi @John Hinckley ,

Having the 3 SPs in the global won't work, as only one default route will be selected. You should put ISP2 in vrf RED and ISP3 in vrf ORANGE. You then need to selectively export routes from RED and ORANGE to the Global and vice versa. You should also set the weight on the neighbor statement to ISP2 and ISP3 so that the default route learnt from these ISPs is preferred over the one from ISP1. 

Regards,

[John Hinckley]

Hi @Harold Ritter , 

If I move those SPs into VRFs, will each VRF be able to get a default route from the respective SP in the other VRF when connectivity to the primary is lost?  The whole purpose of this design is to provide internet redundancy and outbound BGP advertisement when there is loss of service. Thanks. 

[Harold Ritter]

Hi @John Hinckley ,

> Customer 1 VRF:

> Should receive a redistributed default route from ISP 1 and ISP 2 for redundancy but should favor the route from ISP 2.  Customer > 1 should advertise its subnets from the global instance to both ISPs and favor the advertisements to ISP 2.

> Customer 2 VRF:

> Should receive a redistributed default route from ISP 3 and ISP 1 for redundancy but should favor the route from ISP 3.  Customer 2

> should advertise its subnets from the global instance to both ISPs and favor the advertisements to ISP 3.  

My understanding of your design was that Client 1 would use ISP2 as its primary Internet access and ISP1 as a backup and Client 2 would use ISP3 as its primary and ISP1 as a backup. I am right about that? Do you need more redundancy? If so, you could always import the default from the other VRF, but it gets a bit more complicated.

Regards,

[John Hinckley]

No you are spot on, that is exactly what I need.  I also need each VRF to have its inet subnet advertised to both ISPs so NAT and internet hosts in/for each subnet work when the primary path goes down.  

[Harold Ritter]

Hi @John Hinckley ,

It sounds like the solution I proposed should work for you then.

Regards,

[MHM Cisco World]

I start build lab from yesterday, but I want to explain why @Harold Ritter suggest make ISP and Cust in same VRF, 
we push default route from CoreSW to Cust, and let say the default route via ISP2 
the Cust will forward traffic to CoreSW (because all default route have one single Next-hop which is CoreSW)
the CoreSW receive the packet from Cust and use it RIB to forward traffic and it can ISP2 or it can ISP3/1, this because the RIB check destination not source of packet.
we can use PBR or as @Harold Ritter  suggest and this I will use in my lab, using VRF in which we divide the routing table into multi VRF, 
the packet receive from Cust will forward by CoreSW according to VRF table which can sure make it through one ISP.

[MHM Cisco World]

as I already mention I config two VRF in CoreSW each one RED & ORANGE have specific routing table,
I config the BGP and it work prefect but 
let me share the Issue I face and try to find answer for it, 
in each VRF there are default route toward one ISP, RED have ISP2 and ORANGE have ISP3 as default route, 
when I import from one VRF into other, I make CoreSW have for each VRF two path for default route (in my lab path through ISP2 win). 
I try use import map and set the LP weight AS-prepend but I dont luck. 
@Harold Ritter  have idea how we can solve this issue. 

[Harold Ritter]

Hi @MHM Cisco World ,

The design the OP wants to deploy is for Client1 to use the default route via ISP2 as a primary and ISP1 as a backup and Client2 to use ISP3 as a primary and ISP1 as a backup. There is not need to leak between VRF RED to ORANGE, just between global and VRFs.

Regards,  

[MHM Cisco World]

Yes but if you check my previous comment, you see I mention why we need PBR or VRF, 
if we leak between global to VRF then the packet from Cust RED will forward by CoreSW through ISP3 (if CoreSW select default route via ISP3 as best path "win"), but he want and he add this note in his topology that each VRF use specific ISP.

solution here either 

1-config PBR and 
VRF-GRT leaking "for return path"
2-
config VRF for CoreSW_ISP and Cust
config import map and modify LP/Weight/AS-path for backup

[John Hinckley]

Harold's solution worked as he described.  Thank you very much, Harold! 

-John