Core switch to dual ISP routers

[JJay]

Hi,

I am looking for some design help. We currently have dual routers ISP1 and ISP2 and plan to connect them to a single layer 3 core switch. I am trying to decide what way to go with the design. Do I use an IGP from the core to the CE routers and redistribute BGP or run iBGP to the core. I could just do static routing but then the CE addresses at the branches we can not summarize. I had setup a lab in GNS3 using EIGRP to the core and just raising the delay to the backup ISP to prefer one over the other. Looking for a good design guide for this setup.

Thanks

[Reza Sharifi]

Hi,

There is no need for BGP in the core.  If you are running BGP with service providers, you can run OSPF between the core and the CEs (internal to your network) and redistribute between them.

On your dual routers facing the ISP, you would need  eBGP with each  service provider and IBGP between your routers.

HTH

[JJay]

Thanks.

I had actually thought about running EIGRP since they are both cisco. Any reason you would choose OSPF over EIGRP when running all cisco. The ISP routers are not my responsibility but they are not running iBGP between them at the moment just eBGP with the providers. I had mentioned it to them that they should in the past. I was also thinking I need to block the bgp learned routes from one edge router being advertised through the IGP from the core to the other edge router.

[Reza Sharifi]

EIGRP works fine as well, but just because they are Cisco devices it doesn't mean you can't run other protocols like OSPF.  All Cisco devices support OSPF as well. I like to run standard protocols. Since the CE routers are not your responsibility, and you have 2 providers, you want to make sure your fail-over between the providers is tested and actually works. The CEs should simply advertise a default route to your core and than you redistribute that to OSPF or EIGRP.  This way you are not getting any BGP routes.

HTH

[JJay]

This setup is for our private mpls and we also have another connection for the internet which would be the default route. I know what you mean about standard protocols and was also considering OSPF. 

[JJay]

For now I have just decided to use iBGP to the core switch. The design is really simple and I would just use static routing if I could summarize all the routes. Using iBGP no redistribution has to take place as well. I am preferring one outbound path over another at the core using weight and the inbound from the isp is set by prepending the AS to the backup path.

[Chad Parish]

We have a similar setup and we run EIGRP from our cores to our CE routers (using a separate router for each ISP) then eBGP out to their respective ISP PE router.  We then redistribute BGP into EIGRP and raise the delay value on the CE router neighbored to the ISP we wish to use as a backup.

[JJay]

We have dual 4500's for the core with VSS. There are not a ton of routes through BGP really since we only have about 50 branches. For that reason I was also considering iBGP to the core. Do you run iBGP between the CE routers at least? I was just wondering how traffic would be rerouted if a route failed through the primary ISP but not the secondary. Do you block routes from being advertised from one CE router to another through EIGRP?

[Chad Parish]

Between our CE and the ISP PE routers it's eBGP.  This is then redistributed into EIGRP which we use from our CE to our Core routers.

Because we have our interfaces out to the ISP's always enabled, and we increase the delay within the BGP to EIGRP distribution on the router connected to the back-up ISP, traffic will prefer to go towards our primary ISP.  But should the preferred ISP path be unavailable, EIGRP simply routes traffic out to the other ISP as it is now the most desirable path ...because it's the only path:)

Occasionaly we "flip" the WANs to make sure the other path is still working good, which we can easily do by simply reversing the eBGP distributed delay values within our EIGRP config at the CE routers.

Just remember though, if this is for MPLS, you must make the same delay changes at the other end to make sure both ends of your network are still routing out to the same ISP.

[JJay]

Thanks. I think the issue I had in mind would not be an problem on a MPLS network. At the other end if you mean the branches or other office locations they currently only have 1 MPLS connection.

[JJay]

Hi Chad. Do you also do anything to speed up the failover from the primary to secondary if the BGP neighbor drops to the PE? Like lowering the bgp timers or using ip sla?

[Chad Parish]

No need to update the BGP timers as you are only tweaking EIGRP in-so-far as how it views the delay of the two competing BGP paths.   The BGP connectivity to both ISP's is always established, and so it's really just a matter of EIGRP deciding which eBGP advertisements it will prefer to route traffic towards.

So for example, I have two ISP's, we'll say ATT and Sprint.  I have MPLS with both and have established connectivity via eBGP to both of their PE edge routers from my own CE edge routers.  Within those CE routers I have EIGRP configured for dynamic routing back into my LAN.  When I configure EIGRP on each of these CE routers, on the one connected with the ISP I'd rather not see traffic going to, I increase the delay in my redistribute bgp statement.  EIGRP then reports those metrics back to my cores who opt to route traffic out of the lan via the BGP session with the lowest delay (best metrics).  So traffic destined to hit the WAN is sent to the edge router redistributing the lower delay.