Redundant geographically separate Internet connections.

[jkeeffe]

I'm starting to research how to have two Internet connections, each one to a different ISP, and each one miles from the other.

Right now I have two internet connections to two different ISPs, and both are at our main data center. ISP-1 is connected to a 7206-DS3 interface and ISP-2 is connected to a second 7206 with a DS3 interface. We have an ASN and BGP peering going on and everthing is working fine. On the inside we have a firewall with a DMZ.

Now I want to take one of the 7206 routers to ISP-2 and move it several miles away to another location. The challenge is I want to have a redundant firewall and DMZ at the new site as well.

Is there a Cisco white paper that addresses this type of dual ISP, geographically separate, dual/redundant DMZ configuration?

I'm sure that large companies have already done this many times over.

[scottlivingston]

Unless I'm missing something here, the only difference is that the single point of failure just moved from it being a single geographic location to now only having (1) ingress/egress i-net door at both. I will assume you have backhaul/mgt access by other means - other than the i-net door.

Concerning routing between your two IDC's w/ just (1) ASN - keep in mind the rule concerning BGP loop avoidance - same AS # in the path.

I'm sure there is more to look at, but those are the 1st two things that pop into my head.

If I have time later I'll dig around for a paper, but I'm sure someone will beat me to it.

scott

[Collin Clark]

I would start with reading the SRND(s) that are applicable.

http://www.cisco.com/en/US/netsol/ns656/networking_solutions_design_guidances_list.html

Internet Edge Design Architectures SRND

http://www.cisco.com/en/US/solutions/ns340/ns414/ns742/ns656/net_design_guidance09186a008014ee4e.pdf

[jkeeffe]

You're correct in that we're moving from a two ISP/single location, to a single ISP per location at two locations - the thought being if one site goes down, the other takes over. And when both sites are up, they are both active. Right now even though we have a connection to two ISPs, if we lose the site (crashing airplane or something as catastrophic) the other site is there and the miracles of routing and loadbalancing - or whatever Best Practices dictates - take over.

[Collin Clark]

It's possible, but there is a ton of planning & designing to do. You'll want to look at things like your IGP, BGP, ASN, DNS, failover methods, etc. The Internet SRND covered what you are looking at doing, it should be a worth while read.

[billy_maclin]

I have this exact scenario in my network, but the difference is that I have a metro ethernet connection between the 2 sites; it's configured as a trunk carrying the outside, DMZ and specific inside VLANs between the sites for SAN replication, MS clustered servers, firewall failover, etc.

We get to have physical and carrier redundancy including failover to a different building 8 miles away, but for the purposes of our network, the failover location appears to be local to our main office greatly simplifying the design and configuration. The outside, DMZ, inside, failover and state interfaces of the redundant firewall pair (PIX) sit on the same VLANs at both sites, and redundant servers/services also get the same treatment.

The cost for the metro ethernet is pretty high, but something my firm believes is necessary for uninterrupted operation.

I should add that the primary reason the metro ethernet was purchased was to allow for SAN replication, but was expanded to include redundancy for all critical systems.

I would also add that this configuration uses the KISS (keep it simple, stupid) principle, and doesn't require a PhD in BGP or any additional equipment other than the layer 2 switches at each end to carry the trunk.

[gerheauserm]

Billy, you have described the exact scenario I have here. We are getting ready to stand up a 2nd ISP connection at my remote DR site (it is actually connected via dark fiber to my main site), and I want a simple bgp config I can put on my ISP routers, so that when I loose BGP nei with my prime carrier, I will then advertise my /24 public to the 2nd carrier. Do you have a sample?

[dgahm]

Are you going to have both data centers active at the same time? Do you have public address space available to assign a different /23 to the new data center? If yes to both, you will need an IBGP link between the sites and Global Site Selectors (GSS-4492)to do the site load balancing and fault tolerance. If one site goes down the remaining BGP router will only advertise its local networks. The GSS will detect the site outage and provide DNS resolution with only the live site's addresses.

Please rate helpful posts.

[pvinci]

We do this with LISP without BGP.

router lisp

locator-table default

locator-set UPLINK

priority 1 weight 50

priority 1 weight 50

priority 50 weight 50

priority 50 weight 50

  exit

!

eid-table default instance-id 0

  database-mapping /24 locator-set UPLINK

You can find out more information at http://lisp.cisco.com

We provide LISP infrastructure services.

You can learn more about this from our  NANOG55 presentation:

Let me know if you have any questions.

Paul

http://vinciconsulting.com/vxnet