Internet address space advertised from 2 locations

[Scott Brien]

Hi,

We are trying to sort a solution regarding internet address space.

What we are trying to accomplish is as follows:

1) Advertise Public address space to our upstream providers from 2 seperate locations ** please note that that 2 sites are connected via dark fibre and connected via a layer 2 trunk.

2) we have the same providers at both sites and advertise our address space to them currently only at one site.

3) our core network has default routes pointing towards the borders routers which are distributed via EIGRP

The challenge here is how we will advertise our address space out of the 2 locations.

Any help regarding this would be greatly appreciated.

I have attached a high level topology of what we are trying to accomplish here.

Regards,
Scott

[dan_wiebell · Accepted solution]

This would just be a straight-forward multihomed BGP setup wouldn't it? You can advertise the same network out of both BGP routers without problems.

Can you please clarify where the problem lies?

[Jon Marshall · Accepted solution]

Scott

Can you please clarify where the problem lies?

What problem ? You haven't told us about any problem.

In answer to your general query yes you advertise the same address space from both sites but what are you trying to achieve ? Bear in mind if you do advertise from both sites then traffic could come in at either entry point unless you influence the traffic with MED or AS PATH prepending.

Also you need to think about traffic leaving your network. Do you want it to go to either exit point or only one and use the other as redudancy.

If you could clarify what it is you want to do and if you are having problems then we should be able to help you.

Jon

[Scott Brien]

Hi Guys,

My apologies, i probably should have written this as a query moreso than an issue, we are currently in design process of this solution and was seeking confirmation on multiple entry/exit points for the public addressing.

In answer to your query re incoming outgoing traffic, as all devices at L2 all devices are aware of the public and will be able to route to the destination regardless of entry point and outgoing traffic we will use uneqal cost load balancing on the core devices which will receive the default routes and if either path fails the other site will take over.

Thanks for your responses guys.

Scott

[Jon Marshall]

Scott

Actually my apologies. I read the response from Dan as a follow up from you for some reason (must get my eyes tested ) hence the bit about the problem.

Jon

[Jon Marshall]

Scott

If you are happy to use both links then why not simply let each site use its own internet connection with default-routes and then use the other site if their internet connection fails. Would this not be easier than using unequal cost load-balancing.

One other point. You make no mention of firewalls but be aware that if there are firewalls in the path then with this sort of setup you can get asymmetric routing ie. traffic goes out one way and comes back in another. Firewalls don't like this and usually drop the traffic. If you do have firewalls then you may need to manipulate the traffic paths.

Jon

[Scott Brien]

Jon,

no issues!

you are right, there is less complexity to using each site independently and using secondary site in case of failure and also does address the asymetric routing you mentioned as there is indeed a firewall at each site which utilizes NAT heavily.

Thanks!

Scott

[Jon Marshall]

Scott

Using each site to route it's own traffic out does not solve the asymmetric routing issue. If you advertise your address space from both sites presumably the NAT being used on the firewalls is from that address space ?

So the problem is that in site B a user goes out to the internet via site B firewall so firewall B makes an entry in the state table. But because you have advertised your address space from both sites the traffic could come back to site A. The site A firewall has no entry in it's state table for the return traffic so it will drop the packet.

Now there are ways around this depending on the firewalls in use etc. but it is definitely something you need to address. Ways around -

1) If the address space is big enough you could split it and advertise one half from site A and one from site B so that return traffic always goes to the right site.

2) if your firewalls are capable of clustering ie. true active/active and they share the state table then it is not an issue. Note ASA firewalls can't do this. They can run active/active but it is not true active/active ie. it is per context and for each context one ASA is active and one is standby. True active/active is when both firewalls are active at the same time and one can receive and allow though connections initiated through the other.

3) You probably don't want to do this but only use one internet connection as the primary connection and have all traffic from both sites using this and then if it fails move over to the 2nd internet connection.

Jon

[Scott Brien]

Jon,

I have been thinking about this asymetric routing and have a query surrounding it.

Considering we are utililzing layer 2 WAN links with trunks between sites so we can utilize our VLANS cross site and we have SVIs on our core devices and obviously our firewalls for our internet space if traffic flows out one border network and enters the other there will be an ARP entry which will point to the firewall which holds the correct address rather than trying to traverse the other sites firewall

So if we were to implement the following:

Seperate Dynamic NAT pools for each site so devices will know where to send the inbound traffic

Static NAT addresses on each firewall will be entered into the ARP table so inbound traffic will be sent to the correct FW

In response to your options see my  response below in bold:

1) If the address space is big enough you could split it and advertise one half from site A and one from site B so that return traffic always goes to the right site.

We dont want to split the address space because then our statically assigned devices in one site will be inaccessible in case of failure.

2) if your firewalls are capable of clustering ie. true active/active and they share the state table then it is not an issue. Note ASA firewalls can't do this. They can run active/active but it is not true active/active ie. it is per context and for each context one ASA is active and one is standby. True active/active is when both firewalls are active at the same time and one can receive and allow though connections initiated through the other.

Unfortunately we are using FWM at one site and ASA 5585-X at the other so this option will not work however this solution would be ideal

3) You probably don't want to do this but only use one internet connection as the primary connection and have all traffic from both sites using this and then if it fails move over to the 2nd internet connection.

We would consider this as an option as bandwidth between sites scales upto 20GBE

Scott

[Jon Marshall]

Scott

So if we were to implement the following:

Seperate Dynamic NAT pools for each site so devices will know where to send the inbound traffic

Static NAT addresses on each firewall will be entered into the ARP table so inbound traffic will be sent to the correct FW

Yes that should work as long as the subnet between the firewalls and the border routers is a common subnet between the sites. You would need to use separate dynamic pools and make sure there is no overlap in NAT addressing on the firewalls. It does also mean that there may well be a lot of intersite traffic ie. traffic goes out one way and in the other.

One way round this would to be to split the subnet in 2, one half for site A and one for site B eg. address space 1 (ad1) from site A and address space 2 (ad2) from site B.

Then you still advertise both ad1 and ad2 from both sites but in site A you use AS PATH prepending on ad2 and in site B you use AS PATH prepending on ad1. This would mean that with both sites working your ISP should route ad1 directly to site A border router and ad2 directly to site B border router but if one site fails both address spaces are still being advertised out of the other site and so traffic will still be routed to that site for all the address space.

Jon

[Scott Brien]

Jon,

Thank you for your comments.

The direction we have decided to take here is as follows:

We will use AS PATH prepending and split the subnet in two and advertise out both sites, as this is a multi customer environment we will split of a /30 address for each customer internet and another /? depending on requirements for other internet services, this way we will save address space by using the /30 rather than /29 and possibly /28 down the track.

Do you see any issues with the design here?

Regards,

Scott

[Jon Marshall]

edited

[Jon Marshall]

Scott

Please ignore previous reply i was talking rubbish about the static NATs.

It will work but be aware you only providing redundancy for the border router/BGP connectivity. You can't provide firewall redundancy this way. The reason being the static NATs ie.

server site B = 192.168.5.10  public IP 177.10.10.1

if you wanted redundancy you would need to add those statics to both firewalls. But the issue is then traffic from the internet to 177.10.10.1 will come in to site B router because of the AS PATH prepending. The site B border router arps out for 177.10.10.1 but because there is a common subnet the firewall at Site A will also see the arp request and reply. So there is a chance that traffic could be sent to the firewall at Site A.

So you can't have the same statics on both firewalls.

Jon

[Scott Brien]

Jon,

That is fine as the firewalls at the different site have different functions so have norequirements for the same static NAT at each site.

Thank you very much for your assistance regarding this design, greatly appreciated!

Regards,
Scott

[Jon Marshall]

Scott

No problem. glad to have helped.

Thanks for the ratings and good luck with the implementation.

Jon