DMVPN 3 Hubs Failover

[vimal vikraman]

I am trying to work on a DMVPN solution. It has abut 10 spokes and 3 Hubs. Hub1 and Hub2 are located in the same location(Say SIngapore). Hub-3 is located at a different location(say USA).

I am trying to achieve failover between the 3 Hubs. All the 3 Hubs will have separate Public IPs that will be advertised to each ISP.

My queries are as follows:

1. The Hub routers to which I need to terminate the Tunnels (IPSEC/GRE) needs to lie behind existing internet terminating routers and network firewalls.
2. Spoke to Spoke communication is not required
3. Device as well as Link redundancy is required 
4. Hence, if Hub-1 fails, tunnel should shift to Hub-2 and then to Hub-3
5. If ISP-1-SG fails then tunnel should be created via ISP-2-SG link. If both the ISP links at Singapore fails, the tunnel should shift to ISP-1-US and then to ISP-2-US.

Can someone please help me how should i achieve this. Should I go for 3 DMVPNs or one DMVPN with 3 Hubs.

How shall I achieve the routing for auto failover. 

[sumit menaria]

Hi.

Check out http://www.cisco.com/en/US/tech/tk583/tk372/technologies_white_paper09186a008018983e.shtml at the "Dual Hub - Dual DMVPN Layout" section. That should get you going.

Auto failover can be achieved by tweeking the routing protocol metrics ,such as bandwith on tunnel interface (for ospf) and delay on tunnel interface (for EIGRP)

Cheers

Sumit Menaria

Rate if it helps

thebetternetwork.com

[vimal vikraman]

Hi Sumit,

Thanks for your response. I have checked the link, but it mostly talks about the Dual Hub scenario over the same ISP. As per the requirement I have, there are multiple ISPs links and 3 hubs.

I am concerned about how the routing will work for CPE to determine which of the link to be used. Do I have to run separate routing between the CPE router and the IPS's PE routers and separate routing for DMVPN.

Also as the Tunnel end point routers (head end side-Singapore and US) are actually placed behind routers and firewalls, and the ISP links do not terminate on these routers. So, will this cause any issue.

[sumit menaria]

Are these Hubs connected to each other directly via some VPN or its just the internet ?

As I understand ,each Hub router will be having an individual ISP uplink and Public IP assigned to it.For this case ,each Hub will have a separate DMVPN tunnel.So the spokes will be having 3 tunnels configured in all and separate IP subnets are required on all 3 tunnels.

Also do you plan to run the routing protocols on the firewall too ? Or will they be acting in transparent mode.

Now for the routing part ,you can tweak the routing protocol metrics at the Hub and Spoke tunnels by BW/Delay etc to ensure preffered hub is used primarily.

For routing towards ISP ,you have to make sure using BGP (in case you are using that),that you are advertising the Public IP of Hub1 only to ISP1 and Hub2 only to ISP2 ,so that in case ISP1 link goes down ,then the tunnel will be brought down too .Thus making Hub2 as primary in this case and finally Hub3 in case two of the ISPs fail.

[vimal vikraman]

Hi Sumit,

Thanks,

We are planning to open firewall rules for those particular Public IPs on the CPE-1 and CPE-2. One query here - if we advertise IP of one router from one link only, we will not be able to achieve device failover. For ex, if the ISP-1 link goes down but the CPE-1 is still okay, plan is to get the tunnel created from the ISP-2 link to the CPE-1.

I am still a bit confused, which routing protocol should be used where. I need to get the routing done for the tunnels as well as towards the ISP for IP advertisement.

[sumit menaria]

Vimal,

That's where I had my first question if  Hubs (CPE) are connected to each other directly also.If yes ,then you can achieve ISP level redundancy .So even if ISP1 goes down ,the CPE1 DMVPN tunnel will remain up ,by Public reach-ability of its IP by ISP2.

Now for the routing protocol part ,You can use ospf/eigrp. And change the cost at the tunnel interfaces at Hub and Spoke to achieve redundancy.

I have attached a diagram for your purpose of understanding.

Let me know if it helps.

Cheers

Sumit Menaria

Rate if it helps

thebetternetwork.com

[vimal vikraman]

Thanks Sumit for the diagram.

I have few more queries:

How to achieve routing between the ISP and the Spoke/Hub location routers. As there are 2 ISPs there will be 2 routes for the same destination IP(Hub), how to get failover from one route to another for the same destination Hub IP.

Do we need to configure BGP between CE and PE routers at each of the locations? If yes, how to get this routing update in the DMVPN network.

Can we use a default route announcement from Hub towards Spoke - the reason is to minimize the route traffic from the Hub towards Spoke.

How can we restrict Spoke to Spoke communication.

[sumit menaria]

I have tried to answer you queries as below.Let me know if they help you.

How to achieve routing between the ISP and the Spoke/Hub location routers. As there are 2 ISPs there will be 2 routes for the same destination IP(Hub), how to get failover from one route to another for the same destination Hub IP.

The ISPs is used just to reach the Public IP address of the Hub/Spoke from the routers.In this case by using BGP between the internet gateway router and the ISP,you can Prefer one ISP over the other for the outgoing traffic by means of Local preference.For the incoming traffic you can use AS-PATH-PREPENDING for the Public IP address of the Hub/Spoke routers.I am assuming you have two ISP links both from different providers (ASN).Otherwise in case you have two links from same service provider ,then you can use MED at the gateway router to influence incoming routing.

Can we use a default route announcement from Hub towards Spoke - the reason is to minimize the route traffic from the Hub towards Spoke.

Yes ,Generally this hub routers are configured to send default route to the spoke ,however the spoke interface may be kept in the different VRF to ensure it does not interfere with local routing there.It is not a must to have a VRF always at Spoke ,but just in case you do not want all your traffic to go towards the Hub and only the selected one.Here the spokes will only be sending their local subnets through IGP to the HUB routers.

How can we restrict Spoke to Spoke communication.

By using NHRP Phase 1.i.e configure multiple GRE (tunnel mode gre multipoint) only on the hub router and point to point GRE (tunnel mode gre ip)  on spoke routers ,which is also the default behavior

Cheers!

Sumit Menaria

Rate if it helps

thebetternetwork.com