Branch 5505, 1 ISP circuit, Dual-peer VPN Configuration to Data Center & Track Options

[Dean Romanelli]

Hi All,

I have a data center with two ISP lines for redundancy, and two ASA 5520's for VPN redundancy to my branches.  Each of my branches has 1 ASA 5505 with a base license, and 1 ISP circuit. Currently all of my VPN tunnels are built to the primary data center ISP circuit only, so if that one goes down, I am toast.  I need to fix that. Problem is, I'm not sure how I can control the fail-over on the branch 5505 with only 1 ISP line.  Please see my drawing for an example of how it looks right now.

So the issue is that the data center LAN my branch needs to get to is the same regardless what data center circuit it comes in on. And I know ASA rules say only 1 VPN tunnel can be active at a time if the flows are the same.  So in that case I know you typically do:

crypto map outside_map 1 set peer 12.x.xxx.20  50.xxx.xx.190

and then configure route tracking to control when to tear down the primary peer and turn up the back up peer. But in the case where I only have 1 ISP on the branch side, I am only going to have 1 default route: route outside 0.0.0.0 0.0.0.0 3.3.3.2 1, which will be used whether the active far-end peer is to primary or to secondary data center circuit. So since I don't have a second route, I can't configure route tracking on the primary route along with an SLA that sets the trigger conditions, because there's nothing to track.

How would one handle a situation like this? Are there any other features that can be tracked besides routes?  I really need to be able to set "num-packets 5" in the SLA so my sites aren't flapping all day, but again, without something to track, I can't really configure a meaningful SLA.  Any help is appreciated.

[Richard Burts]

I would suggest that you are going down the wrong road when you focus on route tracking as part of your solution. You would do much better to use Dead Peer Detection which uses whether it receives responses to ISAKMP as the measure of when a peer is not working.

HTH

Rick

[Dean Romanelli]

Hi Rick,

I agree wholeheartedly. I've honestly not had much luck with ASA's and redundancy using SLA; It always seems to try and bring the backup tunnel online when there is no legitimate failure on the primary and I end up needing to rip out the redundant configs and just rebuild it manually during a failure; Not efficient. I did not know there was another method of handling this but now that I do, I will certainly give DPD a good look.  A question if I may:

I see on the DPD article on Cisco's site that you can configure "one-way" DPD mode when your device is an ASA.  Would you recommend I use this mode on my branch 5505's?  In my head the way I see this working is below. Can you confirm if I am on the right track?

1) Branch ASA 5505 sends "R-U-THERE" message to primary data center peer every X seconds, and will repeat up to 3 transmissions.

2) After the 3rd failed retransmission, the branch 5505 will tear down the primary peer and build the backup peer.

3) Since I am running EIGRP in the data center, and static routing at the branch, when the primary data center ASA detects failure in it's path and can no longer reach the remote sites, EIGRP will look for an alternate path to the branches. The only other path would be via the other data center circuit. Will that circuit determine it has a valid route to the branch based on it's ability to connect to the public address it peers with the branch to? Additionally, do I need to use HSRP on my two data center circuits to make this work correctly, or is EIGRP/DUAL enough to control it?

4) At this point, let's say the circuit is restored.  How will the branch 5505 know that it is time to flip back to primary?

Diego,

Thanks you for that input as well. I will be reading the link you sent over to me about the timers tonight.

[Richard Burts]

I am not familiar with the one way DPD and so can not offer much advice about it. It seems that it could achieve what you need in terms of allowing the Branch ASA to detect a failure of the primary and to bring up the second peer.

It is not clear how you have configured EIGRP. How is EIGRP at the head end learning the routes for the Remote and how would EIGRP detect the failure and withdraw the remote route in the event of a failure? I have had a couple of customers who did use EIGRP over site to site VPN and would have EIGRP detect the failure and direct traffic over the backup connection. But this was on IOS routers and not on ASA, and both tunnels were up all of the time and EIGRP was running over the tunnels. That is certainly quite different from what you need to do if the remote is just using static routes.

It is my assumption that the Remote will continue to try to send ISAKMP after there has been a failure of the connection to the primary and if the primary starts to respond again then the Remote will switch the VPN back to the primary. But that is my understanding and not based on actual experience.

HTH

Rick

[Dean Romanelli]

Hi Rick,

My apologies. Currently, the data center is using a static route to get to each branch, and there are 142 of them. The static routes to get back to the branches from the data center either point to the ASA connected to ISP 1 or the ASA connected to ISP 2.  However, I don't like that because 50% of my sites have their VPN's built to the ASA connected to data center ISP 1 and the other 50% of my sites have their VPN's built to the ASA connected to data center ISP 2.  There is nothing dynamic or redundant about my set up currently, even though there are two circuits, if 1 of them fails, half of the company is down. 

I want to fix that by essentially making all of my sites connect to the ASA connected to data center ISP 1, and only connect to data center ISP 2 if ISP 1 circuit at the data center is down.  However, on the data center side, unless I configure 142 IP SLA's and use route tracking to switch the static route over the backup if the sites are no longer visible from data center ISP circuit 1, I am thinking I will need to use dynamic routing. So my thought was to run EIGRP at the data center for the local networks, and redistribute the static routes from the branches.  

Of course now that I type that out, I'm not sure that would work. I think I would need to run EIGRP at the branches so the data center knows about the LAN's. Otherwise it would never know how to get to the branch LAN's unless I added the static routes back into EIGRP on the core and redistributed them, which defeats the purpose of having dynamic circuit/path failover.

[Richard Burts · Accepted solution]

Thanks for the additional explanation. It does help clarify your environment. Running EIGRP out to the remote would be an attractive option but I am not sure that it is supported on the ASA. I have run EIGRP out to remote peers when using IOS routers (using both GRE with IPsec tunnels and using VTI tunnels) and it was very effective. But on the ASA I believe that you need to look for a different alternative.

It seems to me that using Reverse Route Injection as part of your site to site VPN ought to work. With RRI the ASA inserts a static route for the remote resources when the VPN tunnel is negotiated and traffic can flow. If you redistribute the statics into EIGRP then EIGRP should be learning the routes from whichever ASA has the currently active tunnel. And that should provide the dynamic failover that you need.

HTH

Rick

[Dean Romanelli]

Hi Diego and Rick,

Sorry for the delay in replying back to you.  

For Diego's default route tracking at the data center instead of the 142 static routes idea; Currently there is a 3rd firewall in the mix, which holds all of the services for non-VPN outside internet users when they come in from the public internet and use our company resources in the DMZ. It is also the firewall that ultimately gives internet access to the branches, as we centralize internet access for all 142 sites to come from the data center ISP 1 150Mbps circuit after they arrive here from the VPN. This 3rd firewall shares the same ISP circuit # 1 as the VPN hub firewall. ISP circuit 2 is connected only to the other VPN-hub firewall.

I've amended the drawing to reflect this in the attachment. I didn't think I needed to include it as it is not currently related to the VPN hubs, but now there is value in mentioning it.

So the challenge I see in doing two default routes and route tracking them, is that currently the 142 static routes on the 6509 core switch tell the data center which hub to point to for getting back to the branch, and the default route on the 6509 is currently pointing to that 3rd firewall facing the public internet, and is how the branches ultimately get to the internet, and how outside non-VPN'd users connect to our DMZ services.  I know you're probably thinking that it doesn't make much sense to have dynamic VPN-hub redundancy with only 1 path out to the public internet for the branches to pull internet access from, and I would agree (working on fixing that next), but for now they can at least use company resources in the event ISP 1 was down.  Anyway, doing default route tracking between the two VPN hubs would mean I need to remove the default route to that 3rd firewall on the 6509. And since that 3rd firewall is where everyone has to pass through to get centralized internet and non-VPN users to use DMZ resources, I think it has to be a default route because I can't do a static route for every subnet on the internet.

The next thought I had when considering this idea is why not just roll the 3rd firewall's services into the VPN hubs and eliminate the 3rd firewall?  That way I could do the default route tracking and everything would be great.  However (and please correct me if I am wrong), I think I will run into problems in that case since the internet service is centralized, and if I remove the 3rd firewall and make ultimate internet services come from the same firewall that the VPN tunnels come in on, won't I have to hairpin (i.e. spin it around and shoot it right back out as soon as I receive it from the tunnel) all branch traffic received on that ASA destined for the internet once it comes in from the tunnel as opposed to following a forward-progressing/linear path (i.e. VPN5520--6509---INET5520---internet as opposed to VPN5520 inbound----VPN5520 outbound---internet)? 

If so, then the other challenge currently is that all internet bound traffic from branches must first hit our webfilter connected to the 6509 before going out to the internet.  So if I were to consolidate the 3rd firewall's services into the VPN-1 firewall, I would need a way to ensure that internet bound traffic will go to the 6509 before going back out the VPN-1 hub to get to the public internet.

Rick,

I believe the 5505's do support EIGRP actually. Here is some output options from one of my branch 5505's:

FW59Dallas-CBL5505# show ver | inc asa
System image file is "disk0:/asa825-k8.bin"
FW59Dallas-CBL5505# config t
FW59Dallas-CBL5505(config)# router ?

configure mode commands/options:
eigrp Enhanced Interior Gateway Routing Protocol (EIGRP)
ospf Open Shortest Path First (OSPF)
rip Routing Information Protocol (RIP)

That said, if Diego's comments are correct regarding that site to site VPN's can't propagate protocols, then we probably can't use it.

I am actually not familiar with RRI. Let me go ahead and read up on that.  Would it be implement on branch side, core side, or both?

[Richard Burts]

I see RRI as being useful at the hub but not so much at the branch. At the hub you need to know where the branch is connected and I believe that RRI could achieve that. At the branch all traffic comes to your network edge device and it knows to send traffic to one peer or the other. So I do not see much benefit for RRI at the branch.

The logic from Diego depends on this " now is going to be all branches on ISP1 or all branches on ISP2". As I read your description I wonder if that is the case. Is it possible that most branches will be using ISP1 but that a branch or two might have experienced an issue with ISP1 and have failed over to ISP2? If that is possible then removing the individual routes and using just default routes with tracking seems to be problematic. Perhaps you can clarify how you see this working?

HTH

Rick

[Dean Romanelli]

Hi Rick,

I actually hadn't considered pathing problems for a few sites having an issue with ISP 1 and needing to be switched over. We've definitely had sites have that problem in the past and moving them to ISP 2 has resolved the issues. So Yeah, I think I definitely need to not do an "All branches on ISP 1 unless ISP 1 completely fails" scenario.  I supposed I could just configure 142 IP SLA's on the core switch but that is so messy.

For the RRI idea, so basically in the tunnel configs on the hub side I would do:

crypto map outside_map 1 match address outside_cryptomap_1
crypto map outside_map 1 set pfs group5
crypto map outside_map 1 set peer 3.3.3.1
crypto map outside_map 1 set transform-set ESP-AES-256-SHA
crypto map outside_map 1 set reverse-route

But on the branch side of the VPN tunnel I do not need to set that bold command in the crypto map configs? I had thought the crypto maps had to match exactly on both ends aside from the obvious ACL names and peer IP address for the tunnel to work?

I also see in this guide that they use a route-map that is calling a prefix-list which contains all of the branch subnets that they are using in the redistribute static command.  Is that mandatory, or can I just do "redistribute static" and not call a route map? It also says you need default-information originate in the core ASA under the routing process you are using or the ASA won't send out any networks.

[Richard Burts]

First let me address something from a previous post. I had talked about running EIGRP out to the remote and said that it was not supported on the ASA. I realize now that this was a bit ambiguous. You interpreted it as believing that EIGRP was not supported. And clearly EIGRP is supported on ASA (assuming reasonably recent code version). What I was trying to say was that running EIGRP over the IPsec tunnel out to the remote is not supported.

142 IP SLAs would be very messy and I believe that RRI is likely to achieve what you want. Let me be clear that I have not used RRI and so I speak from what the documentation says and not from experience.

We do frequently say that the crypto maps must be exact matches on both ends. And for a number of things, such as the content of the access list that identifies traffic to be encrypted, whether to use PFS or not and the PFS group, the type of encryption it is true that they must match. But for RRI I do not believe that they need to match.

Using a route map with a prefix list might be considered as a Best Practice. But it is certainly not required. It is very useful when there are some static routes that you do want to redistribute and some static routes that you do not want to redistribute. I know some people who routinely use a route map to control redistribution to prevent the possibility that at some later time someone would add a static route whose redistribution might create problems. They want to control redistribution and to be very intentional about what is redistributed.

I do not know your environment and so do not know if there are static routes on your hub ASA in addition to the static routes for the branch routes. Are there static default routes on the ASA and if so what would be the impact of redistributing those static default routes into EIGRP?

I am not clear what they said about needing default information originate or the context in which they said it. If you already are running EIGRP on the core ASA do you have default information originate configured? Is the ASA advertising routes?

HTH

Rick

[Dean Romanelli]

Hi Rick,

Noted and understood regarding EIGRP on the 5505's, thanks for the additional explanation on that.

I can see why route-maps would certainly be the best way to go. That said, I'm the only network engineer on staff, so for the most part no one else has access, but I am certainly an advocate of doing things in best practice, so I will configure a route map.  

We actually already run EIGRP in the core, but only for local subnets within the data center building, and only between core switch and 3rd internet ASA. The routes to the branches are still via the 142 static routes. We are redistributing static on the EIGRP process on the core SW currently so that the 3rd internet-facing ASA that supplies the branches with internet knows about the 142 VPN routes set on the core, so I guess I would say testing that part out already works and doesn't step on anything it shouldn't. I just checked the 3rd ASA and it is learning everything via D EX, so we are good there.

As for default-information-originate, it is not currently configured under my EIGRP process on the core switch.  Below is a link to the RRI guide I used which specifies this step being required. One big difference however, is that he used OSPF while I use EIGRP.

http://www.petenetlive.com/KB/Article/0000982

Section snippet from article is below:

6. And finally, remember the ASA wont send out any networks, without a default-information originate command.

Main-GW(config-router)# redistribute static subnets route-map RM-VPN-ROUTES

Main-GW(config-router)# default-information originate

[Richard Burts]

It certainly is an important distinction that the article was dealing with OSPF. And I would argue that the article over stated the importance of default-information originate. It is important if you want OSPF to advertise a default route. I do not believe that it is required if you want OSPF to advertise redistributed routes but not necessarily a default route.

HTH

Rick

[Dean Romanelli]

Hi Rick,

Thanks for checking into that for me.  Actually, now that I think about it, in my situation, I think default-information originate would be counter-intuitive in my dual VPN hub setup  If I were to set default-information originate on my ASA VPN hubs 1 & 2, then redistribute static into EIGRP, my core switch is going to learn the default routes of both ASA VPN hubs via D EX. But I don't think my core switch much cares what each ASA's default route is, nor is it require to see it. It only cares which ASA it is currently learning the routes to the branches from at any given time.  

Am I on the right track with that train of thought?

[Richard Burts]

I believe that you are indeed on the right train of thought. If I understand your environment correctly then your core switch does need to have its default route point toward your third ASA. It would seem to be a significant negative if your core switch were to try to use a default route that pointed to one of the ASAs that terminate VPNs from the remote sites.

HTH

Rick

[Dean Romanelli]

Excellent. Thanks so much for your help on this one Rick.  I really appreciate it.