FMC managed FTDs, Site-to-site VPN, Dual ISP both peers

[edh@oneonta.com]

Hello, I am at my wit's end here. I have searched this every way I can think of online and in these forums, so please forgive me if this has already been addressed. I just lack the terms to search it correctly.

I have three sites with FTDs that are managed by an FMC.  At this time we have all point-to-point tunnels terminating at our HQ site, which acts as the hub for all intersite communication. The HQ site has two ISP connections. One of the remote peers also has two ISP connections and the other has a single ISP. We were on version 7.0.x for a long time, so all the tunnels are static VTI for each of their ISP interfaces. This limited us to having two separate tunnels for the dual to dual connection and if ISP1 went down on either side, both were running on ISP2 to ISP2 tunnel which wasn't ideal since our backup connections were slower. Also, the single ISP connection site wouldn't allow for the redundant tunnels because when trying to add the second VTI as a route for the internal networks it errored because of the duplicate paths to a subnet. Therefore when ISP1 went down at HQ site this peer was offline.

Now that we have upgraded to version 7.4.2.2 I was hoping to use the loopback interface and/or the DVTI feature to be able to have all possible ISP tunnel combinations available but in my lab testing I haven't been able to figure out how to set it up correctly. TAC had said it would be possible with these features long ago when I first complained about not being able to do this when I was able with our 5545-X firewalls when the ASA was still in the mix. I have not been able to find any documentation that clearly describes how to set this up. Is anyone doing this and willing to share how you were able to get it to work? Please?

I am attaching a diagram of what I am hoping to do in case this doesn't make sense.

[Rob Ingram]

edh@oneonta.com I don't foresee why that should not work, what have you configured so far? Where are you stuck?

The most appropriate helpful cisco guides for your scenario:-

https://www.cisco.com/c/en/us/support/docs/security/firepower-management-center-1000/222265-configure-dual-isp-vti-on-ftd-managed-by.pdf

https://secure.cisco.com/secure-firewall/docs/sd-wan-wizard

https://www.cisco.com/c/en/us/td/docs/security/secure-firewall/management-center/device-config/740/management-center-device-config-74/sd-wan-capabilities.html

[edh@oneonta.com]

Where I run into an issue is when in static routing I want to say that a subnet can be reached via two different gateways by the same interface I get an error message. Without doing this I am not sure how I can have a static route that traverses the ISP1 to ISP1 tunnel primarily and the ISP1 to ISP2 tunnel if there is an outage. The way I have this set up now is there are two separate tunnels, ISP1 to ISP1 and ISP2 to ISP2, and the routing is controlled by a metric difference. I tried to make VTIs with the separate ISP interfaces as the tunnel source and borrowing the loopback IP but then the routing becomes two routes with identical gateways. I am lost.

[Rob Ingram]

edh@oneonta.com look at ECMP zones for load balancing across multiple ISPs and VTIs.

Why static routing? best practice is using a dynamic routing protocol.

[edh@oneonta.com]

@Rob Ingram I am not looking for load balancing primarily, although if I can accomplish that and the full redundancy I am seeking that would be great. What I am looking for is all potential peerings to be available in case of different combinations of ISP failures at either end. If that means I need to use a dynamic protocol I am okay with that. Just can't seem to figure out which way to go. I am attaching another picture, this one showing all the peering possibilities I am looking for. Back when we had ASAs this worked by setting different priorities for each VPN tunnel and the tunnels/routes wouldn't be created until the one above it was down. If I try this approach with the FTDs they get grumpy about having two paths to the same place.

[Rob Ingram]

edh@oneonta.com establish two tunnels, which would be active at the sametime. Use a metric/weight on the routing protocol to prefer one path over another. When the primary tunnel is down, DPD would clear the tunnel and the routes would disappear, the route via the least preferred path would be active.