Dual ISP Failover for S2S VPN

[Umair Yaseen]

Hello,

I have setup of Dual ISP for redundancy in network, using static routes (Track) and SLA monitor setup, 

Traffic shift from primary to backup is perfect during primary link failure.

Only issue is that every time on link flip (primary to backup or backup to primary) site to site VPN tunnel not come up.

To bring tunnel up I enter manually "clear crypto ikev1 sa"

Is there any solution to fix the manually clear the "ikev1 sa" every time? 

Please note that I am ASA 9.7 Code.

Thank you. :) 

[Rob Ingram]

Hi,
Do you have Dead Peer Detection (DPD) enabled on all devices? (it should be enabled as default on the ASA, but you may have it disabled).
When you flip the connection how are performing this?
Does IP SLA track this correctly and perform the switch of the default route?

HTH

[Umair Yaseen]

Hello Rob, Thank you for reply,
If DPD is enabled by default then Yes it's enabled as I didn't disable it manually.
After link flip Internet works fine on backup, but I needs to clear ikev1 sa manually every time.
And yes default routes are working perfectly as Internet works after link flap/switch but not VPN Tunnel.

[balaji.bandi]

Do you have any high level network diagram to look, what device involved between your ASA and ISP router

also if possible please provider to configuraiton to review.

[Sheraz.Salim · Accepted solution]

here is a good post from Aref. As we know, there is no preemption in IPsec site-to-site VPN on Cisco ASA to the primary peer. If you configure a crypto map with two peers, one as the primary, and another as the secondary, the ASA will try always to initiate the tunnel with the primary peer. If the primary peer fails and become unreachable, then the ASA will initiate the tunnel with the secondary peer. When the primary comes back online, the ASA will not do anything as long as the secondary is reachable. In other words, the primary peer will not take over the control from the secondary; instead the tunnel will remain initiated with the secondary peer.

In a different scenario, that would be an issue, because maybe the secondary peer is located at the edge of a disaster recovery site where you don’t have a complete mirror of your primary site, or maybe the device has less resources, or maybe the link where it is connected is slow.

As a fix, Cisco recommends configuring a SLA monitor to keep track of the primary peer IP address and creating a conditional route of the primary peer IP address tied to the SLA monitor. Then to create an EEM applet to trigger the command “clear crypto ipsec sa peer <the secondary peer IP address>” to clear the tunnel with the secondary peer and re-initiate it with the primary, every time the message ID 622001 is seen twice on the ASA.

The message ID 622001 is generated every time the ASA adds or removes a route from its routing table. That means when the ASA generates the first message 622001 when the primary peer failed, and the second message 622001 when the primary peer came back online.

The message 622001 is similar to these:

%ASA-6-622001: Adding tracked route 60.60.60.1 255.255.255.255 169.254.255.254, distance 1, table default, on interface outside

Or

%ASA-6-622001: Removing tracked route 60.60.60.1 255.255.255.255 169.254.255.254, distance 1, table default, on interface outside

The problem with that fix would arise if you have multiple conditional routes on the ASA tied to multiple SLA monitors. Because, like I said before, every time the ASA adds or removes a route from or to its routing table, it will generate a message ID 622001, with no distinction.

That would mean that the EEM applet would trigger the command you configure to reset the VPN tunnel with no valid reason or when it is not required. Because that would happen when the message 622001 has been generated twice on the ASA, regardless which conditional route has been changed, whether the one for the VPN you set or a different one that has nothing to do with the VPN. For both a message 622001 will be generated.

Here is a quick work around you would configure to make the ASA initiate the VPN tunnel with the primary peer, as long as it is reachable. What I would do is configure a SLA monitor, checking the availability of the primary peer, and creating a conditional route for the secondary peer pointing to a dummy next hop. This means that as long as the primary peer IP address is reachable, the secondary peer IP route will be installed in the routing table pointing to a fake next hop; in other words the secondary peer IP will never be reachable as long as the primary peer IP is alive.

When the primary peer fails, the track of the SLA monitor will cause the removal of the conditional route of the secondary that was pointing to the dummy next hop, and now the route that will be used to reach the secondary peer would be the default route on the ASA.

When the primary peer comes back online, the SLA monitor will go again in UP state, and it will trigger the installation of the secondary route again into the routing table of the ASA, which will send the traffic to the dummy next hop, so the secondary peer will not be reachable any longer.

According to those route removals and additions, the VPN tunnel will automatically switch between the peers. When the primary peer is reachable, the VPN tunnel will always be initiated with the primary peer. And when the primary peer is unreachable, the VPN tunnel will be initiated with the secondary peer.

Here is the topology and the configuration I have used in my lab to prove this:

// asav-1:

HQ-ASA# sh ip

System IP Addresses:

Interface Name IP address Subnet mask Method

GigabitEthernet0/0 inside 192.168.0.1 255.255.255.0 CONFIG

GigabitEthernet0/1 outside 10.10.10.1 255.255.255.0 CONFIG

object-group network VPN-LOCAL

network-object 192.168.0.0 255.255.255.0

object-group network VPN-REMOTE

network-object 192.168.130.0 255.255.255.0

access-list VPN extended permit ip object-group VPN-LOCAL object-group VPN-REMOTE

nat (inside,outside) source static VPN-LOCAL VPN-LOCAL destination static VPN-REMOTE VPN-REMOTE no-proxy-arp route-lookup

crypto ipsec ikev1 transform-set ESP-AES256-SHA esp-aes-256 esp-sha-hmac

crypto map VPN 100 match address VPN

crypto map VPN 100 set peer 50.50.50.1 60.60.60.1

crypto map VPN 100 set ikev1 transform-set ESP-AES256-SHA

crypto map VPN interface outside

crypto isakmp identity address

crypto ikev1 enable outside

crypto ikev1 policy 10

authentication pre-share

encryption aes-256

hash sha

group 2

lifetime 86400

tunnel-group 50.50.50.1 type ipsec-l2l

tunnel-group 50.50.50.1 ipsec-attributes

ikev1 pre-shared-key cisco?123

tunnel-group 60.60.60.1 type ipsec-l2l

tunnel-group 60.60.60.1 ipsec-attributes

ikev1 pre-shared-key cisco?123

sla monitor 100

type echo protocol ipIcmpEcho 50.50.50.1 interface outside

timeout 1000

frequency 3

sla monitor schedule 100 life forever start-time now

track 100 rtr 100 reachability

route outside 60.60.60.1 255.255.255.255 169.254.255.254 1 track 100

route outside 0.0.0.0 0.0.0.0 10.10.10.2 1

// asav-2

REMOTE-01# sh ip

System IP Addresses:

Interface Name IP address Subnet mask Method

GigabitEthernet0/0 outside 50.50.50.1 255.255.255.0 CONFIG

GigabitEthernet0/1 inside 192.168.130.1 255.255.255.0 CONFIG

object-group network VPN-LOCAL

network-object 192.168.130.0 255.255.255.0

object-group network VPN-REMOTE

network-object 192.168.0.0 255.255.255.0

access-list VPN extended permit ip object-group VPN-LOCAL object-group VPN-REMOTE

nat (inside,outside) source static VPN-LOCAL VPN-LOCAL destination static VPN-REMOTE VPN-REMOTE no-proxy-arp route-lookup

crypto ipsec ikev1 transform-set ESP-AES256-SHA esp-aes-256 esp-sha-hmac

crypto map VPN 100 match address VPN

crypto map VPN 100 set peer 10.10.10.1

crypto map VPN 100 set ikev1 transform-set ESP-AES256-SHA

crypto map VPN interface outside

crypto isakmp identity address

crypto ikev1 enable outside

crypto ikev1 policy 10

authentication pre-share

encryption aes-256

hash sha

group 2

lifetime 86400

tunnel-group 10.10.10.1 type ipsec-l2l

tunnel-group 10.10.10.1 ipsec-attributes

ikev1 pre-shared-key cisco?123

// asav-3

object-group network VPN-LOCAL

network-object 192.168.130.0 255.255.255.0

object-group network VPN-REMOTE

network-object 192.168.0.0 255.255.255.0

access-list VPN extended permit ip object-group VPN-LOCAL object-group VPN-REMOTE

nat (inside,outside) source static VPN-LOCAL VPN-LOCAL destination static VPN-REMOTE VPN-REMOTE no-proxy-arp route-lookup

crypto ipsec ikev1 transform-set ESP-AES256-SHA esp-aes-256 esp-sha-hmac

crypto map VPN 100 match address VPN

crypto map VPN 100 set peer 10.10.10.1

crypto map VPN 100 set ikev1 transform-set ESP-AES256-SHA

crypto map VPN interface outside

crypto isakmp identity address

crypto ikev1 enable outside

crypto ikev1 policy 10

authentication pre-share

encryption aes-256

hash sha

group 2

lifetime 86400

tunnel-group 10.10.10.1 type ipsec-l2l

tunnel-group 10.10.10.1 ipsec-attributes

ikev1 pre-shared-key cisco?123

// Testing:

HQ-ASA# sh vpn-sessiondb l2l

Session Type: LAN-to-LAN

Connection : 50.50.50.1

Index : 11058 IP Addr : 50.50.50.1

Protocol : IKEv1 IPsec

Encryption : IKEv1: (1)AES256 IPsec: (1)AES256

Hashing : IKEv1: (1)SHA1 IPsec: (1)SHA1

Bytes Tx : 292572 Bytes Rx : 292572

Login Time : 00:21:21 UTC Tue Jul 24 2018

Duration : 0h:58m:10s

// Breaking the link with the primary peer:

HQ-ASA# %ASA-6-622001: Removing tracked route 60.60.60.1 255.255.255.255 169.254.255.254, distance 1, table default, on interface outside

After the VPN tunnel with the primary times out the ASA reinitiate the tunnel with the secondary peer:

HQ-ASA# sh vpn-sessiondb l2l

Session Type: LAN-to-LAN

Connection : 60.60.60.1

Index : 33744 IP Addr : 60.60.60.1

Protocol : IKEv1 IPsec

Encryption : IKEv1: (1)AES256 IPsec: (1)AES256

Hashing : IKEv1: (1)SHA1 IPsec: (1)SHA1

Bytes Tx : 3528 Bytes Rx : 3528

Login Time : 01:23:06 UTC Tue Jul 24 2018

Duration : 0h:00m:42s

// Re-enabling the link with the primary peer:

HQ-ASA# %ASA-6-622001: Adding tracked route 60.60.60.1 255.255.255.255 169.254.255.254, distance 1, table default, on interface outside

After the VPN tunnel with the secondary times out the ASA reinitiate the tunnel with the primary peer:

HQ-ASA# sh vpn-sessiondb l2l

Session Type: LAN-to-LAN

Connection : 50.50.50.1

Index : 56849 IP Addr : 50.50.50.1

Protocol : IKEv1 IPsec

Encryption : IKEv1: (1)AES256 IPsec: (1)AES256

Hashing : IKEv1: (1)SHA1 IPsec: (1)SHA1

Bytes Tx : 0 Bytes Rx : 0

Login Time : 01:25:01 UTC Tue Jul 24 2018

Duration : 0h:00m:01s

If you are wondering how I could use the ? in the pre-shared key for the VPN tunnels, then here is the trick: When you type a pre-shared key, and you want to use the ? as a character, it would be enough to type ctrl+v right before typing the ? and then you continue typing the rest of the pre-shared key. That is valid on the ASA for anywhere you might need to use the ? as a normal character, not only for pre-shared keys. link is here.

[Umair Yaseen]

Thank you so so much Sheraz for brief reply, It;s very helpful.