CISCO ASA site-to-site VPN and Multi-WAN remote end issue

[AlexDC]

HiVP

We are looking to replace our existing cisco ASR, acting as VPN concentrators, with cisco ASAs.

The architecture looks something like that :

The ASR is configured to accept VPN sessions via 2 different interface directly connected to the internet and the MPLS.

By default the remote site establish a VPN tunnel to the ASR via MPLS and if it fails then establish a new tunnel via Internet. The remote site LAN is advertised back into the core with reverse route injection.

When the remote site rebuild a VPN session via the internet the ASR isn't bothered and replace the existing static route (reverse route-injection) with the new route. No drama.

We done the exact same setup with the ASA but it doesn't like it. When the remote site rebuild a VPN session via the internet ,which replaces the existing MPLS tunnel, the route appears to be broken and no return traffic (from the inside) to the remote site LAN is routed through.

However we can see the received packets increasing from this new tunnel on the ASA but transmitted packet are stuck at 0.

Like the ASR the ASA is configure with 2 dynamic crypto map on 2 different interfaces.

Any idea ? (tested with different ASA firmware from old ones to the very latest)

thanks.

[Rob Ingram]

@AlexDC 

So a tunnel is establish on the ASA, but no encaps.

Do you have a NAT exemption rule with the correct destination (internet) interface?

Run packet-tracer when the VPN is terminated on the internet interface to simulate the traffic, provide the output for review.

[AlexDC]

Hi Rob,

if I enable both tunnel individually everything works.

It only fails when the traffic fails over from the MPLS to the Internet and vice versa.

When the failover occurs, the ASA as an active tunnel (from the MPLS) but doesn't has the time to realise that it doesn't exist anymore before the new session establishes, this is when it glitches. 

In a glitched state if I logout the new VPN internet base tunnel the remote end rebuild the tunnel and everything works.

This doesn't happen on the ASR, the ASR will keep the old tunnel active until it expires (DPD) but will make the new tunnel the default route back to the remote LAN.

I hope that clarifies things.

thanks,

[Rob Ingram]

@AlexDC 

So are the RRI routes not cleared from the old tunnel and new routes not created?

Do you have DPD configured on the ASA to clear IPSec SAs for the dead tunnel?

Packet-tracer would still be useful to highlight what is going on.

[AlexDC]

Hi Rob,

I think this help me to realise that problem isn't related to the tunnel but how ASA deals with the routes.

Nothing appears in the logs but if I am SSH to the ASA while triggering a failover the ASA ouptut this error :

ERROR: Cannot add route entry, conflict with existing routes

We also redistribute the RRI back into OSPF.

So at this stage I don't know if this an OSPF redistribution issue or if the ASA complains about the existing RRI for the ghost session:

V 10.0.0.0.2 255.255.255.255 connected by VPN (advertised), remote_private

is there a way to force the ASA to replace the existing route ?

regards,

[Rob Ingram]

@AlexDC like I said, do you have DPD enabled on the ASA to clear the dead tunnel (IPSec SAs) and with it the old routes learnt via RRI?

[AlexDC]

@Rob Ingram 

Yes we do but even after the old tunnel is cleaned up the route is not added.

the only way to bring the route up is to logout the session for a new session to establish and the route to be added.

[Rob Ingram]

https://www.ciscolive.com/global/on-demand-library.html?search=vpn&search=vpn&search.event=ciscoliveus2020#/session/1573153557826001JtCQ

[AlexDC]

@Rob Ingram 

Thanks for the resources you shared watch the videos but I am failing to see how this would explain why our actual setup is working on our ASR but not on the ASA.

Why is the ASA rejecting the new route where an ASR is just replacing it with the newest.

You might assume that our implementation is not the best, and I'd agree with that . But the remote end are not cisco device and the VPN implementation is limited to policy base routing.

Further to this we do not have 2 ASA to terminate on, instead they are in a HA cluster so terminating on different box is not possible.

regards,

[Rob Ingram]

@AlexDC the error "ERROR: Cannot add route entry, conflict with existing routes" would imply that the old route is still in place. If the ASA was attempting to add the new route, a new tunnel would already have been established via the other link. If you look at the presentations I shared it suggests using a shorter DPD time interval on the ASA headend. Therefore the old tunnel is deleted on the ASA, the routes removed before the spoke has deleted it's tunnel and attempted to establish a new tunnel.

I am sure there is a logical reason why the ASR is acting differently to the ASA, I don't have access to the devices....

No official guide is applicable in all scenarios, but they do have useful information that can be applied in most scenarios.

[AlexDC · Accepted solution]

To those wondering what is the difference between how RRI is done on an ASR and an ASA is simple :

• On The ASR, a static route is created with the  the remote network, the mask and the next hop :

S 192.168.1.1 [1/0] via 10.0.0.1

the ASR also maintains an independent VPN Routing Table which allows for routes to be tagged and weighted thanks to the distance metric:

192.168.1.1/255.255.255.255 [1/0] via 10.0.0.1 tag 0 count 2 rtid 10 on GigabitEthernet0/0 RRI

IOS also allows the RRI routes to be weighted using : set reverse-route distance 20

The combination of the next hop + the ability to set a metric on the route allows the ASR to accept identical routes for different peer.

• On the ASA, a VPN type route is created and associated with the Interface terminating the VPN tunnel on the ASA. which is causing the issue described in the original post above.

V 192.168.1.1 255.255.255.255 connected by VPN (advertised), outside

In the scenario of the failover described above, because there are no distinction possible if peer remote_site is attempting to rebuild a different VPN session using a different source IP address the ASA will drop the RRI as the previous route already exist.