Multiple set-peer statements - failover behaviour

[layer3]

Hello all,

I would like to confirm the behaviour of using multiple statements of the 'set-peer' command under a single crypto-map sequence for IPSEC tunnel failover. In particular, what happens when a peer that was previously unavailable comes back to service?

For example:

I have two sites; Site 1 and Site 2 have two routers. All are connected via a routed cloud.

The routers at each site have the following crypto-map configured:

Site1 - Both Router A and Router B:

crypto map Encryption 10 ipsec-isakmp

set peer RouterD
set peer RouterC

set transform-set 3desMd5
match address 101

Site2 - Router C and Router D:

crypto map Encryption 10 ipsec-isakmp
set peer RouterA

set peer RouterB

set transform-set 3desMd5
match address 101

HSRP is used on the LAN side between the two routers at each site to prefer Router A and Router C respectively as the active device.

According to the Cisco documentation:

'you can specify multiple peers by repeating this command (set-peer). The peer that packets are actually sent to is determined by the last peer that the router heard from (received either traffic or a negotiation request from) for a given data flow. If the attempt fails with the first peer, Internet Key Exchange (IKE) tries the next peer on the crypto map list'.

http://www.cisco.com/en/US/docs/ios/security/command/reference/sec_s2.html#wp1046908

If RouterA has an IPSEC tunnel to routerD.. and router D fails, a tunnel is build between routerA and routerC. When router D comes backup.. does the traffic switch back to Router D to Router A?

What sequence of events would constitute the tunnel not failing back over once the primary router has recovered from a failure?

Thanks in advanced.

[Lei Tian]

Hi,

Your understanding is correct. If RouterA has an IPSEC tunnel to routerD.. and router D fails, a tunnel is build between routerA and routerC. When router D comes backup, there is no auto switch back from current peer to primary peer. The switch back only happens if the current peer goes down or the current peer has been idle for a certain time.

You can consider running routing protocol on top of VPN to get failover and tailback automatically; or you can use EEM + IP SLA to achieve that.

HTH,

Lei Tian

[layer3]

Hi Lei,

Thanks for the prompt response.

I have tested this using eBGP between the provider and the two sites. BGP is configured to use the path between routerA and routerD as primary (see diagram).

Interesting traffic is defined as each sites LAN range respectively and the IPsec tunnel end-points are the CE WAN IP addresses.

When router D fails, routerC builds a tunnel to routerA. However, when I restore router D, router A automatically reverts back to router D. The crypto session table on router A shows an IDLE entry for router C and an Active entry for Router D at this stage.

According to the documentation, shouldn't router A continue to use the tunnel to Router C, instead of reverting back to routerD?

eg/ there is a flow from a host from site2 to site1 via the primary path. RouterD fails... the flow now is initiated from Router C to router A. When Router D comes back up, should RouterA continue to send packets to Router C because its the last successful peer that RouterA received a packet for?

Thanks.

[Lei Tian]

Hi,

Can you do a show ip route site2_subnet before and after RouterD comes back?

Regards,

Lei Tian

[layer3]

Hi Lei,

Here is the output:

Router D is down:

RouterA#show crypto session
Crypto session current status

Interface: FastEthernet0/0
Session status: UP-IDLE
Peer: 10.2.1.58 port 500
  IKE SA: local 10.2.36.178/500 remote 10.2.1.58/500 Active
  IPSEC FLOW: permit ip 192.168.60.0/255.255.252.0 192.168.0.0/255.255.252.0
        Active SAs: 0, origin: crypto map

Interface: FastEthernet0/0
Session status: UP-ACTIVE
Peer: 10.2.1.218 port 500
  IKE SA: local 10.2.36.178/500 remote 10.2.1.218/500 Active
  IPSEC FLOW: permit ip 192.168.0.0/255.255.252.0 192.168.60.0/255.255.252.0
        Active SAs: 2, origin: crypto map

RouterA#show ip route 192.168.60.0
Routing entry for 192.168.60.0/22
  Known via "bgp 65000", distance 20, metric 0
  Tag 65530, type external
  Last update from 10.2.36.177 00:03:48 ago
  Routing Descriptor Blocks:
  * 10.2.36.177, from 10.2.36.177, 00:03:48 ago
      Route metric is 0, traffic share count is 1
      AS Hops 2
      Route tag 65530

HOSTA#traceroute 192.168.60.100 numeric

Type escape sequence to abort.
Tracing the route to 192.168.60.100

  1 192.168.0.2 32 msec 68 msec 8 msec
  2 10.2.1.218 68 msec 12 msec 12 msec
  3 192.168.60.100 84 msec *  108 msec
HOSTA#

---------------------------------------

After Router D has recovered:

RouterA#show crypto session
Crypto session current status

Interface: FastEthernet0/0
Session status: UP-ACTIVE
Peer: 10.2.1.58 port 500
  IKE SA: local 10.2.36.178/500 remote 10.2.1.58/500 Active
  IKE SA: local 10.2.36.178/500 remote 10.2.1.58/500 Inactive
  IPSEC FLOW: permit ip 192.168.60.0/255.255.252.0 192.168.0.0/255.255.252.0
        Active SAs: 0, origin: crypto map
  IPSEC FLOW: permit ip 192.168.0.0/255.255.252.0 192.168.60.0/255.255.252.0
        Active SAs: 2, origin: crypto map

Interface: FastEthernet0/0
Session status: UP-IDLE
Peer: 10.2.1.218 port 500
  IKE SA: local 10.2.36.178/500 remote 10.2.1.218/500 Active

HOSTA#traceroute 192.168.60.100 numeric

Type escape sequence to abort.
Tracing the route to 192.168.60.100

  1 192.168.0.2 20 msec 44 msec 0 msec
  2 10.2.1.58 84 msec 16 msec 12 msec
  3 192.168.60.100 48 msec *  108 msec

RouterA#show ip route 192.168.60.0
Routing entry for 192.168.60.0/22
  Known via "bgp 65000", distance 20, metric 0
  Tag 65530, type external
  Last update from 10.2.36.177 00:02:15 ago
  Routing Descriptor Blocks:
  * 10.2.36.177, from 10.2.36.177, 00:02:15 ago
      Route metric is 0, traffic share count is 1
      AS Hops 2
      Route tag 65530

Thanks.

[Lei Tian]

Hi,

Thanks for the information. The next question I have is, when Router D is down, will Router C become the HSRP primary? When Router D comes back, does Router D get the HSRP primary  back?

Regards,

Lei Tian

[layer3]

Lei,

Yes, that is correct. Router C becomes the HSRP primary when Router D is down (or when Router D's WAN link goes down). When Router D comes back up, Router D becomes the active HSRP and Router C is standby.

Thanks.

[Lei Tian · Accepted solution]

Ok, that explains the behavior. When Router D comes back, it becomes the HSRP primary, the traffic leaving from site 2 will use Router D as the exist point; Router D will initial ipsec session to Router A. It looks like switch back to Router D, but it is just because Router D initial the ipsec session to Router A once it comes back.

HTH,

Lei Tian

[layer3]

Hi Lei,

Thanks for helping clarify this behavior. Much appreciated.

[tech.linkwave]

Can you please share the Configuration ? i have 2 routers at my Corp and 1 at Branch and want to configure the Ipsec Fail over.

Thanks,

Sandy 

[layer3]

Hi Lei,

I've been doing some further testing on this topology and have found that using HSRP to engineer traffic to use the primary path works when there is a single failure (i.e Router D alone shuts down then recovers or Router A shuts/recovers). However, when there is multiple failures (i.e. Router D and Router A shuts down then recovers) traffic then takes the path of Router A <--> Router C & Router B <--> Router D in a criss-cross fashion.

I've attached some diagrams to help illustrate my failure condition during (I'm a visual person ):

1. Normal operation - Router A to Router D
2. 1 Failure - Router D fails
3. 2 Failure - Router A fails
4. Recovery - Router A & Router D comes backup.

Note: During steps 1 - 3 above I initiate ICMP packets from HOSTB to HOSTA. After the recovery (step 4), I then initiate ICMP from HOSTA to HOSTB. At that point, Router A and Router D use their secondary crypto peer for traffic.

In the above situation, even though HSRP is the primary on both Router A and Router D, they do not use their primary crypto peer. As described in our previous posts.

My question is, if Router D goes down alone, then recovers it uses its primary tunnel. However, if both Router D and A goe down, then recover, they don't use their primary tunnel. What is causing this behavior?

Thanks in advanced.

[Lei Tian]

Hi,

My thinking is after Router A and Router D both come backup, it will take some time for the route to propagate to other sites. Once Router A comes up, after the BGP peer is up, it will learn routes advertised by Router C right away; however, it will take sometime for Router A to receive the routes advertised by Router D.  Therefor, Router A will build ipsec tunnel with Router C, because Router C is the only reachable peer at that moment. Once tunnel is built up, it will not switch back to the primary peer.

You can verify that by checking the reachability between Router A and Router D after they come backup.

HTH,

Lei Tian

[Paul Stajewski · Accepted solution]

I think you would be better off with what Lei suggested earlier, and that is to run an IGP on top of your VPN tunnels, or re configure your BGP to advertise your LAN subnets over the VPN instead. This way you do not have to rely on the crypto maps to choose the path, your routing protocol will do that instead.