Site-to-Site IKEv2 VPN - Multiple Tunnels

[Devlin Thornicroft]

Hello all

We are currently setting up a number of Site-to-site IKEv2 VPN tunnels between our data centres using ASR 1002-X routers. We are doing the following:

- Using RSA certificates for authentication

- Each IPsec-protected tunnel is in its own unique VRF

- We are using CRLs for revocation checking

- We are doing certificate chaining - e.g. we have 2x Trustpoints; one containing the Sub-CA (issuing) certificate + the router identity certificate and one containing the Root-CA certificate

We are finding that with 1x Tunnel, all is well and good. The VPN establishes and we can see the CRL check. Brilliant. When we introduce another Tunnel into the mix it starts to get weird. It seems that with our configuration the routers can only support one Tunnel at a time. They essentially flap up and down with Tunnel 1 establishing for a few seconds, it goes down, then Tunnel 2 comes up, then goes down. And so on. 

Does anyone have any experience in a similar setup, if so how did you work around it? Our configuration:

hostname R01-OP

!

ip vrf tunnel1
rd 97:97
!

ip vrf tunnel2
rd 98:98
!

ip vrf crl
rd 99:99

!

ip host vrf crl <SUB-CA> <SUB-CA IP ADDRESS>

!

crypto pki trustpoint SUB-CA
chain-validation continue ROOT-CA
vrf crl
revocation-check crl
rsakeypair RSA-KEYPAIR
!
crypto pki trustpoint ROOT-CA
revocation-check crl

vrf crl
rsakeypair KEYPAIR

!

crypto pki certificate map CERT-MAP 10
subject-name co r02-op

!

crypto ikev2 proposal IKEv2-PROP
encryption aes-gcm-128
prf sha256
group 14
!
crypto ikev2 policy IKEv2-POL
proposal IKEv2-PROP
no crypto ikev2 policy default
!
!
crypto ikev2 profile IKEv2-PROF
match certificate CERT-MAP
identity local dn
authentication remote rsa-sig
authentication local rsa-sig
pki trustpoint SUB-CA
pki trustpoint ROOT-CA
!
no crypto ikev2 http-url cert

!

crypto ipsec transform-set FOUNDATION esp-gcm
mode tunnel

!
crypto ipsec profile FOUNDATION-PROF
set transform-set FOUNDATION
set pfs group14
set ikev2-profile IKEv2-PROF

!

interface Tunnel998
ip vrf forwarding tunnel1
ip address 1.1.1.1 255.255.255.252
tunnel source Port-channel1.998
tunnel destination 172.16.1.2
tunnel protection ipsec profile FOUNDATION-PROF
!

interface Tunnel999
ip vrf forwarding tunnel2
ip address 2.2.2.2 255.255.255.252
tunnel source Port-channel1.999
tunnel destination 172.16.2.2
tunnel protection ipsec profile FOUNDATION-PROF
!

interface GigabitEthernet0/0/0

description TO-SUB-CA-FOR-CRL
ip vrf forwarding crl
ip address xx.xx.xx.xx xx.xx.xx.xx

!

ip route vrf crl <TO SUB CA>

The router on the other side has identical configuration albeit with the IP addresses etc. swapped around.

Any help or advice on this matter would be greatly appreciated.

Thank you

[Philip D'Ath]

I can't see any "crypto pki server" line.  What is acting as your CA server?

Are you trying to use a single router as a CA server to serve up multiple CRLs?

[Devlin Thornicroft]

Hello all

Thanks for your replies.

We are using a full blown enterprise MS PKI solution hence the lack of the command 'crypto pki server'. Currently we have multiple IKEv1 VPNs between the two routers authenticating via PSK.

Richard - to address your comment, as p.dath mentioned I would have thought as long as the source and destination endpoints were unique we'd be good to go.

Finally we are chaining the certificates because of a mandate from our customer that we have to. A simple single trustpoint would have suited me fine. It don't believe this to necessarily be the source of the issue. What the issue is, I don't know. Got a TAC raised so will see what comes of that and update here.

Thanks

[Philip D'Ath]

Would you mind sharing the software version you are running?  This is starting to smell like a bug.

[Devlin Thornicroft]

Happy to. Unfortunately it won't be until Monday as I'm now out of the office. If I could ask you to bear with me till then. 

Thanks again!

[Devlin Thornicroft]

Hi all

Thought I'd update this thread in the hope that it may help someone else in the future. With the help of TAC we resolved our issue. What we needed to do was to create a separate IKEv2 profile and IPsec profile for each IPsec VPN. Under each IKEv2 profile we had to ensure we had a unique identity set. Something a bit like this:

Site 1 Router:

crypto ikev2 profile IKEv2-PROFILE-1
match identity remote fqdn SITE-2-TUN-1
identity local fqdn SITE-1-TUN-1
authentication remote rsa-sig
authentication local rsa-sig
pki trustpoint SUB-CA

!

crypto ikev2 profile IKEv2-PROFILE-2
match identity remote fqdn SITE-2-TUN-2
identity local fqdn SITE-1-TUN-2
authentication remote rsa-sig
authentication local rsa-sig
pki trustpoint SUB-CA

These were then tied to the IPsec profiles as follows:

crypto ipsec profile IPSEC-PROFILE-1
set transform-set TS
set pfs group14
set ikev2-profile IKEv2-PROFILE-1

!

crypto ipsec profile IPSEC-PROFILE-2
set transform-set TS
set pfs group14
set ikev2-profile IKEv2-PROFILE-2

Site 2 Router:

crypto ikev2 profile IKEv2-PROFILE-1
match identity remote fqdn SITE-1-TUN-1
identity local fqdn SITE-2-TUN-1
authentication remote rsa-sig
authentication local rsa-sig
pki trustpoint SUB-CA

!

crypto ikev2 profile IKEv2-PROFILE-2
match identity remote fqdn SITE-1-TUN-2
identity local fqdn SITE-2-TUN-2
authentication remote rsa-sig
authentication local rsa-sig
pki trustpoint SUB-CA

!

crypto ipsec profile IPSEC-PROFILE-1
set transform-set TS
set pfs group14
set ikev2-profile IKEv2-PROFILE-1

!

crypto ipsec profile IPSEC-PROFILE-2
set transform-set TS
set pfs group14
set ikev2-profile IKEv2-PROFILE-2

With this we now have 2+ IKEv2 VPNs between our sites. 

[Philip D'Ath]

Would you be able to share:

debug crypto pki transactions

debug crypto pki validation

debug crypto ikev2

[Philip D'Ath]

Why bother chaining the router certificate?  Why not just put it into its own trustpoint (this is what I usually do)?

What is the motivation of using a sub-ca?

[Richard Burts]

I am not clear if it changes things when the implementation uses VRFs and certificates. But my experience has been that a single tunnel between devices works fine. But attempting to use a second VPN tunnel between the same devices does not work. My understanding of the crux of the problem is that while you may have two separate tunnels (and certainly two tunnels between the same peers does work) that you are attempting to create a second crypto peer relationship between the same two devices and IOS does not do that.

HTH

Rick

[Philip D'Ath]

The SA's are keyed on (source ip, destination ip), and they are using a different source and destination address for each tunnel, so it "should be" unique and ok.