Solved: AWS VPN tunnels using VRF & BGP

ccallison · ‎02-13-2015

I would like to use a single 2901 router to connect to multiple AWS VPCs by using VRF instances.

I have used the base configuration provided by Amazon and a few good tips from this forum on the VRF configuration, but I am trying to take it one step further and use BGP instead of static routing.

When I configured BGP as a non-VRF IPv4 address family, the sessions came up perfectly.

The output of "show ip bgp summary" and "show ip route bgp" are below the configuration text.

Because Amazon may supply me with duplicate IP segments for some of the tunnels, I need to make sure that the tunnel endpoints stay within the VRF and do not end up in the global routing table.

When I reconfigured BGP as a VRF IPv4 address family, the BGP sessions no longer come up.

The BGP section of the config that I changed and the output of "sho ip bgp vpnv4 vrf EU-TEST summ" and "sho ip route vrf EU-TEST bgp" are also include at the end of this message.

Question #1

What do I have to do to get BGP to work as a VRF instance?

Once I get it working, I think that I will have to use VRF export/import statements to get routes to be shared between the VRF routing table and the global routing table.

Question #2:

Is that the correct line of thinking and if so, how do I change the next-hop address when the routes are exported from the VRF routing table?

I thought of adding another VRF loopback address that is in the same IP address space as the inside network of the router and using a "set next-hop" in the export route map.

VRF CONFIGURATION SNIPPETS

ip vrf EU-TEST
rd 65000:107
route-target export 65000:107
route-target import 65000:107
!
crypto keyring keyring-vpn-0 vrf EU-TEST
local-address <EXTERNAL-VRF-IP> EU-TEST
pre-shared-key address <AMAZON-ENDPOINT1> key ***keystring***
crypto keyring keyring-vpn-1 vrf EU-TEST
local-address <EXTERNAL-VRF-IP> EU-TEST
pre-shared-key address <AMAZON-ENDPOINT2> ***keystring***
!
crypto isakmp policy 200
encr aes
authentication pre-share
group 2
lifetime 28800
!
crypto isakmp policy 201
encr aes
authentication pre-share
group 2
lifetime 28800
!
crypto isakmp profile isakmp-vpn-0
vrf EU-TEST
keyring keyring-vpn-0
match identity address <AMAZON-ENDPOINT1>
local-address <EXTERNAL-VRF-IP>
crypto isakmp profile isakmp-vpn-1
vrf EU-TEST
keyring keyring-vpn-1
match identity address <AMAZON-ENDPOINT2>
local-address <EXTERNAL-VRF-IP>
!
crypto ipsec transform-set ipsec-prop-vpn-0 esp-aes esp-sha-hmac
mode tunnel
crypto ipsec transform-set ipsec-prop-vpn-1 esp-aes esp-sha-hmac
mode tunnel
!
crypto ipsec profile ipsec-vpn-0
set transform-set ipsec-prop-vpn-0
set pfs group2
!
crypto ipsec profile ipsec-vpn-1
set transform-set ipsec-prop-vpn-1
set pfs group2
!
interface Loopback107
ip vrf forwarding EU-TEST
ip address <EXTERNAL-VRF-IP>
no ip route-cache cef
!
interface Tunnel107
ip address 169.254.254.58 255.255.255.252
ip virtual-reassembly in
no ip route-cache cef
ip tcp adjust-mss 1387
tunnel source <EXTERNAL-VRF-IP>
tunnel mode ipsec ipv4
tunnel destination <AMAZON-ENDPOINT1>
tunnel vrf EU-TEST
tunnel protection ipsec profile ipsec-vpn-0
!
interface Tunnel207
ip address 169.254.254.62 255.255.255.252
ip virtual-reassembly in
no ip route-cache cef
ip tcp adjust-mss 1387
tunnel source <EXTERNAL-VRF-IP>
tunnel mode ipsec ipv4
tunnel destination <AMAZON-ENDPOINT2>
tunnel vrf EU-TEST
tunnel protection ipsec profile ipsec-vpn-1
!
interface GigabitEthernet0/0
ip address <EXTERNAL-IP-ADDRESS>
duplex auto
speed auto
!
router bgp 65534
bgp log-neighbor-changes
no bgp default ipv4-unicast
!
address-family ipv4
bgp router-id <EXTERNAL-VRF-IP>
network 10.3.0.0 mask 255.255.0.0
network 172.18.0.0
neighbor 169.254.254.57 remote-as 9059
neighbor 169.254.254.57 timers 10 30 30
neighbor 169.254.254.57 activate
neighbor 169.254.254.57 soft-reconfiguration inbound
neighbor 169.254.254.61 remote-as 9059
neighbor 169.254.254.61 timers 10 30 30
neighbor 169.254.254.61 activate
neighbor 169.254.254.61 soft-reconfiguration inbound

swpl-vpn01# sho ip bgp summ

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
169.254.254.57 4 9059 15 14 4 0 0 00:01:31 1
169.254.254.61 4 9059 13 16 4 0 0 00:01:32 1

swpl-vpn01#sho ip route bgp

B 10.67.0.0/16 [20/0] via 169.254.254.61, 00:01:38

<BGP address family changed from non-VRF to VRF>

router bgp 65534
bgp log-neighbor-changes
no bgp default ipv4-unicast
!
address-family ipv4 vrf EU-TEST
bgp router-id <EXTERNAL-VRF-IP>
network 10.3.0.0 mask 255.255.0.0
network 172.18.0.0
neighbor 169.254.254.57 remote-as 9059
neighbor 169.254.254.57 timers 10 30 30
neighbor 169.254.254.57 activate
neighbor 169.254.254.57 soft-reconfiguration inbound
neighbor 169.254.254.61 remote-as 9059
neighbor 169.254.254.61 timers 10 30 30
neighbor 169.254.254.61 activate
neighbor 169.254.254.61 soft-reconfiguration inbound

swpl-vpn01#sho ip bgp vpnv4 vrf EU-TEST summ

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
169.254.254.57 4 9059 0 0 1 0 0 never Idle
169.254.254.61 4 9059 0 0 1 0 0 never Idle

swpl-vpn01#sho ip route vrf EU-TEST bgp

Gateway of last resort is 208.85.206.193 to network 0.0.0.0

Jon Marshall · ‎02-15-2015

I haven't done what you are trying to do so i may be completely wrong but don't you need to put the actual tunnel itself into the VRF with the "ip vrf forwarding EU-TEST" command under your tunnel interfaces.

The "tunnel vrf EU-TEST" tells the router the tunnel endpoints ie. the source and destination are in the VRF but not the actual tunnel interface itself.

Like i say haven't done your specific setup before so please feel free to ignore.

Jon

View solution in original post

Reza Sharifi · ‎02-13-2015

Not sure why you are trying to use VRF with AWS. I have a setup with BGP to AWS.

If this is direct connect to AWS, you can simply create a sub-interface for each BGP session with AWS. The way, you don't have to worry about import/export anymore. It will be in the global config with sub-interfaces and /30 per session.

HTH

ccallison · ‎02-13-2015

It's not direct connect, it's VPN tunnels. You can only have one VPN tunnel per region to a single IP address and I need to connect to three VPCs in a region, so I need 3 VPN tunnels and 3 IP addresses on my side. Our ISP won't allow us to split up our IP segment, so I have to use VRF for the loopback interfaces.

Reza Sharifi · ‎02-13-2015

Ok, I understand it now. Can you try adding "redistribute connected" under address-family ipv4 vrf EU-TEST and see if the BGP sessions come up?

ccallison · ‎03-02-2015

I've been able to get the configuration working and I'm posting my template for anyone's use. The VPN endpoint,the VPN tunnels and the inside interface that I use as a transit to the LAN are all in the VRF and the BGP connections are in the VRF, so everything is isolated.

I like being able to keep Amazon's 169.254.0.0/16 routes out my network, since I typically include them in bogon lists.

I dumped the configuration to a spreadsheet and made formulas out of all of commands, so now I can recreate it and lay it down with very few changes on any IOS router.

It works perfectly when passing traffic off to another device (for production environments, I planned on having this traffic pass through a firewall anyway), but for the corporate tunnels that don't require as much security, I was hoping to be able to pass the traffic back to the inside interface of the router and use import/export route-maps to pass routes between a global OSPF and VRF BGP process and not use any static routes.

This is still a pretty good tradeoff, though.

Jon Marshall · ‎03-03-2015

Thanks for posting this.

Glad you got it all working in the end.

Jon

Jon Marshall · ‎02-15-2015

I haven't done what you are trying to do so i may be completely wrong but don't you need to put the actual tunnel itself into the VRF with the "ip vrf forwarding EU-TEST" command under your tunnel interfaces.

The "tunnel vrf EU-TEST" tells the router the tunnel endpoints ie. the source and destination are in the VRF but not the actual tunnel interface itself.

Like i say haven't done your specific setup before so please feel free to ignore.

Jon

ccallison · ‎02-17-2015

Yes, that may be the missing piece. I'm going to make some changes and retest.

ccallison · ‎02-17-2015

Yep, that was the missing statement. BGP popped up right away. Thanks for the extra pair of eyes, Jon.

Jon Marshall · ‎02-17-2015

No problem, sometimes a second pair of eyes is all that's needed :-)

Glad you got it working.

Jon

ccallison · ‎02-13-2015

debugging BGP gave me a few more clues and now with the addition of these commands, the session gets to active but then switches back and forth between active and idle.

router bgp 65534
!
address-family ipv4 vrf EU-TEST
neighbor 169.254.254.57 update-source Tunnel107
neighbor 169.254.254.61 update-source Tunnel207
neighbor 169.254.254.57 ebgp-multihop 255
neighbor 169.254.254.61 ebgp-multihop 255

ip route vrf EU-TEST 169.254.254.57 255.255.255.255 Tunnel107
ip route vrf EU-TEST 169.254.254.61 255.255.255.255 Tunnel207

swpl-vpn01# sho ip bgp vpnv4 vrf EU-TEST summ

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
169.254.254.57 4 9059 0 0 1 0 0 never Idle
169.254.254.61 4 9059 0 0 1 0 0 never Active