05-24-2016 07:26 AM - edited 03-12-2019 07:21 AM
Hello community,
I'm currently evaluating CSR at AWS (60-day trial) and already got around the usual problems and architectural speciality of AWS network design.
I cannot open a TAC case, because we did not buy licence yet. We will, after this last issue is fixed.
Current setup:
This is working fine - no issues here. All HA tests run like expected. So far, everything is fine.
We now needed to create a VPN connection to a special onPremise location of our company. We needed to create a policy based VPN to that location (no support for route based VPN there). This is a two-to-one VPN. Two CSRs connecting to one onPrem gateway. Both tunnels run the same encryption domain. Routing onPrem is based on tunnel state. We put this tunnel in the front-door VRF. Routes are injected to front-door VRF routing table by VPN process (reverse-route static in crypto map). To get these routes exported to customer VRFs, there is a matching network statement in BGP process of front-door VRF.
Well, this is also working fine if we do this only with CSR A. Reachablity is end-to-end. CSR B is routing to CSR because of working VRF-aware VPN. However, if we establish the second tunnel from CSR B, there is something strange happening.
Tunnel is established fine. Traffic received through this tunnel at CSR B is accepted and routed to destination. Traffic created at front-door VRF on CSR B is routed into its own VPN fine. However, traffic originating from a customer VRF which is reaching CSR B (traceroute proved this) is not routed into VPN tunnel, despite routing table of customer VPN is saying so. CSR A is running the same config, there is no problem. Only on CSR B.
I do not understand this. If we remove tunnel configuration from CSR A and only create tunnel on CSR B, it is still not working. I don't understand why, because I did a config compare and found no difference.
Has anyone any idea, whats going on?
How can I debug this problem?
CSR-A:
B 172.29.13.176/28 [20/0] via 1.1.119.182 (vrf-default), 3w4d
CSR-B:
with route (not working for this customer VRF)
B 172.29.13.176/28 [20/0] via 1.1.119.182 (vrf-default), 00:00:02
without route (working, because routed over transit to CSR-A)
B 172.29.13.176/28 [20/0] via 192.168.254.53 (vrf-default), 00:38:23
This problem is hard to describe, I would really appriciate discussing this with a TAC engineer in a WebEx. Is this possible?
Thank you.
Solved! Go to Solution.
05-24-2016 12:45 PM
Hello Tobias,
The issue you are describing is going to be outside our CSR platform expertise. It sounds like CSR is running fine and HA is also running fine, and now you are trying find a solution to a networking/VPN issue you are facing.
Our team will need to find a resource internally to address your issue, please allow us a day or so to get back to you with an answer
Regards
Tony
05-24-2016 12:45 PM
Hello Tobias,
The issue you are describing is going to be outside our CSR platform expertise. It sounds like CSR is running fine and HA is also running fine, and now you are trying find a solution to a networking/VPN issue you are facing.
Our team will need to find a resource internally to address your issue, please allow us a day or so to get back to you with an answer
Regards
Tony
07-11-2016 06:34 AM
If anyone is else is reading this:
The problem was solved by TAC.
Root cause:
The old crypto map feature of IOS (ported to IOS-XE) is not able to handle such VRF infrastructure. Cisco is working on the retirement of crypto maps and provide policy based VPN support at VTI (just put an crypto-ACL on a VTI). No roadmap with release dates yet.
Workaround:
Use VASI. Beside the bad name (vrf aware software instructure), it adds logical interface of type vasileft and vasiright which just do the same, you could do with cables in the old days: just bridge vrfs and create a transit network as overlay. Put the policy based vpn in its own vrf, create vasi links between this vrf and all others, add transit routes over these vasi links, and everything should work. One last thing I which took me another TAC session to figure out: crypto map MUST be applied to physical egress interface (should be fvrf), regardless of ivrf. So far, so normal. But you have to create a route in ivrf which routes the policy based vpn peer gw address to fvrf egress interface - otherwise ivrf does no even try to put the traffic into encryption domain.
Config stub:
vrf definition vrf-amcs
address-family ipv4
exit-address-family
crypto isakmp profile isakmp-vpn-amcs
vrf vrf-amcs
keyring keyring-vpn-vrf-default
match identity address 1.1.1.182 255.255.255.255 vrf-default
local-address GigabitEthernet1
crypto ipsec transform-set ipsec-prop-vpn-amcs esp-aes 256 esp-sha-hmac
mode tunnel
crypto map cmap-vpn-amcs 10 ipsec-isakmp
set peer 1.1.1.182
set security-association lifetime seconds 21600
set transform-set ipsec-prop-vpn-amcs
set pfs group2
set reverse-route distance 10
set isakmp-profile isakmp-vpn-amcs
match address vpn-to-amcs
reverse-route static
ip access-list extended vpn-to-amcs
permit ip 172.23.0.0 0.0.255.255 172.29.13.176 0.0.0.15
interface GigabitEthernet1
vrf forwarding vrf-default
ip address dhcp
negotiation auto
crypto map cmap-vpn-amcs
interface vasileft10
vrf forwarding vrf-10
ip address 192.168.255.10 255.255.255.254
interface vasiright10
vrf forwarding vrf-amcs
ip address 192.168.255.11 255.255.255.254
no keepalive
ip route vrf vrf-10 172.29.13.176 255.255.255.240 192.168.255.11
ip route vrf vrf-default 172.29.13.176 255.255.255.240 192.168.254.53 20
ip route vrf vrf-amcs 172.23.2.0 255.255.255.0 192.168.255.10
04-10-2017 02:12 PM
I am running into similar issue. You mentioned that "Use VASI. Beside the bad name (vrf aware software infrastructure)" Is this option enabled by default or do you have to enable it on Cisco 1000v?
Here are the relevant error logs. Cisco CSR 1000v with Crypto Map in VRF isn't matching the Crypto ACL to complete IPsec Phase 1.
*Apr 10 18:25:42.935: ISAKMP-ERROR: (1021):Expected csr-to-asa profile doesn't match, aborting exchange
*Apr 10 18:25:42.936: ISAKMP-ERROR: (1021):(1021): FSM action returned error: 2
*Apr 10 18:25:42.936: ISAKMP: (1021):Input = IKE_MESG_FROM_PEER, IKE_MM_EXCH
*Apr 10 18:25:42.936: ISAKMP: (1021):Old State = IKE_I_MM5 New State = IKE_I_MM6
I have the exact configuration that you have applied. I also referred VRF cheat sheet still no luck:
https://supportforums.cisco.com/document/51931/vrf-aware-ipsec-cheat-sheet
I have attached the running-config as well. I would really appreciate, if you could share more details from Cisco TAC on how to fix this issue.
04-11-2017 01:13 AM
There is no need to activate this VASI feature on IOS-XE (like on Nexus platforms with the feature command). Just make sure you have a recent version to avoid hitting bug CSCva27661 (or use ax-licence).
I've checked your running-config, there is not a single vasi interface yet. Please give it a try and create a pair (see my sample config above and don't forget the transit routes).
04-13-2017 02:17 PM
Hello Tobias,
Thanks for checking the configuration. I saw your vasi interface configuration and added the same to my configuration. Unfortunately, even after creating vasi interface and adding static routes, I am unable bring UP the VPN connection.
Remote 10.10.0.0/16
Local (CiscoCSRv) 10.8.0.0/16 --> I am configuring vasi interface here.
I am trying to reach remote network 10.10.0.0/16
Could you please check the config below and let me know if I missed something?
ip vrf default
description Internet VRF
rd 64512:100
!
ip vrf temp
rd 64512:101
!
ip vrf vasi-vrf
description vasi vrf
!
ip vrf vpn-ea76668b
rd 64512:1
route-target export 64512:0
route-target import 64512:0
!
ip vrf vpn-ec76668d
rd 64512:3
route-target export 64512:0
route-target import 64512:0
!
ip vrf vpn0
rd 64512:0
!
!
!
!crypto keyring internet-keyring
local-address GigabitEthernet1
pre-shared-key address <IP>
!
!
!
!
!
crypto isakmp policy 200
encr aes
authentication pre-share
group 2
lifetime 28800
crypto isakmp profile csr-to-asa
vrf vpn-ea76668b
keyring internet-keyring
match identity address <IP>
local-address GigabitEthernet1
!
crypto ipsec security-association replay window-size 1024
!
crypto ipsec transform-set ipsec-prop-vpn-aws esp-aes esp-sha-hmac
mode tunnel
crypto ipsec transform-set test esp-aes esp-sha-hmac
mode tunnel
crypto ipsec df-bit clear
!
!
crypto ipsec profile ipsec-vpn-aws
set transform-set ipsec-prop-vpn-aws
set pfs group2
!
!
crypto map csr-to-asa 10 ipsec-isakmp
set peer <IP>
set transform-set test
set pfs group2
set isakmp-profile csr-to-asa
match address csr-to-asa
reverse-route static
!
interface Loopback1
ip vrf forwarding vpn-ea76668b
ip address 10.8.0.1 255.255.255.255
!
ip route 10.8.0.0 255.255.0.0 192.168.255.11
ip route vrf vpn-ea76668b 10.10.0.0 255.255.0.0 192.168.255.10
ip access-list extended csr-to-asa
permit ip 10.8.0.0 0.0.255.255 10.10.0.0 0.0.255.255 log
permit ip 10.7.0.0 0.0.255.255 10.10.0.0 0.0.255.255 log
!
!
!
interface GigabitEthernet1
ip address dhcp
negotiation auto
crypto map csr-to-asa
!
interface GigabitEthernet2
ip vrf forwarding temp
ip address dhcp
negotiation auto
!
interface vasileft1
ip address 192.168.255.10 255.255.255.254
no keepalive
!
interface vasiright1
ip vrf forwarding vpn-ea76668b
ip address 192.168.255.11 255.255.255.254
no keepalive
!
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide