Solved: Cisco Catalyst SD-WAN connecting to AWS via IPSEC to VPN0

brian.jones · ‎01-03-2024

I have a small SD-WAN setup for a company with two "DC" locations using SDCI/Cloud On-Ramp to my main AWS subscription and two remote sites that connects via the SD-WAN to the SDCI to access the workload. I have two test AWS subscription that are not part of my main AWS subscription that I want to connect to my SD-WAN but I don't want deploy edges to those subscriptions as it is for short term projects, but I need to access those workloads from my two remote sites. I have a Site-to-Site VPN currently between the test AWS subscriptions and my main remote site to facilitate the connectivity. I want to move as much as I can to SD-WAN however, I cannot deploy a "service" side VPN for this.

I want to connect the test AWS subscription to the "DC" locations" via IPSec and have the "DC" edges route the traffic to the remote site or the main AWS subscription as required. I found little documentation on how to use IPSec on the transport side, but I haven't seen anything that says I cannot.

Can someone direct me on if this is possible and if so, the docs on how to do it?

Kanan Huseynli · ‎01-09-2024

Reason for complicated pass is:

From Cisco SD-WAN 19.2.x release onwards, the pre-shared key needs to be at least 16 bytes in length. The IPsec tunnel establishment fails if the key size is less than 16 characters when the router is upgraded to version 19.2.

On SD-WAN devices, there is security check, if PSK length is <16 then it fails. I remember one related topic in community as well (long ago somebody had similar issue), but I just forgot and reminded when reading security config guide:

https://www.cisco.com/c/en/us/td/docs/routers/sdwan/configuration/security/vedge-20-x/security-book/config-sec-param.html#c_Configuring_IKE_Enabled_IPsec_Tunnels_12216.xml

HTH,
Please rate and mark as an accepted solution if you have found any of the information provided useful.

View solution in original post

Kanan Huseynli · ‎01-09-2024

For bidirectional flow, put ipsec interface into VPN 20.

Remove all ipsec configuration from VPN0, add to VPN 20.Also, add simple static route (not ipsec route) with remote tunnel IP being next-hop.

Check below commands, adjust if needed (mostly, copy pasted from your .txt):

vpn 20
no  ip ipsec-route 172.16.4.0/24 vpn 0 interface ipsec1 
vpn 0 
no interface ipsec1


vpn 20
interface ipsec1 
ip address 169.254.1.1/30 
tunnel-source-interface ge0/0 
tunnel-destination 100.0.50.2 
ike
version 2 
rekey 14400 
cipher-suite aes128-cbc-sha1 
group 2 
authentication-type 
pre-shared-key 
pre-shared-secret $8$EWhg0Phf6OeE4qWWhhU8Sa5/SQO2io3gsaY7WB/a5FI= 
! 
! 
! 
ipsec 
rekey 3600 
replay-window 512 
cipher-suite aes256-cbc-sha1 
perfect-forward-secrecy group-2 
! 
mtu 1400 
no shutdown 
vpn 20
ip route 172.16.4.0/24 169.254.1.2

Reason it does work when you ping from vedge side, but does not work when you ping from remote side is nat-filter. When you ping from vedge side, for return traffic nat-filter is created and traffic is allowed even though there is no VPN20 route information in VPN0 (where ipsec interface in your configuration case). When you ping from IOS side, traffic "falls" in VPN0 and there is no route to 172.16.2.0/24.

Note: I checked with route-leaking, but still no luck, nat drops traffic. However, above method puts ipsec in VPN20 (think it like interface tunnel in VRF but underlay -transport interface is in global from IOS XE configuration).

HTH,
Please rate and mark as an accepted solution if you have found any of the information provided useful.

View solution in original post

dheeraj dubey · ‎01-04-2024

.

iamdheerajdubey · ‎01-04-2024

hmm, interesting requirement!

We can use IPsec on vpn0 while deploying an SIG gateway.

My hunch would be to see if you can deceive the edge (the DC router) thinking it is a forming IPsec with SIG gateway but in reality, we are tunneling it to AWS.

brian.jones · ‎01-04-2024

That is an interesting idea. I would assume I have to setup a generic SIG and set up the Service Routes to point it to the correct "SIG", which in this case would be the other two subscriptions. Service routes are "static" routes, so if I have two SDCI routers, how would I set one to be "active". I guess it would be similar to setting up an "active" and "backup" SIG in the configuration.

brian.jones · ‎01-04-2024

Looking into this further in my lab, it appears that you can only have 1 SIG/Service. I can have an active and backup tunnel but it is to the same service. In my case, I would need to have 2 SIGs as each AWS subscription is separate. I am going to try with the 1 SIG to see if it works in the lab, but I don't know if this will fully resolve my design need.

iamdheerajdubey · ‎01-04-2024

At the moment I can only think of this as the workaround..do let me know if you can successfully form the tunnel...We would need another separate edge to form the other AWS tunnel and then we can play with ad value to make one primary and backup

do rate the post if it was helpful.

MHM Cisco World · ‎01-04-2024

https://blogs.cisco.com/networking/cisco-sd-wan-integration-with-aws-transit-gateway-connect-raises-the-bar-for-cloud-performance-and-scale

check this
MHM

brian.jones · ‎01-04-2024

it would not be a "integrated" solution. I can only use the native VPN toolset (i.e. IPsec) to connect the subscriptions to my 8000v that houses my SDCI.

MHM Cisco World · ‎01-04-2024

https://www.cisco.com/c/en/us/support/docs/routers/sd-wan/214521-site-to-site-lan-to-lan-ipsec-between.html

check this
MHM

brian.jones · ‎01-04-2024

So translating this to vManage, I have to build an VPN interface template and add an ipsec route to the service vpn to point to the VPN interface.

brian.jones · ‎01-05-2024

The link references IKEv but I am using IKEv2, which is not working. Here are the configurations:

Remote (Non SD-WAN Router)

crypto ikev2 proposal IKE-PROPOSAL-1
encryption aes-cbc-128
integrity sha1
group 2
crypto ikev2 policy POLICY1
proposal IKE-PROPOSAL-1
crypto ikev2 keyring KEYRING-1
peer vEdge2-1
address 100.0.2.2
pre-shared-key Testing
!
crypto ikev2 profile IKE-PROFILE-1
match address local 100.0.50.2
match identity remote address 100.0.2.2 255.255.255.255
authentication remote pre-share
authentication local pre-share
keyring local KEYRING-1
dpd 10 3 periodic
no config-exchange request
crypto ipsec transform-set T1 esp-aes 256 esp-sha256-hmac
mode tunnel
crypto ipsec profile P1
set transform-set T1
set pfs group2
set ikev2-profile IKE-PROFILE-1

interface Tunnel0
ip address 169.254.1.2 255.255.255.252
ip tcp adjust-mss 1400
tunnel source 100.0.50.2
tunnel mode ipsec ipv4
tunnel destination 100.0.2.2
tunnel protection ipsec profile P1
end

Vedge

interface ipsec1
ip address 169.254.1.1/30
tunnel-source 100.0.2.2
tunnel-destination 100.0.50.2
ike
version 2
rekey 14400
cipher-suite aes128-cbc-sha1
group 2
authentication-type
pre-shared-key
pre-shared-secret $8$EWhg0Phf6OeE4qWWhhU8Sa5/SQO2io3gsaY7WB/a5FI=
local-id 100.0.2.2
remote-id 100.0.50.2
!
!
ipsec
rekey 3600
replay-window 512
cipher-suite aes256-cbc-sha1
perfect-forward-secrecy group-2
!
mtu 1400
no shutdown
!

ip route 172.16.4.0/24 169.254.1.2

vpn 20
interface ge0/2
ip address 172.16.2.1/24
no shutdown
!
ip ipsec-route 172.16.4.0/24 vpn 0 interface ipsec1
route-import static route-policy Site4-to-Site2
!

MHM Cisco World · ‎01-05-2024

Let me check again

MHM

MHM Cisco World · ‎01-05-2024

let make IKEv1 work first then we go to IKEv2
ip route 172.16.4.0/24 169.254.1.2 <<- remove this
use tunnel-source-interface

MHM

brian.jones · ‎01-05-2024

In the vManage template for VPN0 under static routes, you don't have an option to specify the tunnel-source-interface. You have "next-hop", "null 0", VPN, DHCP. Also, I can only put in a static route. If I try to put in a IPSEC route, vManage comes back with "IPSEC Route cannot be configured for VPN0".

brian.jones · ‎01-05-2024

Here is the debug crypto isakmp from the remote router.

*Jan 5 20:29:20.198: ISAKMP: (0):Input = IKE_MESG_FROM_PEER, IKE_MM_EXCH
*Jan 5 20:29:20.198: ISAKMP: (0):Old State = IKE_READY New State = IKE_R_MM1

*Jan 5 20:29:20.199: ISAKMP: (0):processing SA payload. message ID = 0
*Jan 5 20:29:20.199: ISAKMP: (0):processing vendor id payload
*Jan 5 20:29:20.200: ISAKMP: (0):vendor ID seems Unity/DPD but major 215 mismatch
*Jan 5 20:29:20.200: ISAKMP: (0):vendor ID is XAUTH
*Jan 5 20:29:20.200: ISAKMP: (0):processing vendor id payload
*Jan 5 20:29:20.200: ISAKMP: (0):vendor ID is DPD
*Jan 5 20:29:20.200: ISAKMP: (0):processing vendor id payload
*Jan 5 20:29:20.200: ISAKMP: (0):processing IKE frag vendor id payload
*Jan 5 20:29:20.200: ISAKMP: (0):Support for IKE Fragmentation not enabled
*Jan 5 20:29:20.200: ISAKMP: (0):processing vendor id payload
*Jan 5 20:29:20.200: ISAKMP: (0):vendor ID seems Unity/DPD but major 69 mismatch
*Jan 5 20:29:20.200: ISAKMP: (0):vendor ID is NAT-T RFC 3947
*Jan 5 20:29:20.200: ISAKMP: (0):processing vendor id payload
*Jan 5 20:29:20.200: ISAKMP: (0):vendor ID seems Unity/DPD but major 123 mismatch
*Jan 5 20:29:20.200: ISAKMP: (0):vendor ID is NAT-T v2
*Jan 5 20:29:20.201: ISAKMP: (0):found peer pre-shared key matching 100.0.2.2
*Jan 5 20:29:20.201: ISAKMP: (0):local preshared key found
*Jan 5 20:29:20.201: ISAKMP: (0):Scanning profiles for xauth ... IKE-PROFILE-2
*Jan 5 20:29:20.201: ISAKMP: (0):Checking ISAKMP transform 1 against priority 10 policy
*Jan 5 20:29:20.201: ISAKMP: (0): encryption AES-CBC
*Jan 5 20:29:20.201: ISAKMP: (0): keylength of 128
*Jan 5 20:29:20.202: ISAKMP: (0): hash SHA
*Jan 5 20:29:20.202: ISAKMP: (0): default group 2
*Jan 5 20:29:20.202: ISAKMP: (0): auth pre-share
*Jan 5 20:29:20.202: ISAKMP: (0): life type in seconds
*Jan 5 20:29:20.202: ISAKMP: (0): life duration (basic) of 15840
*Jan 5 20:29:20.202: ISAKMP: (0):atts are acceptable. Next payload is 0
*Jan 5 20:29:20.202: ISAKMP: (0):Acceptable atts:actual life: 86400
*Jan 5 20:29:20.202: ISAKMP: (0):Acceptable atts:life: 0
*Jan 5 20:29:20.202: ISAKMP: (0):Basic life_in_seconds:15840
*Jan 5 20:29:20.203: ISAKMP: (0):Returning Actual lifetime: 15840
*Jan 5 20:29:20.203: ISAKMP: (0):Started lifetime timer: 15840.

*Jan 5 20:29:20.206: ISAKMP: (0):processing vendor id payload
*Jan 5 20:29:20.226: ISAKMP: (0):vendor ID seems Unity/DPD but major 215 mismatch
*Jan 5 20:29:20.227: ISAKMP: (0):vendor ID is XAUTH
*Jan 5 20:29:20.227: ISAKMP: (0):processing vendor id payload
*Jan 5 20:29:20.227: ISAKMP: (0):vendor ID is DPD
*Jan 5 20:29:20.227: ISAKMP: (0):processing vendor id payload
*Jan 5 20:29:20.227: ISAKMP: (0):processing IKE frag vendor id payload
*Jan 5 20:29:20.227: ISAKMP: (0):Support for IKE Fragmentation not enabled
*Jan 5 20:29:20.227: ISAKMP: (0):processing vendor id payload
*Jan 5 20:29:20.227: ISAKMP: (0):vendor ID seems Unity/DPD but major 69 mismatch
*Jan 5 20:29:20.227: ISAKMP: (0):vendor ID is NAT-T RFC 3947
*Jan 5 20:29:20.227: ISAKMP: (0):processing vendor id payload
*Jan 5 20:29:20.227: ISAKMP: (0):vendor ID seems Unity/DPD but major 123 mismatch
*Jan 5 20:29:20.227: ISAKMP: (0):vendor ID is NAT-T v2
*Jan 5 20:29:20.227: ISAKMP: (0):Input = IKE_MESG_INTERNAL, IKE_PROCESS_MAIN_MODE
*Jan 5 20:29:20.227: ISAKMP: (0):Old State = IKE_R_MM1 New State = IKE_R_MM1

*Jan 5 20:29:20.228: ISAKMP: (0):constructed NAT-T vendor-rfc3947 ID
*Jan 5 20:29:20.228: ISAKMP-PAK: (0):sending packet to 100.0.2.2 my_port 500 peer_port 500 (R) MM_SA_SETUP
*Jan 5 20:29:20.228: ISAKMP: (0):Sending an IKE IPv4 Packet.
*Jan 5 20:29:20.229: ISAKMP: (0):Input = IKE_MESG_INTERNAL, IKE_PROCESS_COMPLETE
*Jan 5 20:29:20.229: ISAKMP: (0):Old State = IKE_R_MM1 New State = IKE_R_MM2

*Jan 5 20:29:20.281: ISAKMP-PAK: (0):received packet from 100.0.2.2 dport 500 sport 500 Global (R) MM_SA_SETUP
*Jan 5 20:29:20.281: ISAKMP: (0):Input = IKE_MESG_FROM_PEER, IKE_MM_EXCH
*Jan 5 20:29:20.281: ISAKMP: (0):Old State = IKE_R_MM2 New State = IKE_R_MM3

*Jan 5 20:29:20.284: ISAKMP: (0):processing KE payload. message ID = 0
*Jan 5 20:29:20.286: ISAKMP: (0):processing NONCE payload. message ID = 0
*Jan 5 20:29:20.287: ISAKMP: (0):found peer pre-shared key matching 100.0.2.2
*Jan 5 20:29:20.287: ISAKMP: (1004):received payload type 20
*Jan 5 20:29:20.287: ISAKMP: (1004):His hash no match - this node outside NAT
*Jan 5 20:29:20.287: ISAKMP: (1004):received payload type 20
*Jan 5 20:29:20.287: ISAKMP: (1004):His hash no match - this node outside NAT
*Jan 5 20:29:20.287: ISAKMP: (1004):Input = IKE_MESG_INTERNAL, IKE_PROCESS_MAIN_MODE
*Jan 5 20:29:20.287: ISAKMP: (1004):Old State = IKE_R_MM3 New State = IKE_R_MM3

*Jan 5 20:29:20.287: ISAKMP-PAK: (1004):sending packet to 100.0.2.2 my_port 500 peer_port 500 (R) MM_KEY_EXCH
*Jan 5 20:29:20.287: ISAKMP: (1004):Sending an IKE IPv4 Packet.
*Jan 5 20:29:20.288: ISAKMP: (1004):Input = IKE_MESG_INTERNAL, IKE_PROCESS_COMPLETE
*Jan 5 20:29:20.288: ISAKMP: (1004):Old State = IKE_R_MM3 New State = IKE_R_MM4

*Jan 5 20:29:20.288: ISAKMP: (1002):retransmitting phase 1 MM_KEY_EXCH...
*Jan 5 20:29:20.288: ISAKMP: (1002):peer does not do paranoid keepalives.
*Jan 5 20:29:20.289: ISAKMP-ERROR: (1002):deleting SA reason "Death by retransmission P1" state (R) MM_KEY_EXCH (peer 100.0.2.2)
*Jan 5 20:29:20.289: ISAKMP-ERROR: (1002):deleting SA reason "Death by retransmission P1" state (R) MM_KEY_EXCH (peer 100.0.2.2)
*Jan 5 20:29:20.289: ISAKMP: (1002):Deleting the unauthenticated sa
*Jan 5 20:29:20.289: ISAKMP: (1002):Unlocking peer struct 0x80007F51140A88A8 for isadb_mark_sa_deleted(), count 0
*Jan 5 20:29:20.289: ISAKMP: (1002):Deleting the peer struct for unauthenticated sa
*Jan 5 20:29:20.289: ISAKMP: (0):Deleting peer node by peer_reap for 100.0.2.2: 80007F51140A88A8
*Jan 5 20:29:20.290: ISAKMP: (1002):Input = IKE_MESG_INTERNAL, IKE_PHASE1_DEL
*Jan 5 20:29:20.290: ISAKMP: (1002):Old State = IKE_R_MM4 New State = IKE_DEST_SA

*Jan 5 20:29:20.319: ISAKMP-PAK: (1004):received packet from 100.0.2.2 dport 500 sport 500 Global (R) MM_KEY_EXCH
*Jan 5 20:29:20.319: %CRYPTO-6-IKMP_NOT_ENCRYPTED: IKE packet from 100.0.2.2 was not encrypted and it should've been.
*Jan 5 20:29:30.127: ISAKMP: (1003):retransmitting phase 1 MM_KEY_EXCH...
*Jan 5 20:29:30.225: ISAKMP: (1003):: incrementing error counter on sa, attempt 4 of 5: retransmit phase 1
*Jan 5 20:29:30.227: ISAKMP: (1003):retransmitting phase 1 MM_KEY_EXCH
*Jan 5 20:29:30.228: ISAKMP-PAK: (1003):sending packet to 100.0.2.2 my_port 500 peer_port 500 (R) MM_KEY_EXCH
*Jan 5 20:29:30.228: ISAKMP: (1003):Sending an IKE IPv4 Packet.
*Jan 5 20:29:30.308: ISAKMP: (1004):retransmitting phase 1 MM_KEY_EXCH...
*Jan 5 20:29:30.308: ISAKMP: (1004):: incrementing error counter on sa, attempt 1 of 5: retransmit phase 1
*Jan 5 20:29:30.308: ISAKMP: (1004):retransmitting phase 1 MM_KEY_EXCH
*Jan 5 20:29:30.308: ISAKMP-PAK: (1004):sending packet to 100.0.2.2 my_port 500 peer_port 500 (R) MM_KEY_EXCH
*Jan 5 20:29:30.313: ISAKMP: (1004):Sending an IKE IPv4 Packet.