ASA-AWS - Problem Routing between VPCs

[rubenr]

Note – IP subnets changed for privacy.

We have two VPCs:

FW VPC, network 1.1.0.0/22

• This hosts a Cisco ASAv virtual firewall with three interfaces
  • 1.0.22/24- OUTSIDE interface connected to remote VPN via internet
  • 1.2.198/24- INSIDE interface used to communicate within the AWS infrastructure 
  • 1.1.162/24-DMZ - for future use

PRD VPC, network 1.1.4.0/22

• This is currently hosting one server instance on 1.1.4.153

And a Remote Office subet, network 10.6.1.1/29

• This is currently hosting one server instance on 10.6.1.38

A gateway and route has been configured between the FW VPC and the PRD VPC, and the firewall on 1.1.2.198 can communicate with the virtual server instance on 1.1.4.153 OK.

We have configured a IPsec VPN between the Cisco ASAv and a Cisco ASA physical at a remote location (not using the AWS VPN gateway), the remote network is 10.6.1.1/29.

The VPN is up and passing traffic, but the issue we're experiencing is the routing of traffic destined for the remote network at the other end of the VPN via the Cisco ASAv within the AWS environment.

We can see test traffic from remote end host 10.6.1.38 going over the VPN to the ASAv in AWS, and this device then passes the traffic onto the next hop (1.1.2.1), as it should.  At this point no response is received.  It's not known at this point if the packet does not react the destination instance 1.1.4.153 or if just the reply does not route back correctly.  A packet capture on the ASAv inside interface shows this behaviour:

   1: 20:56:41.596236       10.6.1.38 > 1.1.4.153: icmp: echo request 

   2: 20:56:46.588058       10.6.1.38 > 1.1.4.153: icmp: echo request 

   3: 20:56:51.595702       10.6.1.38 > 1.1.4.153: icmp: echo request 

   4: 20:56:56.587860       10.6.1.38 > 1.1.4.153: icmp: echo request 

   5: 20:57:01.595214       10.6.1.38 > 1.1.4.153: icmp: echo request

If traffic is generated within AWS on host 1.1.4.153 destination 10.6.1.38 it does not arrive at the Cisco ASAv.

The route the traffic should take is:

Step 1               Step 2       Step 3       Step 4

1.1.4.153 --> 1.1.4.1 --> 1.1.2.1 --> 1.1.2.198 --> VPN

Generating host AWS router AWS router Cisco ASAv

At the moment either step 2 or step 3 is failing as the traffic does not reach step 4 so cannot be tunnelled down the VPN.

Can you assist with correcting the routing so that traffic from 1.1.4.0/22 destined to 1.6.1.32/29 is routed via the Cisco ASAv on address 1.1.2.198

[whitey]

Have you disabled the AWS source/destination check on the interfaces?

[rubenr · Accepted solution]

Hi Whitey.

Yes, Source/Destination has been disabled.

We found that our scenario isn't supported in AWS as we are trying to
provide a transitive peering setup:

https://docs.aws.amazon.com/AmazonVPC/latest/PeeringGuide/invalid-peering-configurations.html#transitive-peering

Thanks,

Ruben

[whitey]

If you create a tunnel between the edge VGW and you central ASA you can work around the lack of transitive routing. The AWS infrastructure just sees the tunnel IPs and so your not bound to the transitive restrictions.

Alternately you could deploy another ASA in the edge and tunnel from that over a peering connection.