ASAv in AWS cannot route on outside interface

[eebarker]

Hi

We are setting up an ASAv in AWS and have management access to it but no matter what we try on the ASAv and AWS config we cannot get traffic to route successfully from the ASAv outside interface to the Internet .

The setup so far is that we successfully created the instance, allocated a day0 config and have management (ASDM and ssh) access to the ASAv via an Elastic IP allocated to the management interface, with the management interface set up as -

interface Management0/0
nameif management
security-level 100
ip address dhcp setroute

We have added inside and outside interfaces, allocated them addresses in the private and public subnets in AWS and at the moment allowed all traffic on the outside interface.  another Elastic IP is configured on the outside interface. The routing table in AWS has been set on the outside subnet that the ASAv's outside interface is in and the ARP table shows a mac address for the AWS gateway at 10.14.8.1 when Our outside interface is 10.14.8.204.  With packet captures and ASDM logging we can see traffic arrive on the outside interface and see the ASAv responding and sending traffic back, but that traffic never reaches the internet destination.

The outside interface is configured as follows with the route -

interface GigabitEthernet0/1

 description *** Internet service – AWS Public subnet and Elastic IP ***

 nameif outside

 security-level 0

 ip address 10.14.8.204 255.255.255.0

 no shut

route outside 0.0.0.0 0.0.0.0 10.14.8.1 1

as mentioned we have taken packet traces and can see internet traffic directed at the Elastic IP reaching the ASAv's outside interface and being responded to, but that traffic never gets back to the internet destination.

Would really appreciate any help as am completely stuck on this at the moment,

Euan

[Flavio Miranda]

Hi @eebarker

If you outside ip address is 10.14.8.204 then you have a NAT somewhere?

-If I helped you somehow, please, rate it as useful.-

[eebarker]

The outside IP of 10.14.8.204 has an Elastic IP which is allocated by AWS of 18.194.x.x (not revealing for security) which is the public IP we send traffic to and from.  The Public Elastic IP is not visible on the firewall as the nat happens in AWS, not on the firewall.

[Flavio Miranda]

And what about DNS? 

[eebarker]

DNS will use our Data centre based DNS servers once we have a VPN running from the outside interface to the on-prem data centre VPN.  DNS isn't the issue at the moment.  from packet captures we've seen VPN traffic initiated from the on-prem data centre VPN IP reach the ASAv's outside interface and be responded to by the ASAv but as mentioned that return packet doesn't get back to the VPN router in our data centre.  we can also try pinging 8.8.8.8 and see the packet leave the ASAv but again no response.

we've disabled all security groups on the ASAv interfaces in AWS config, the routing table allocated looks correct.

Thanks for your questions,

[Flavio Miranda]

Sorry for making questions but I'm trying to understand this setup and then try to help. Are you controlling security at the AWS side or are you allowing everything and controlling on the ASAv side? 

Honestly, this sounds to me that you may have some issue on the AWS side and not with ASAv side as the ASAv side as you describe that you see traffic going back and forth from ASAv.

-If I helped you somehow, please, rate it as useful.-

[eebarker]

we  set all interfaces in the aws setup into a security group that allows all traffic as we do want the ASAv to control the traffic, not the AWS environment,.  

I tend to agree the issue is with AWS and thought maybe the issue was having two elastic IP's one on management interface and one on outside.  our latest step has been to make management a standard interface so we only have one Elastic IP on the ASAv.  We do now have vpn working from inside interface through management interface to our data centre, and inside nodes in AWS can reach the internet via the ASAv.

However, this isn't right. I would have thought we must be able to have the outside interface having an Elastic IP and working to route actual traffic in/out as well as management interface just handling management traffic with a separate VRF and with an Elastic IP so we can manage it over the internet directly.

[ak47mp]

I am taking a shot in the dark, and am having the same problems as you are, but have you created a tunneled route?  ip route inside 0.0.0.0 0.0.0.0 [aws gw] tunneled 

If you can get internet routing working I definitely want to compare notes. 

[eebarker]

Hi

It's not tunneled, the route command is just

ip route outside 0.0.0.0 0.0.0.0 10.14.8.1 1

where 10.14.8.1 is the AWS gateway in the public subnet of the VPC we have this instance in.

Is there a reason it should be tunneled ?

[ak47mp]

The main default route allows the return path to clients. For example when you create the tunnel return traffic should go back out the outside interface gw.

But when users are tunneled, all their traffic is going to leave the Inside interface. You need to route the 0/0 traffic to the inside gateway. By adding the tunneled keyword that route will only apply to packets that are arriving over the tunnel.

Give it a try, worth a shot. Also, might be good for you to post things like your inside and outside interface configs along with what you have configured for ip pool.

Routing the traffic for the ip pool subnet in the aws route tables is another important piece here.

[eebarker]

All

Thanks for the questions and advice. This is now working with an Elastic IP allocated to Management and Outside interfaces. I ended up creating another instance from scratch and changing the process of instance creation so that the management interface (eth0 on AWS instance) is in the public subnet of the VPC, the outside interface (gi 0/0 on ASA, eth1 on instance) is in a separate 'transit' AWS private subnet with an Elastic IP associated, and then the inside interface (gi 0/1, eth2 on instance) is in the standard private subnet for the VPC.

Nothing about the old config seems wrong in comparison to what we now have on the ASAv, and setup we have/had on the AWS network subnets, route tables, security groups, interface security etc, is the same as we had before.  It just seems that AWS and it's semi-hidden networking processes /  requirements need the setup process to be slightly different to get what we needed to work.

If useful I can provide a setup steps guide I've been using for this once names/network IP's are edited.

Cheers,

Euan

[JXBJXB]

Old thread, but @eebarker if you have a step-by-step I would love to see it. We are running into the exact same issue. Thanks!

[eebarker]

Don't like to put this straight onto forums as it was for a customer project which I've removed names etc. of but the subnets , VPC/EC2 id's etc. are still visible in screen shots so don't want to risk sticking onto a forum with open access, despite the very small risk. 

Message me direct if you want a copy and I can email it across.