ASA Statically Routed Subnet Confusion

[bitflowcisco]

Hello,

This question is in the context of servers sitting in a colocation environment behind an ASA5510 with security plus license.

Our colo provider is going to be statically routing a /28 public subnet to our ASA5510 (say 1.1.1.0/28).  We will also be getting a single IP (say 2.2.2.2/30)  on a small router-to-router subnet (2.2.2.0/30) to which the 1.1.1.0/28 subnet will be statically routed to our ASA5510 from our colo provider.

I will obviously set the outside interface of the ASA to be 2.2.2.2/30 so that the colo provider can route the 1.1.1.0/28 subnet to it.  I will also set a default route to 2.2.2.1 which is the IP of our colo providers gateway (and the router that will be statically routing the 1.1.1.0/28 subnet to us).

We have various servers in the same rack as the ASA (connected via a 3750G switch).  Some of these servers need to be exposed to the internet (web, email, etc servers) and some do not (database servers).

I'm considering 2 different ways of designing the network but I have questions about both and not sure which way to go:

1) Scenario #1: Using NAT and private IP's for all servers.

In this scenario where/how do I assign the internal network (say 10.1.1.0/24) and the public routed subnet (1.1.1.0/28)?  I assume the internal 10.1.1.0/24 is an inside network assigned to the interface connected to the 3750G (to which all the servers connect).  However, where do I assign the public routed subnet (1.1.1.0/28) since it is somewhat "nebulous" in that it has to reside somewhere on the ASA so that it can then NAT to the internal (10.1.1.0/24) IP's.  Also, is it considered an outside or inside network - and on which interface? My confusion is that If its added to the outside interface then won't that conflict with the 2.2.2.2 IP to which the colo provider is routing our 1.1.1.0/28 subnet to? And if its on the inside interface connected to the 3750G then wont that conflict with the 10.1.1.0/24 private IP range of the servers? I'm missing something here.... (please help:)

2) Scenario #2: Using public IP's for all servers

This scenario seems more straightforward to me: I would want to assign IP's from the statically routed subnet (1.1.1.0/28) to my servers so that range would be configured as an inside network on the interface connected to the 3750G (to which all the servers connect).  This would be configured on a specific VLAN (say vlan 50).  I would then have another VLAN (say VLAN 100) on the 3750G that has a private IP range (say 10.1.1.0/24) so that non-public servers (database, etc) would reside on there. All public servers that need access to private servers would have a NIC on both VLANS (50 + 100).  My question is is this the correct way of approaching this?  I also like this because I dont have to worry about NAT and the ASA can act as a router/firewall and things are clear in terms of whats happening.

Ultimately I'm not sure which is the best way to go in terms of having all servers on a private IP range and just NAT to them (as per scenario 1), or implement scenario 2 where servers have two interfaces.

Any advice or suggestions would be great. The main thing thats bugging me from scenario 1 is I'm not sure where/how to assign the statically routed subnet (1.1.1.0/28) on the ASA? (inside? outside? which interface?).

Thanks very much

Cheers

[Jouni Forss]

Hi,

Scenario 1

You would configure the setup in the following way (will list point that you already mentioned also)

• Configure the link network/subnet between you and the ISP
• ISP would route the additional public subnet towards the "outside" IP address of the ASA
• You will simply configure Static NAT using those public IP addresses  
  • The Public IP address of the Static NAT is naturally considered to be located on the "outside" interface of the ASA.

So  even though it might seem wierd this is how you do it. There is really  no other configurations related to the additional public subnet. Its  simply used in the NAT configurations and will work as usual.

Now  if the situation were so that the ISP didnt actually route the public  subnet towards your ASA "outside" interface and actually used it as a  "secondary" IP address on their router inteface facing the ASA then you  might run into some problems depending on your software.

In some ASA softwares (Think it was around 8.4(2) to 8.4(3)) there was problems with ARP and the solution was to

• Change  the setup to the setup I mentioned at the very top which is to use only  the link network between the ASA and ISP router and route the  additional subnet towards the ASA "outside" interface
• Other option was to update to 8.4(5) if I remember correctly and use the command "arp permit-nonconnected".
• There might have been some other configuration to get around the problem but wasnt really that convinient

Scenario 2

You would configure the setup in the following way

• Configure the link network/subnet between you and the ISP
• ISP would route the additional public subnet towards the "outside" IP address of the ASA
• Configure a interface on the ASA (physical or logical) with the public  subnet and use the public IP addresses directly on the servers that need  them

With regards to the other Vlan for the "non  public" servers I would avoid using a server connected to multiple  networks. I would let the different networks communicate through the  ASA. Though I have to admit the server/IT side isnt that familiar to me  as I would like it to be. But I personally have bad expirience from  situations where servers are connected to 2 different networks. You  might run into routing problems on the actual server itself if its own  routing table isnt configured correctly. You might end up in a situation  where traffic comes to the server from one interface but gets forwarded  out of the other and ASA will block the connection.

Pros/Cons

Scenario 1

• Pros  
  • Public IP addresses can be used for any network/host behind you ASA. Not just the server DMZ.
• Cons   
  • You  might run into problems if there is need for the LAN users to reach the  server by using the public IP address (and not use private IP  directly). In this case the solution is to use public (or local) DNS for  the servers and enable the DNS replies rewrite with the "dns" parameter  in the Static NAT command.
  • Above situation (connecting from LAN  to Static NAT public IP) is a problem because ASA doesnt allow  connecting to the NAT IP address facing "outside" from behind some other  interface on the ASA. Atleast without special NAT setups that again  would needlesly complicated your ASA configurations.

Scenario 2

• Pros   
  • Public  IP addresses are directly on the servers and therefore there are no  problems like described above with connectivity from the LAN to the  servers.
• Cons   
  • IP addresses can naturally only be used on the DMZ segment/interface of the ASA
  • Using  the subnet in your network takes atleast 1 IP address from that subnet  to be used as the gateway IP address for that network while using the  network in the Scenario 1 way would not consume that IP address.

There  are some of my thoughts. I will add if something else comes to minds  but the above points are the most common things I have run into with  similiar cases. There might ofcourse be other points that could be made that I havent thought about.

Hope the information was helpfull If so please do rate the answer.

Naturally ask more if needed

- Jouni

[James Leinweber]

I'm with Jouni on recommending against the servers-with-dual-NICs as a general rule.  What I do in a fairly similar situation is scenario 3:

1) configure the ASA with outside, dmz, and pci interfaces

2) use the public, routable IP's on the DMZ servers which face outward for DNS, SMTP, HTTP, etc.

3) use private rfc-1918 address on the inside PCI servers hosting applications and databases

The ASA itself serves as the router between the DMZ and PCI servers.  If the ASA doesn't have enough throughput for that, then you either need a bigger ASA or you are back to the multiple interfaces on the servers scenario.

In scenario 3 you do potentially need 3 kinds of NAT rules:

a) identity NAT between the DMZ and PCI subnets

b) PAT for the PCI servers going out the general internet for patching

c) if you don't have a dual-interface management staging host on-link with the PCI servers, or an IPsec tunnel to the ASA, then you might need some NAT rules to allow limited inbound connections to the PCI servers for maintenance.

A new option, scenario 4, is to dual-stack the DMZ servers with IPv4 and IPv6, which you should be doing anyway, put the PCI servers on v6-only, and just have public addresses all around.

-- Jim Leinweber, WI State Lab of Hygiene

[bitflowcisco]

Hi James,

Thakns for your reply and sorry for the delay in response, I was out of town.  A few followup questions:

1) Why are you also against multi homed servers? Please see my response to Joni in terms of how I'd setup the routing tables - it seems simple to me?

2) Some of the DMZ servers would need access to prviate servers (eg: web server needs to talk to DB server).  How would this be handled?  Also, for nightly backups each server would be pushing a full gigabit of traffic to a backup server/SAN on the private network and this would crush the ASA.  If you don't like the idea of multi-homed srevers (that would talk on a private network and not bother the ASA), then the only other option I see is Scenario #1 where I have an inside and outside network with the routed public subnet statically NATed to inside servers.  Thoughts?

thanks again

[James Leinweber]

Wearing my security hat, I dislike dual-homed servers in general, because they potentially provide alternate paths to evade firewall rules.   Wearing my network engineering hat, I agree that running backups through the firewalls tends to be a bottleneck, so my tape servers are dual-homed, just like yours.  And my app servers tend to be on the same subnet as the DB servers, to keep that traffic from going through the firewall too.  PCI fanatics will cringe at that, and recommend buying more and higher performance firewall hardware.   I'd need a bigger budget.

Using public DMZ and private PCI addresses without dual-homing, since both subnets are directly attached to the ASA, you simply set up identity NAT between the DMZ subnet and PCI subnet and let them talk using whatever access-list rules seem approriate.   The ASA will automatically route between directly attached subnets; you only need route statements for stuff that isn't directly attached.

On the site to site IPsec VPN tunnel, once again, just have an identity NAT rule covering the two ends, and they'll talk to each other fine.  I usually seem to need egress list rules permitting the tunnel traffic too.

-- JIm L

[bitflowcisco]

Dear Jouni,

Thank you very much vor your informative response.  My apologies for taking a couple of days to respond as I was out of town.

I have a few follow-up questions.

Scenario 1

1) LAN users (on the private IP range) will need to access the servers based on their public IP address.  Using a split-DNS is not an option and I'd prefer no to play aroudn using DNS rewrites because sometimes the public IP address is used directly.  You mentioned that there can be some NAT rules configured to bypass this? Can you please elaborate on that? I'll be using the GUI ASDM interrface to configure this.  Is it a matter of simply checking the "Enable traffic between two or more hosts connected to the same interface" option?

2) If I wanted to setup a site-to-site IPSEC VPN to a mirror site hosted in another datacentre does this complicate anything in this scenario? Basically the private IP range at this facility will need to be able to communicate to the private IP range at the mirror facility.  Access between the two should be pretty free, but we may want to implement a few basic firewall rules.  Of course we would not want to NAT this communication.

3) If I wanted to setup a Client VPN (using the Cisco VPN client) and get an IP address on the same private network for management purposes, would this complicate anything? As per the site-to-site scenario above we might want to implement a few basic firewall rules but generally it would be full access.

Scenario 2:

1) I am leaning towards this scenario because it seems the simplest to me.  I am concerned however that you do not recommend multi-homed servers - why is this? As I see it, I would configure the routing table on each server to route any prviate IP addresses to the local network router (not an ASA, just the 3750G that the private interface on the server is connected to).  All other routes would go to the ASA.  This seems very simple to me?

2) Similar to the details in Scenario #1 question 2, what happens if we want to setup a site-to-site IPSEC VPN link to a mirror site in a different datacenter? I would want to communicate from the private subnet on this datacentre to the prviate range on the mirror datacentre.  I assume I would also have to update the routing table on each server to also send any traffic to the private-range on the mirror site to the ASA so that it can send it across the VPN link?  I suppose this means the ASA would then need an interface & IP address on the private network too?

3) Similar to Scenario #1, questio 3, what happens if we want to setup a client VPN (using the cisco VPN client) and get an IP address on the private range? Mainly for admin related tasks.  Assume the ASA would then need an Interface & IP address on the private network?

Thanks again for all your assistance, looking forward to your responses.

Best

[Jouni Forss]

Hi,

Scenario 1

1.) I think in some cases people try to overcome the fact that a server is configured with a private IP address locally and a public IP address with Static NAT on the firewall by also configuring the Private to Public Static NAT from the DMZ towards the LAN so that the DNS reply wont be a problem.

I personally avoid doing this as it seems to me again a configuration that will make the ASA configuration more complicated than it has to be. I havent had to resort to this kinda of NAT setup personally as we have usually corrected the situation by other means.

2.) This should be no problem at all. You can define different NAT rules for the same hosts/servers. Typically with L2L VPN or Client VPN your NAT0 / NAT Exempt traffic going to the VPN. This will always override the Static NAT configuration. At the sametime the NAT0 / NAT Exempt is only configured to apply for the VPN connections so it doesnt interfere with the Static NAT operation to public network in any way. I can't give you any specific answers (though you naturally may ask about them when you reach that point here on the forums ) But I dont see a problem related to this situation.

3.) Can you clarify this a bit? What do you mean with the same private network? Or are you just generally speaking about the networks behind the firewall? There should be no problem providing VPN Client connectivity to the servers. NAT and VPN configurations depends on how you want to implement the setup.

In both VPN Client and Site to Site VPN (L2L VPN) you have atleast a couple of options how to approach controlling the traffic that comes from the VPN connection

• You can apply a Filter ACL on the VPN connection that only applies to the connections of the said VPN connection. The configuration format is different for Client and Site to Site connections.
  
• You can change the default behaviour of the ASA so that it DOESNT automatically allow all connections coming from a VPN connection through the "outside" interface of the firewall (where the VPN connections usually come from) This way you can controll all traffic for VPNs in your "outside" interface ACL. I tend to like this setup more than the previous even though in certain places it might "bloat" the "outside" interface ACL. Still it keeps the ACL setup consistent
  

Scenario 2

1.) I guess my opinion on this is more related to my own job and our companys/customers roles. Since I mostly manage firewall and vpn environments in my work I tend to have almost nothing to do with the actual server side either hosted by us or at customer premises. In several cases I just have witnessed what not knowing how routing works can cause for the customer network. People managing the servers have made the setup without consulting anyone managing the network and suddenly something doesnt work and ofcourse its the firewalls fault and people managing the firewalls Usually we end up noticing that there is asymmetric routing and connections get blocked. This is my main reason

Naturally if we are talking about a case where either a single person or a small group of people handle both the IT and the Networking this might be ok and not a problem at all. When you know how the setup is and what you have to take into consideration regarding the network this might be fine. So in your case it might not be a problem at all. Though there is always a possibility that you will run into some problems.

Also not overloading the ASA with heavy traffic in this case it also a good point to go for this.

2.) If you are saying that the traffic coming from the VPN connections needs to reach the servers through the private IP address range interface and not the public then you will simply need to have a route from the ASA to that network. You dont necesarily need a direct interface on the ASA for that network. You could route it through some existing ASA interface and the LAN router could handle it. It all depends on the whole setup.

3.) Again I dont see a problem with this. It comes down to your whole network setup on how you provide the connection to the servers private IP address interface through the VPN connection. Just make sure the server has the route to the VPN Pool network through the correct interface which in this case would be the private IP address interface of the server.

Hopefully the above information has been helfpull

Do ask more if I missed something totally (Still a bit tired)

- Jouni

[bitflowcisco]

Dear James & Jouni,

Thank you very much for your responses.  Just an update that we ended up taking your suggestion and went with private IP's on all servers and static nat from the outside for internet accessible servers.  It is working very well.

Now, a followup question:

-We created a site-to-site VPN link to another site and included the remote (local) IP range (192.168.200.x) in the crypto map for the vpn link.

-This works great and there is full communication between the subnets.

-We then tried to add a static route for 192.168.0.0/16 on the inside interface to a local router on the inside that handles routes to all our other internal subnets (192.168.0.0/16).

-As soon as I added this route it seems to take precedence over the more specific crypto-map (192.168.200.0/24) and now all communication over the site-to-site VPN link terminates and is instead routed over the static route.

-Why is this? The crypto map is more specific (192.168.200.0/24) than the static route (192.168.0.0/16).

How do we go about resolving this? Can we adjust the "metric" of the static route so that the more specific crypto map takes precedence?

Please advise as it seems right now we can either have access to the remote site OR the local subnets but not both.

Thanks

[Jouni Forss]

Hi,

You could add this configuration line

crypto map

Notice that this configuration lines

This will create a route to your ASAs routing table for the remote 192.168.200.0/24 network automatically based on the L2L VPN configuration ACL. Its called Reverse Route Injection

EDIT: Other option to do this would probably be to simply add a Static route for the network 192.168.200.0/24 pointing to your public Internet gateway. I think this is what the "reverse-route" will basicly do also.

Hope this helps

- Jouni

[bitflowcisco]

Hi Jouni,

You're amazing, thank you! that worked like a charm - although I dont see it putting a static route in the static route section of ASDM but as soon as I enabled reverse route in ASDM for the crypto map it worked .

One curious question, however.  If I try to ping directy form the ASA I can't ping anything thats on the other side of the site-to-site link. i.e. nothing on 192.168.200.0/24.  However,I can ping anything locally on 192.168.0.0/16

Only thing I can think of is that the crypto map not matching when the source is the asa?

[Jouni Forss]

Hi,

If youre looking at Static Routes from the ASDM I'd imagine it wont show the dynamically added Static route (well that sounded funny ) in the configuration section atleast.

However if you look at the actual routing table you should see the route. Or optionally use the "show route" on the CLI then the ASA should list the added route.

The reason why you cant ping remote hosts directly from the ASA is because the ASA would use its "outside" interface IP address as the source. So you would basically need to have the "outside" interface IP address in the crypto map ACL on both sites configurations and NAT configurations for it to work. (I guess only the remote sites NAT configurations)

I personally havent done this that many times. I have included the ASA "outside" IP address to the crypto map ACL in occasions where I want to encrypt the Syslog traffic sent by the ASA.

- Jouni

[bitflowcisco]

Hi Jouni,

Again, thanks for your informative answer. 

I just disabled reverse-route on the crypto map and put in a static route and that also works so this now makes sense. 

However, I started thinking.  I find this route a little confusing - I'm sending a private range to the outside world.  How does this tell the asa to send it through the tunnel? I'm assuming the crypto map will see an IP with source & destination that matches its interesting traffic and thus pick it up and take it over the VPN tunnel?

But what happens with this route if an internal IP address that is not part of the crypo map source tries to access the remote destination?  Will this route just try to send a private internal IP out over the internet to my internet gateway?

Just want to make sure I'm not missing something here... ?

Thanks

[Jouni Forss]

Configuring the static route was just something I was thinking for testing purposes.

I guess the route is there either way. Either through the "reverse-route" or "route" command.

I have never tested with the "reverse-route" configuration would anything special happen to traffic from local network (that isnt part of the L2L VPN configurations) to the remote network.

I'd imagine at the least this would be dropped somewhere on the ISP network if it got through the ASA.

Then again if you dont want it to get that far you can always add some lines to your local ASA interface ACLs. You would first have to allow traffic to all the required private network ranges and then block all the rest of the traffic to any of the 3 major private address ranges.

- Jouni