ASA 5505 9.1 and NAT issues to single dynamic IP

[cesaregiuliani]

Good afternoon everybody, 

a few days ago I tried setting up my ASA 5505 to allow access from the outside network to an Exchange server (ports HTTPS and SMTP) in my inside LAN.

Everything seems to be working... until my outside IP address changes (for example due to a router reset or a disconnection caused by the ISP). 

As soon as the outside address changes the NAT rules are deleted and these 2 lines pop up in the syslog :

<166>%ASA-6-305012: Teardown static TCP translation from inside:192.168.1.150/25 to outside:79.6.105.13/25 duration 0:01:17.
<166>%ASA-6-305012: Teardown static TCP translation from inside:192.168.1.150/443 to outside:79.6.105.13/443 duration 0:01:17.

In the same time, the consolle connection shows these two messages :

Asa5505# ERROR: NAT unable to reserve ports.
ERROR: NAT unable to reserve ports.

I have moved both Anyconnect VPN essentials and http ports to 10443 and 8080 respectively so port 443 should be free for nat.

This is the configuration file, I  have marked the lines related to network objects and relative nat statements, I hope it helps to find out where's the problem.

Obviously the lines in red are the ones disappearing... I'm quite desperate, actually.

ASA Version 9.1(5) 
!
hostname Asa5505
domain-name home
enable password XXXXXX encrypted
names
!
interface Ethernet0/0
 description ADSLPPoE
 switchport access vlan 2
!
interface Ethernet0/1
 description Internal_LAN
!
interface Ethernet0/2
 description Management_Net 
 switchport access vlan 3
!
interface Ethernet0/3
 shutdown
!
interface Ethernet0/4
 shutdown
!
interface Ethernet0/5
 description Uplink
 switchport trunk allowed vlan 1,3
 switchport trunk native vlan 1
 switchport mode trunk
!
interface Ethernet0/6
 description Wireless-POE
 switchport trunk allowed vlan 1,3
 switchport trunk native vlan 1
 switchport mode trunk
!
interface Ethernet0/7
 description Webcam-POE 
!
interface Vlan1
 nameif inside
 security-level 100
 ip address 192.168.1.250 255.255.255.0 
!
interface Vlan2
 nameif outside
 security-level 0
 pppoe client vpdn group AliceADSL
 ip address pppoe setroute 
!
interface Vlan3
 no forward interface Vlan1
 nameif management
 security-level 100
 ip address 10.5.1.250 255.255.255.0 
!
ftp mode passive
clock timezone CEST 1
clock summer-time CEDT recurring last Sun Mar 2:00 last Sun Oct 3:00
dns domain-lookup inside
dns domain-lookup outside
dns server-group DefaultDNS
 name-server 192.168.1.4
 domain-name home
object network Exchange-HTTPS
 host 192.168.1.150
object network Exchange-SMTP
 host 192.168.1.150
object network Network_Inside
 subnet 192.168.1.0 255.255.255.0
object network Network_Management
 subnet 10.5.1.0 255.255.255.0
access-list Outside_ACL extended permit tcp any object Exchange-HTTPS eq https 
access-list Outside_ACL extended permit tcp any object Exchange-SMTP eq smtp 
pager lines 24
logging enable
logging asdm warnings
mtu inside 1500
mtu outside 1492
mtu management 1500
no failover
icmp unreachable rate-limit 1 burst-size 1
no asdm history enable
arp timeout 14400
no arp permit-nonconnected
!
object network Exchange-HTTPS
 nat (inside,outside) static interface service tcp https https 
object network Exchange-SMTP
 nat (inside,outside) static interface service tcp smtp smtp 
object network Network_Inside
 nat (inside,outside) dynamic interface
object network Network_Management
 nat (management,outside) dynamic interface
access-group Outside_ACL in interface outside
timeout xlate 3:00:00
timeout pat-xlate 0:00:30
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute
timeout tcp-proxy-reassembly 0:01:00
timeout floating-conn 0:00:00
dynamic-access-policy-record DfltAccessPolicy
user-identity default-domain LOCAL
http server enable 8080
http 10.5.1.0 255.255.255.0 management
no snmp-server location
no snmp-server contact
snmp-server enable traps snmp authentication linkup linkdown coldstart warmstart
crypto ipsec security-association pmtu-aging infinite
crypto ca trustpool policy
telnet timeout 5
ssh stricthostkeycheck
ssh timeout 5
ssh key-exchange group dh-group1-sha1
console timeout 0
management-access management
vpdn group AliceADSL request dialout pppoe
vpdn group AliceADSL localname aliceadsl
vpdn group AliceADSL ppp authentication pap
vpdn username aliceadsl password ***** store-local

dhcpd address 192.168.1.100-192.168.1.130 inside
dhcpd dns 192.168.1.4 192.168.1.150 interface inside
dhcpd wins 192.168.1.4 interface inside
dhcpd enable inside
!
dhcpd address 10.5.1.30-10.5.1.40 management
dhcpd dns 208.67.222.222 208.67.220.220 interface management
dhcpd enable management
!
threat-detection statistics access-list
no threat-detection statistics tcp-intercept
webvpn
 port 10443
 anyconnect-essentials
!
class-map inspection_default
 match default-inspection-traffic
!
!
policy-map type inspect dns preset_dns_map
 parameters
  message-length maximum client auto
  message-length maximum 512
policy-map global_policy
 class inspection_default
  inspect dns preset_dns_map 
  inspect ftp 
  inspect h323 h225 
  inspect h323 ras 
  inspect ip-options 
  inspect netbios 
  inspect rsh 
  inspect rtsp 
  inspect skinny  
  inspect esmtp 
  inspect sqlnet 
  inspect sunrpc 
  inspect tftp 
  inspect sip  
  inspect xdmcp 
!
service-policy global_policy global
prompt hostname context 
no call-home reporting anonymous
call-home
 profile CiscoTAC-1
  no active
  destination address http https://tools.cisco.com/its/service/oddce/services/DDCEService
  destination address email callhome@cisco.com
  destination transport-method http
  subscribe-to-alert-group diagnostic
  subscribe-to-alert-group environment
  subscribe-to-alert-group inventory periodic monthly
  subscribe-to-alert-group configuration periodic monthly
  subscribe-to-alert-group telemetry periodic daily
Cryptochecksum:XXXXXXXX
: end
no asdm history enable

Thanks in advance for your precious help !
C.

[nkarthikeyan]

Hi,

I guess thats the known bug....

https://tools.cisco.com/bugsearch/bug/CSCun95075

You can update your OS with resolved version as per this bug id closure information.

Regards

Karthik

[Marius Gunnerud]

when your IP changes and you lose connection, could you issue the command show xlate | include 192.168.1.150 and see if the outside interface IP it is translating to corresponds to the new IP the ASA has now received.  If the translated IP referenses the old public IP, issue the command clear xlate local 192.168.1.150 and see if that solves the issue.  This could help us narrow down what is actually causing your problem.

@karthik  -  This could very well be a bug, but I do not believe it is the bug you refer to in that link as that refers to twice NAT.

--

Please remember to select a correct answer and rate helpful posts

[cesaregiuliani]

First of all, thank you both for answering me.

Tomorrow morning I'll try both suggestions. I'll upgrade the OS version to 9.22 wich is the latest currently available for ASA5505, perform a WR ER in order to "clean up the mess" then restore the config from terminal (I've saved it in a txt with all the statements in the correct order so I can copy-paste them using putty). I'll check the behaviour by forcing a disconnection (power off the DSL PPPoE modem) to see if the show xlate gives me some interesting results. In both cases, I'll keep you updated.

As a side note, it's quite "fun" to see how complicated things could get when you wanna do something so simple as a port forward. The funniest thing is that on a 350€ + licenses piece of hardware you have to waste a lot of time in order to achieve something that - on a 30€ router - can be done in less than 3 minutes :) Setting up AnyConnect + SSL Clientless is far more easy!!

Thank you again !!

C.

[cesaregiuliani]

Update 29th of June :

Tried both suggestions: flashing to 9.22 didn't fix the problem. The only significant change between 9.1(5) and 9.2(2) is that as soon as I reload the configuration after a connection drop both nat rules are restored. In 9.1(5) the nat statements were removed from the runnning configuration when the PPPoE connection was lost, and the config was updated (or maybe saved?), so after a reload those statements were gone and I had to copy-paste them back in conf-t in order to restore them.

I tried using show xlate both before, during, and after the connection drop. As expected before the disconnection of PPPoE the static PAT rules are there, and the dynamic ones as well. During disconnection, all the xlate table is clean empty and the aforementioned error "Asa5505# ERROR: NAT unable to reserve ports. ERROR: NAT unable to reserve ports." pops up in the terminal. After a few minutes (needed by the DSL modem to perform its reset and bring up the DSL line again) the connection is established once more, but the only rules appearing in xlate are the ones created by the dynamic statements for management and LAN. If i reload the ASA using reload noconfirm every rule is restored and everything works again.

Two brief questions :

1) in my NAT statements for PAT, does it change anything if I modify them (for example) from 

nat (inside,outside) static interface service tcp https https

to

nat (inside,outside) dynamic interface service tcp https https 

? Since it seems like the dynamic PAT is restored after a connection drop I was asking myself what happens if I change the rules this way.

2) if there's not any ohter way to fix this, is it possible to schedule a reload of the ASA as soon as the PPPoE connection drops in order to make this problem "self fixing" ? I can't predict how many times a day the line drops and I can't be there 24/7 with my consolle cable connected in order to restore the nat statements ^^

Thank you for your precious help and patience !
C.

[Marius Gunnerud · Accepted solution]

I am not sure why those commands are being removed from the configuration,  You might want to open a TAC case as this could very well be a bug.

? Since it seems like the dynamic PAT is restored after a connection drop I was asking myself what happens if I change the rules this way.

You could try it but dynamic is used when you have multiple source addresses translating to a single address...but you can give it a try if you want.  But your configuration is a bit off.  You would need to do something like the following:

object network SERVER
  host 192.168.1.150

object service HTTP
 service tcp destination eq www

nat (inside,outside) source dynamic SERVER interface service HTTP HTTP

2) if there's not any ohter way to fix this, is it possible to schedule a reload of the ASA as soon as the PPPoE connection drops in order to make this problem "self fixing" ? I can't predict how many times a day the line drops and I can't be there 24/7 with my consolle cable connected in order to restore the nat statements ^^

If you upgrade the ASA to version 9.2.1 you can use the embedded event manager (EEM) to trigger a reload when a "line down" and/or "protocol down" message is reported.  You could match on syslog id 411002 which is for line protocol down. and then configure an event to take place...ie the reload.  This feature is very new in the ASA so the options to match on syslog patterns is not present...yet.  Now I have not had the chance to play around with this yet, but from the looks of the configuration guides the commands would be as follows:

event manager applet EXAMPLE
  event syslog id 411002
  action 1 cli command “reload noconfirm”

http://www.cisco.com/c/en/us/td/docs/security/asa/asa92/configuration/general/asa-general-cli/monitor-eem.html#pgfId-1922696

--

Please remember to select a correct answer and rate helpful posts

[Marius Gunnerud]

To be perfectly honost, this is a very simple thing to do, and under normal circumstances would only take a minute or two to set up.  Unfortunately it seems that you might be running into a bug.

Another thing you could do is check your ASDM do see if the confiugration there matches what is in the CLI.  I have seen a VPN configuration not work because there were configurations present in the ASDM when the client configured it, but for some reason it was not pushed out to the ASA.

--

Please remember to select a correct answer and rate helpful posts

[cesaregiuliani]

I'm getting bored about this issue. It's simply absurd.

I tried both your suggestions:

using the nat declaration:

nat (inside,outside) source dynamic SERVER interface service HTTP HTTP

creates an xlate rule from network 0.0.0.0/0 to 0.0.0.0/0

That statement persists even after a PPPoE connection drop..but simply doesn't work at all. Now I'm experimenting using a script like the one you linked me in order to force reload on a connection drop; I found that the most intelligent way to do that is to catch the PPPoE link down event in order to force reload, otherwise, catching a generic link down event would result in unpredictable behaviors but the situation is still unclear, unstable and not reliable. I'm quite pissed off to be honest. I don't know if there's something "hiding" behind my configuration forcing the ASA to behave this way. But it seems... strange after a few "wr er" that some chunks of config might have survived somewhere...

[Marius Gunnerud]

I did not expect that NAT statement to work as the object group is a host object which would require a static nat statement.

It is a very strange issue, and it cold be a bug.  I suggest opening a TAC case for this, because if it is a bug that is not reported yet, Cisco can get onto it and create a fix.

--

Please remember to select a correct answer and rate helpful posts

[cesaregiuliani]

I finally came to the conclusion that the strange NAT behavior is related to a bug that seems to pop up when you connect using a PPPoE modem instead of a router, so I have chosen to follow your second suggestion Marius.

This is the implemented workaround:

1) I have made a copy of the working configuration to a file on the flash disk0 called failsafe.cfg

2) Whenever the modem resets or the ADSL line is dropped, I catch the PPPoE link down syslog event, rewrite the running-configuration with stored failsafe.cfg containing all my NAT statements and force a reload of the appliance. This is the "magic script"

event manager applet Restart_PPPoE

event syslog id 403503

action 1 cli command "copy /noconfirm failsafe.cfg startup-config"

action 2 cli command "reload noconfirm"

output none

exit

This workaround doesn't solve the problem, but it works (mail access is working since the 1st of july without interruptions)

I hope that following these suggestions might help other 

[Exoset Inc]

Hi experts,

I'm facing the exact same problem as Cesare but unfortunatly I have a 5520, which means I can't use the workaround of EEM because my last available IOS for 5520 is 9.1.5 and EEM is available from 9.2.x :(.

I know I have to upgrade my 5520 for a 5525 soon enough but not for a basic feature like keeping static nat after a PPPOE link down...

Thx