Hi, Well thats a strange

jlmickens · ‎08-28-2014

I'm in a state of confusion at this point. I have an ASA 5520, v9.1(4), that I'm just connecting up to a switch for direct client access. The switch is configured to use the ASA as it's default route. I have the NTP on the switch set up, but it's not syncing. In looking at the traffic, it's coming in the ASA's ClientDA interface as it should, and going out the Outside interface, as it should, but it's not NATing to the Outside interface. This is causing the traffic to not return. I have the dynamic PAT set up, but it doesn't seem to be working. Similarly, the NTP requests from the site-to-site VPNs aren't NATing to the Outside interface either. Using the Packet Tracer in the ASDM tells me that the traffic is denied by policy, but I do a packet capture and it shows the traffic going out, but not being translated. Any ideas?

jlmickens · ‎08-28-2014

Here's the config.

jlmickens · ‎09-02-2014

Sorry this took so long to respond - it was a holiday weekend here in the States. Here's another packet trace using whois instead of ntp.

FWANVPN01# packet-tracer input clientDA tcp 10.99.97.2 whois 72.240.1.140 whois

Phase: 1
Type: ROUTE-LOOKUP
Subtype: input
Result: ALLOW
Config:
Additional Information:
in 0.0.0.0 0.0.0.0 Outside

Phase: 2
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group ClientDA_access_in in interface ClientDA
access-list ClientDA_access_in extended permit ip 10.99.97.0 255.255.255.248 host 72.240.1.140
Additional Information:

Phase: 3
Type: NAT
Subtype:
Result: ALLOW
Config:
nat (ClientDA,Outside) source dynamic ClientDA-subnet interface
Additional Information:
Dynamic translate 10.99.97.2/43 to 72.240.36.39/43

Phase: 4
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:

Phase: 5
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:

Phase: 6
Type: NAT
Subtype: rpf-check
Result: ALLOW
Config:
nat (ClientDA,Outside) source dynamic ClientDA-subnet interface
Additional Information:

Phase: 7
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:

Phase: 8
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:

Phase: 9
Type: FLOW-CREATION
Subtype:
Result: ALLOW
Config:
Additional Information:
New flow created with id 16581577, packet dispatched to next module

Result:
input-interface: ClientDA
input-status: up
input-line-status: up
output-interface: Outside
output-status: up
output-line-status: up
Action: allow,<

jlmickens · ‎09-02-2014

Here's the other output you requested:

FWANVPN01# sh conn | i 10.99.97.2
TCP ClientDA 10.99.97.2:22 Inside 172.16.30.200:63707, idle 0:00:30, bytes 8072, flags UIO
UDP Outside 72.240.1.140:123 ClientDA 10.99.97.2:123, idle 0:00:20, bytes 335712, flags -
FWANVPN01#
FWANVPN01# sh local-host 10.99.97.2
Interface ClientDA: 1 active, 2 maximum active, 0 denied
local host: <10.99.97.2>,
    TCP flow count/limit = 1/unlimited
    TCP embryonic count to host = 0
    TCP intercept watermark = unlimited
    UDP flow count/limit = 1/unlimited

Conn:
UDP Outside 72.240.1.140:123 ClientDA 10.99.97.2:123, idle 0:00:47, bytes 335712, flags -
TCP ClientDA 10.99.97.2:22 Inside 172.16.30.200:63707, idle 0:00:57, bytes 8072, flags UIO
Interface management: 0 active, 0 maximum active, 0 denied
Interface Inside: 8 active, 50 maximum active, 0 denied
Interface Outside: 79 active, 135 maximum active, 0 denied

Jouni Forss · ‎08-28-2014

Hi,

Without seeing the configuration we can't really tell if there is any problems with it.

I have once seen an issue where a normal Dynamic PAT configuration simply was not applied to the traffic at all and the traffic was just passed without NAT. In this case the source addresses were specified inside an "object-group" which was then used in the "nat" command.

In that case adding a Dynamic PAT configuration that matched all source addresses corrected the situation. But naturally it seemed like a bug.

In that case we added a NAT configuration in this format

nat (sourceint,destint) after-auto source dynamic any interface

It would still be good to see the CLI output of the "packet-tracer" and possibly some firewall configurations to see if there are any problem there.

Hope this helps :)

- Jouni

Jouni Forss · ‎08-28-2014

Ah, you posted the configuration while I was typing the reply.

Seems to me that the Dynamic PAT is at the top of the Section 1 Manual NAT so it should be matched first.

The NAT configuration I suggested used a format that would make it a Section 3 Manual NAT (since I used the "after-auto" parameter) In your current case it would mean though that my example NAT configuration would not be applied.

It seems your NAT configuration is all done in Section 1. I guess it would still be possible to use the "any" parameter in the "nat" command AS LONG AS you specify the source interface in the "nat" command as "ClientDA". If it was set as "any" it could override all your NAT0 configurations for the VPNs.

I typically place NAT0 in Section 1 Manual NAT and normal Dynamic PAT configurations to Section 3 Manual NAT so that I minimize the risk of causing problems between the NAT configurations. Static NAT I typically configure in Section 2 Auto NAT.

On the basis on your currrent NAT configuration it would seem to me that you could make it a lot smaller with some small modifications.

- Jouni

jlmickens · ‎08-28-2014

Thanks for the quick response. I'm not sure what you mean by Section 1, Section 2 and Section 3.

I had initially tried an "any" parameter in the NAT, but then found some literature that mentioned using the object-based parameters instead. I couldn't get it working either way. Moving the rule around didn't help either. I've been using the ASDM for the configuration.

At this point I can't make any changes because I'm outside my maintenance window, but here's a CLI packet-tracer:

FWANVPN01# packet-tracer input ClientDA udp 10.99.97.2 ntp 72.240.1.140 ntp de$

Phase: 1
Type: FLOW-LOOKUP
Subtype:
Result: ALLOW
Config:
Additional Information:
Found flow with id 15541707, using existing flow
Module information for forward flow ...
snp_fp_inspect_ip_options
snp_fp_adjacency
snp_fp_fragment
snp_ifc_stat

Module information for reverse flow ...
snp_fp_inspect_ip_options
snp_fp_adjacency
snp_fp_fragment
snp_ifc_stat

Phase: 2
Type: NAT
Subtype: rpf-check
Result: ALLOW
Config:
nat (ClientDA,Outside) source dynamic ClientDA-subnet interface
Additional Information:
 Forward Flow based lookup yields rule:
 out id=0x755f7630, priority=6, domain=nat-reverse, deny=false
        hits=16, user_data=0x6f7dd430, cs_id=0x0, use_real_addr, flags=0x0, protocol=0
        src ip/id=10.99.97.0, mask=255.255.255.248, port=0, tag=0
        dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0, dscp=0x0
        input_ifc=ClientDA, output_ifc=Outside

Result:
input-interface: ClientDA
input-status: up
input-line-status: up
Action: allow

Jouni Forss · ‎08-28-2014

Hi,

Well thats a strange output.

I guess it means that there is an existing connection through the firewall between these hosts? If you check with some command like

show conn | inc 10.99.97.2

show local-host 10.99.97.2

Can you test the "packet-tracer" with some other destination port for example and see what the output is on that one?

The Sections 1-3 in the software levels 8.3+ set the priority of the NAT configuration. Almost all of your NAT configurations are in Section 1 so they are matched against traffic in the order they show up in the CLI configuration.

If you wanted you would have the option to use NAT configurations in different Sections to further alter the priority on which the configurations are matched against traffic arriving on the firewall. In my typical setup NAT0 is in Section 1 because NAT0 by nature is a configuration that should override other NAT configurations present on the firewall. Next in Section 2 I would tend to configure all the Static NAT and Static PAT configurations as these should not override NAT0 but should still come before Dynamic PAT/NAT rules. And finally I usually configure all Dynamic PAT and even Dynamic Policy PAT configurations in Section 3 in which case NAT0 and Static NAT configurations are processed before in the other sections and if the traffic does not match any of those higher priority configurations it will fall into the Dynamic PAT configurations at the very lowest priority.

You can actually see the mentions of the Section in the CLI of the ASA if you use the command

show nat

The following command would show even more information about the configuration but tends to be hard to read.

show nat detail

I guess you are probably better off reading the information from a document I wrote early in 2013. Naturally you can ask more if needed. Heres a link to the document (sadly the forum update messed with its formatting a bit)

https://supportforums.cisco.com/document/132066/asa-nat-83-nat-operation-and-configuration-format-cli

At the moment there does not seem to be a clear reason why the NAT would not be performed as the Dynamic PAT configuration used is at the very top prioty. You also have a Section 2 Auto NAT (also called Network Object NAT) doing the same translation so there is actually 2 Dynamic PAT configurations for the subnet.

So it seems we are probably talking about somekind of software bug. It would not be anything new considering all the problems related to the new NAT I have seen in the past year or more.

But can you post another "packet-tracer" output using different port so I can see an output of connection that is not yet present on the firewall.

- Jouni

jlmickens · ‎10-24-2014

So today I go into the switch that this issue was affecting and behold that the time is properly synced. A packet capture on the ASA reveals that the NAT is now working properly. The device says it's been up for 136 days, so it didn't reboot.

The only thing I did was add a second ASA as a standby unit. No other configuration changes were made.

smh...

asa dynamic pat not working ?