NAT reverse path failure

[Mark Crawford]

Hi All,

I hope somenone can help with this particulary frustrating issue I've got.

I should say that I'm fairly new to Cisco firewalls, so please bear with me.

I have a piece of software on my print server that sends toner level information to my print supplier. The software periodically uses a telnet session to test connectivity, but this is failing.

My print server has a static NAT through the firewall. The settings are;

Inside IP: 11.0.214.8

Outside: 10.199.155.250

According to packet tracer, the traffic is getting from my print server out to my suppliers server, but is getting dropped on the way back in by a NAT rule.

The message in the log is;

"Asymmetric NAT rules matched for forward and reverse flows; Connection for protocol 0 src outside:168.63.28.202 dst inside:11.0.214.8 denied due to NAT reverse path failure."

Can anyone shed some light on what I might be doing wrong?

Regards

Mark

[Rishabh Seth]

Hi Mark,

From the description I see that you have a public IP on the inside and a private IP on outside. Is it correct as per your network?

In case it is dropping in rpf then, there might be other NAT rule which might be getting evaluated for the return path. You can check in the packet tracer output whether same NAT rule is evaluated for incoming and outgoing traffic. 

If different NAT rules are hit then try to rearrange NAT rules.

Also share the ASA version that you are running and more details about the NAT rule.

Thanks,

RS

[Mark Crawford]

Hi Rishabh,

The IP config is correct. That's the way our vendor set everything up.

I can confirm that the same rule is being evaluated for both outgoing and incoming traffic.

My ASA version is 9.1(5)

The NAT rule in config is this:

nat (inside,outside) source static OR-SRV-ORDC-PS1_Inside OR-SRV-ORDC-PS1_Outside

Mark

[Akshay Rastogi]

Hi Mark,

For checking testing, you could perform followings thing. There is possibility that there might be some more preferred MAnual NAT which is getting hit with  traffic.

First thing is, place this NAT on line 1 with below command :

nat (inside,outside) 1 source static OR-SRV-ORDC-PS1_Inside OR-SRV-ORDC-PS1_Outside

If this also doesn't work and try with Object NAT once:

Object network obj-internal-printserver

 host <internal server ip>

 nat (inside,outside) static <mapped-ip>

use 'interface' keyward instead of IP address if you are using the outside interface ip as natted ip.

Note: Remove your already existing Manual NAT. It looks more of an NAT rearrangement issue. More information could be find out if you share more details on other existing nat statements.

Hope it helps.

Regards,

Akshay Rastogi

[Mark Crawford]

Hi Akshay,

My NAT rules are as follows in the order that they are in this list;

nat (inside,outside) source static OR-SRV-ORDC-PS1_Inside OR-SRV-ORDC-PS1_Outside
nat (inside,outside) source dynamic RIVERS_NEW_LAN interface
nat (inside,outside) source static Smoothwall_BYOD_Interface BOYD_LAN_Outside
nat (inside,outside) source static Parent_reporting_system_inside Parent_reporting_system_outside
nat (inside,outside) source static Spiceworks_Inside Spiceworks_Outside

If I create an object NAT and remove the manual NAT, the traffic is picked up by the dynamic NAT and dropped on the return.

Mark

[Akshay Rastogi]

Hi Mark,

It is always recommanded to have Auto/Object NAT for Dynamic Statements if you are not using destination nat. It might cause wrong NAT to match. Manual NAT are processed from Top to Bottom:

So Change your dynamic statement something like:

object network obj-Rievers_new_lan

 subnet <this subnet>

 nat (inside,outside ) dynamic interface

Also try to configure Object NAT for your Actual required traffic as i had mentioned in my last post.

Note : Remove both the Manual NAT statement.

Regards,

Akshay Rastogi

[Mark Crawford]

Hi Akshay,

I have made the modification you suggested, but the traffic is still being dropped, but now by the object rule for the static mapping.

Mark

[Akshay Rastogi]

Hi Mark,

Could you please share the complete output of packet-tracer on the ASA including packet-tracer command you are running.

Regards,

Akshay Rastogi

[Mark Crawford]

ORA-ASA-RTR-001# packet-tracer input outside tcp 168.63.28.202 23 11.0.214.8 23

Phase: 1
Type: ROUTE-LOOKUP
Subtype: input
Result: ALLOW
Config:
Additional Information:
in   11.0.192.0      255.255.224.0   inside

Phase: 2
Type: ROUTE-LOOKUP
Subtype: input
Result: ALLOW
Config:
Additional Information:
in   0.0.0.0         0.0.0.0         outside

Phase: 3
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group external_access_in in interface outside
access-list external_access_in extended permit ip any object RIVERS_NEW_LAN 
Additional Information:
              
Phase: 4
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:

Phase: 5
Type: IP-OPTIONS
Subtype: 
Result: ALLOW
Config:
Additional Information:

Phase: 6
Type: NAT
Subtype: rpf-check
Result: DROP
Config:
object network OR-SRV-ORDC-PS1
 nat (inside,outside) static 10.199.155.250
Additional Information:

Result:       
input-interface: outside
input-status: up
input-line-status: up
output-interface: inside
output-status: up
output-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule

[Rishabh Seth]

Hi Mark,

Is the object definition mentioned below is correct?

OR-SRV-ORDC-PS1_Inside = 10.199.155.250
OR-SRV-ORDC-PS1_Outside = 11.0.214.8
OR-SRV-ORDC-PS1 = 11.0.214.8

in your first post you have mentioned that the ouside IP is 10.199.155.250 and inside IP is 11.0.214.8. 

Going by that description, I would expect the input interface in the packet tracer command to be Inside and not Outside.

Also try the Object NAT as:

object network OR-SRV-ORDC-PS-test
host 10.199.155.250
object network OR-SRV-ORDC-PS-test
nat (outside,inside) static 11.0.214.8

Hope it helps.

Thanks,

RS

[Mark Crawford]

Hi Rishabh,

The definitions are:

OR-SRV-ORDC-PS1_Inside = 11.0.214.8

OR-SRV-ORDC-PS1_Outside = 10.199.155.250

OR-SRV-ORDC-PS1 = 11.0.214.8

On the packet trace command I set the input interface as Outside because it's only on the return path that traffic is dropped.

[Rishabh Seth]

Testing reverse traffic that way would not be correct. Try the traffic from actual host to destination through correct interface.

In actual scenario the return traffic will the session created for outgoing traffic and will get processed.

Let us know if the packet tracer from source machine to destination server is working fine or not.

And is the traffic always initiated from the printer to outside server or the other way is also expected.

Thanks,

RS

[Mark Crawford]

Using packet trace, traffic from the internal source to external destination is allowed.

However, the reporting software on my print server is unable able to communicate with the external destination.

[Rishabh Seth]

If packet tracer is showing that the traffic is allowed then I would suggest you to see connections in the ASA for the relevant host.

use following command.

show conn address <ip address> long detail

If you see the connection in the connection table then try to find the state of the connection with the felt of connection flags in above mentioned command.

It is possible that the remote device is not responding to your printer application.

Thanks,

RS

[Peter Koltl]

Similar issue solved here:

https://supportforums.cisco.com/discussion/12708501/nat-reverse-path-failure