Solved: Access to DMZ - RPF-Check Drop

jerry.henzel · ‎08-01-2013

Hello

Following http://www.cisco.com/en/US/products/ps6120/products_configuration_example09186a0080bf150c.shtml to get access to a webserver on my internal zone.

Right now I cannot get the ASA to translate 443 from the outside to an internal IP.

Access going out is fine. I can capture the packets and see them coming in on the outside via 443.

Anyone with suggestions. Thanks

Jerry

PS - Config file attached

Packet-tracer shows:

Phase: 1

Type: CAPTURE

Subtype:

Result: ALLOW

Config:

Additional Information:

MAC Access list

Phase: 2

Type: ACCESS-LIST

Subtype:

Result: ALLOW

Config:

Implicit Rule

Additional Information:

MAC Access list

Phase: 3

Type: ROUTE-LOOKUP

Subtype: input

Result: ALLOW

Config:

Additional Information:

in 192.168.16.0 255.255.255.0 inside

Phase: 4

Type: ACCESS-LIST

Subtype: log

Result: ALLOW

Config:

access-group outside_www in interface outside

access-list outside_www extended permit tcp any object db1 eq https

Additional Information:

Phase: 5

Type: NAT

Subtype: per-session

Result: ALLOW

Config:

Additional Information:

Phase: 6

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Config:

Additional Information:

Phase: 7

Type: VPN

Subtype: ipsec-tunnel-flow

Result: ALLOW

Config:

Additional Information:

Phase: 8

Type: ACCESS-LIST

Subtype: log

Result: ALLOW

Config:

access-group db1_pci out interface inside

access-list db1_pci extended permit ip any any

Additional Information:

Phase: 9

Type: NAT

Subtype: rpf-check

Result: DROP

Config:

object network riverroad_inside

nat (inside,outside) dynamic interface

Additional Information:

Result:

input-interface: outside

input-status: up

input-line-status: up

output-interface: inside

output-status: up

output-line-status: up

Action: drop

Drop-reason: (acl-drop) Flow is denied by configured rule

Detailed output shows

Phase: 1

Type: CAPTURE

Subtype:

Result: ALLOW

Config:

Additional Information:

MAC Access list

Phase: 2

Type: ACCESS-LIST

Subtype:

Result: ALLOW

Config:

Implicit Rule

Additional Information:

MAC Access list

Phase: 3

Type: ROUTE-LOOKUP

Subtype: input

Result: ALLOW

Config:

Additional Information:

in 192.168.16.0 255.255.255.0 inside

Phase: 4

Type: ACCESS-LIST

Subtype: log

Result: ALLOW

Config:

access-group outside_www in interface outside

access-list outside_www extended permit tcp any object db1 eq https

Additional Information:

Phase: 5

Type: NAT

Subtype: per-session

Result: ALLOW

Config:

Additional Information:

Phase: 6

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Config:

Additional Information:

Phase: 7

Type: VPN

Subtype: ipsec-tunnel-flow

Result: ALLOW

Config:

Additional Information:

Phase: 8

Type: ACCESS-LIST

Subtype: log

Result: ALLOW

Config:

access-group db1_pci out interface inside

access-list db1_pci extended permit ip any any

Additional Information:

Phase: 9

Type: NAT

Subtype: rpf-check

Result: DROP

Config:

object network riverroad_inside

nat (inside,outside) dynamic interface

Additional Information:

Result:

input-interface: outside

input-status: up

input-line-status: up

output-interface: inside

output-status: up

output-line-status: up

Action: drop

Drop-reason: (acl-drop) Flow is denied by configured rule

Jouni Forss · ‎08-02-2013

Hi,

The traffic simply aint matching any NAT rule on the way.

Looking at your Manual NAT rules in the configuration it seems to me that most are "static" configurations with "destination" parameters so I dont see how any of them could override this NAT

I guess we can determine if any other NAT rule is causing this by configuring the NAT in the following way. Remove the previous Network Object NAT for the TCP/443 before configuring this

object network SERVER

host 192.168.16.32

object service HTTPS

service tcp source eq 443

nat (inside,outside) 1 source static SERVER interface service HTTPS HTTPS

This should also do Static PAT configuration even though I personally dont use this configuration format.

- Jouni

View solution in original post

Jouni Forss · ‎08-01-2013

Hi,

Are you sure you are targetting the public IP address?

It seems to me that you might be attempting to target the servers local IP address in the "packet-tracer" command according to the output

- Jouni

jerry.henzel · ‎08-01-2013

Hi Jouni

I ran packet-tracer input outside tcp 1.2.3.4 443 192.168.16.32 443 det with the corresponding output below

Thanks

Jerry

Phase: 1

Type: CAPTURE

Subtype:

Result: ALLOW

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0xcbd63380, priority=13, domain=capture, deny=false

hits=147540, user_data=0xcbd8f9b8, cs_id=0x0, l3_type=0x0

src mac=0000.0000.0000, mask=0000.0000.0000

dst mac=0000.0000.0000, mask=0000.0000.0000

input_ifc=outside, output_ifc=any

Phase: 2

Type: ACCESS-LIST

Subtype:

Result: ALLOW

Config:

Implicit Rule

Additional Information:

Forward Flow based lookup yields rule:

in id=0xcb15acf8, priority=1, domain=permit, deny=false

hits=2188941, user_data=0x0, cs_id=0x0, l3_type=0x8

src mac=0000.0000.0000, mask=0000.0000.0000

dst mac=0000.0000.0000, mask=0100.0000.0000

input_ifc=outside, output_ifc=any

Phase: 3

Type: ROUTE-LOOKUP

Subtype: input

Result: ALLOW

Config:

Additional Information:

in 192.168.16.0 255.255.255.0 inside

Phase: 4

Type: ACCESS-LIST

Subtype: log

Result: ALLOW

Config:

access-group outside_www in interface outside

access-list outside_www extended permit tcp any object db1 eq https

Additional Information:

Forward Flow based lookup yields rule:

in id=0xcbd35cf8, priority=13, domain=permit, deny=false

hits=14, user_data=0xc917a560, cs_id=0x0, use_real_addr, flags=0x0, protocol=6

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0

dst ip/id=192.168.16.32, mask=255.255.255.254, port=443, tag=0, dscp=0x0

input_ifc=outside, output_ifc=any

Phase: 5

Type: NAT

Subtype: per-session

Result: ALLOW

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0xcaa8c5a0, priority=0, domain=nat-per-session, deny=false

hits=69418, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=6

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0, dscp=0x0

input_ifc=any, output_ifc=any

Phase: 6

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0xcb160598, priority=0, domain=inspect-ip-options, deny=true

hits=60891, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0, dscp=0x0

input_ifc=outside, output_ifc=any

Phase: 7

Type: VPN

Subtype: ipsec-tunnel-flow

Result: ALLOW

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0xcbaae4d0, priority=13, domain=ipsec-tunnel-flow, deny=true

hits=499, user_data=0x0, cs_id=0x0, flags=0x0, protocol=0

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0, dscp=0x0

input_ifc=outside, output_ifc=any

Phase: 8

Type: ACCESS-LIST

Subtype: log

Result: ALLOW

Config:

access-group db1_pci out interface inside

access-list db1_pci extended permit ip any any

Additional Information:

Forward Flow based lookup yields rule:

out id=0xcb254968, priority=13, domain=permit, deny=false

hits=2442, user_data=0xc9179ca0, cs_id=0x0, use_real_addr, flags=0x0, protocol=0

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0, dscp=0x0

input_ifc=any, output_ifc=inside

Phase: 9

Type: NAT

Subtype: rpf-check

Result: DROP

Config:

object network riverroad_inside

nat (inside,outside) dynamic interface

Additional Information:

Forward Flow based lookup yields rule:

out id=0xcb248478, priority=6, domain=nat-reverse, deny=false

hits=36, user_data=0xcb246c18, cs_id=0x0, use_real_addr, flags=0x0, protocol=0

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0

dst ip/id=192.168.16.0, mask=255.255.255.0, port=0, tag=0, dscp=0x0

input_ifc=outside, output_ifc=inside

Result:

input-interface: outside

input-status: up

input-line-status: up

output-interface: inside

output-status: up

output-line-status: up

Action: drop

Drop-reason: (acl-drop) Flow is denied by configured rule

ASA1(config)# packet-tracer input outside tcp 1.2.3.4 443 192.168.16.32 443 det

Phase: 1

Type: CAPTURE

Subtype:

Result: ALLOW

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0xcbd63380, priority=13, domain=capture, deny=false

hits=151421, user_data=0xcbd8f9b8, cs_id=0x0, l3_type=0x0

src mac=0000.0000.0000, mask=0000.0000.0000

dst mac=0000.0000.0000, mask=0000.0000.0000

input_ifc=outside, output_ifc=any

Phase: 2

Type: ACCESS-LIST

Subtype:

Result: ALLOW

Config:

Implicit Rule

Additional Information:

Forward Flow based lookup yields rule:

in id=0xcb15acf8, priority=1, domain=permit, deny=false

hits=2191219, user_data=0x0, cs_id=0x0, l3_type=0x8

src mac=0000.0000.0000, mask=0000.0000.0000

dst mac=0000.0000.0000, mask=0100.0000.0000

input_ifc=outside, output_ifc=any

Phase: 3

Type: ROUTE-LOOKUP

Subtype: input

Result: ALLOW

Config:

Additional Information:

in 192.168.16.0 255.255.255.0 inside

Phase: 4

Type: ACCESS-LIST

Subtype: log

Result: ALLOW

Config:

access-group outside_www in interface outside

access-list outside_www extended permit tcp any object db1 eq https

Additional Information:

Forward Flow based lookup yields rule:

in id=0xcbd35cf8, priority=13, domain=permit, deny=false

hits=15, user_data=0xc917a560, cs_id=0x0, use_real_addr, flags=0x0, protocol=6

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0

dst ip/id=192.168.16.32, mask=255.255.255.254, port=443, tag=0, dscp=0x0

input_ifc=outside, output_ifc=any

Phase: 5

Type: NAT

Subtype: per-session

Result: ALLOW

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0xcaa8c5a0, priority=0, domain=nat-per-session, deny=false

hits=69453, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=6

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0, dscp=0x0

input_ifc=any, output_ifc=any

Phase: 6

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0xcb160598, priority=0, domain=inspect-ip-options, deny=true

hits=60928, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0, dscp=0x0

input_ifc=outside, output_ifc=any

Phase: 7

Type: VPN

Subtype: ipsec-tunnel-flow

Result: ALLOW

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0xcbaae4d0, priority=13, domain=ipsec-tunnel-flow, deny=true

hits=500, user_data=0x0, cs_id=0x0, flags=0x0, protocol=0

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0, dscp=0x0

input_ifc=outside, output_ifc=any

Phase: 8

Type: ACCESS-LIST

Subtype: log

Result: ALLOW

Config:

access-group db1_pci out interface inside

access-list db1_pci extended permit ip any any

Additional Information:

Forward Flow based lookup yields rule:

out id=0xcb254968, priority=13, domain=permit, deny=false

hits=2443, user_data=0xc9179ca0, cs_id=0x0, use_real_addr, flags=0x0, protocol=0

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0, dscp=0x0

input_ifc=any, output_ifc=inside

Phase: 9

Type: NAT

Subtype: rpf-check

Result: DROP

Config:

object network riverroad_inside

nat (inside,outside) dynamic interface

Additional Information:

Forward Flow based lookup yields rule:

out id=0xcb248478, priority=6, domain=nat-reverse, deny=false

hits=37, user_data=0xcb246c18, cs_id=0x0, use_real_addr, flags=0x0, protocol=0

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0

dst ip/id=192.168.16.0, mask=255.255.255.0, port=0, tag=0, dscp=0x0

input_ifc=outside, output_ifc=inside

Result:

input-interface: outside

input-status: up

input-line-status: up

output-interface: inside

output-status: up

output-line-status: up

Action: drop

Drop-reason: (acl-drop) Flow is denied by configured rule

Jouni Forss · ‎08-01-2013

Hi,

You need to use the public IP address NOT the real IP address as the destination.

You are simulating a packet coming from the Internet towards the server so its destination IP address will naturally be the public IP address.

So change the destination IP address in the "packet-tracer" command to the public IP address you are using.

- Jouni

jerry.henzel · ‎08-01-2013

Hi

Got you. Think I am getting brain-dead today.

Looks better. Sounds like it points to an ACL issue

Thanks

Jerry

ASA1(config-network-object)# packet-tracer input outside tcp 66.208.204.49 443 192.168.16.32 443

Phase: 1

Type: CAPTURE

Subtype:

Result: ALLOW

Config:

Additional Information:

MAC Access list

Phase: 2

Type: ACCESS-LIST

Subtype:

Result: ALLOW

Config:

Implicit Rule

Additional Information:

MAC Access list

Phase: 3

Type: ROUTE-LOOKUP

Subtype: input

Result: ALLOW

Config:

Additional Information:

in 192.168.16.0 255.255.255.0 inside

Phase: 4

Type: ACCESS-LIST

Subtype:

Result: DROP

Config:

Implicit Rule

Additional Information:

Result:

input-interface: outside

input-status: up

input-line-status: up

output-interface: inside

output-status: up

output-line-status: up

Action: drop

Drop-reason: (acl-drop) Flow is denied by configured rule

Jouni Forss · ‎08-01-2013

Hi,

The destination of the "packet-tracer" is still a Private IP address of 192.168.16.32

It should be whatever IP address is configured on the interface "outside"

- Jouni

jerry.henzel · ‎08-01-2013

Hi

You are saying I should be running packet-tracer input outside tcp 192.168.16.32 443 66.208.204.49 443

Thanks

Jerry

Jouni Forss · ‎08-01-2013

Hi,

You should be using some random source address that according to the ASAs routing table is located behind the "outside" interface.

As the destination address you should use the public IP address used for the server in the Static PAT configuration.

So something like

packet-tracer input outside tcp 1.1.1.1 12345 443

I am not sure what your actual server IP address is as you have masked that

object network webserver-tcp443

host xx.xx.xx.xx

When you are configuring Static PAT it should in the following format

object network WEB-TCP443

host

nat (inside,outside) static interface service tcp 443 443

Provided ofcourse the server is found behind "inside" interface.

- Jouni

jerry.henzel · ‎08-01-2013

Hi

So filling in the gaps then

my outside ip is 66.208.204.49

my inside ip for the webserver is 192.168.16.32

therefore: packet-tracer input outside tcp 1.1.1.1 12344 66.208.204.49 44

with the following config

object network webserver-tcp443

host 66.208.204.49 192.168.16.32 (edit typo)

description Webserver on DB1

nat (inside,outside) static interface service tcp https https

Thanks

Jerry

jerry.henzel · ‎08-01-2013

Hi

When I run that I get an ACL error

ASA1(config)# packet-tracer input outside tcp 1.1.1.1 12345 66.208.204.49 443

access-group outside_www in interface outside

access-list outside_www extended permit tcp any object webserver-tcp443 eq https

Jerry

Phase: 1

Type: CAPTURE

Subtype:

Result: ALLOW

Config:

Additional Information:

MAC Access list

Phase: 2

Type: ACCESS-LIST

Subtype:

Result: ALLOW

Config:

Implicit Rule

Additional Information:

MAC Access list

Phase: 3

Type: ROUTE-LOOKUP

Subtype: input

Result: ALLOW

Config:

Additional Information:

in 66.208.204.49 255.255.255.255 identity

Phase: 4

Type: NAT

Subtype: per-session

Result: ALLOW

Config:

Additional Information:

Phase: 5

Type: ACCESS-LIST

Subtype:

Result: DROP

Config:

Implicit Rule

Additional Information:

Result:

input-interface: outside

input-status: up

input-line-status: up

output-interface: NP Identity Ifc

output-status: up

output-line-status: up

Action: drop

Drop-reason: (acl-drop) Flow is denied by configured rule

Jouni Forss · ‎08-01-2013

Hi,

There should be a UN-NAT Phase as one of the very first phases.

So it seems to indicate that its not hitting the NAT rule still.

Can you remove this NAT rule

nat (any,outside) source dynamic any interface inactive

Even though its set as "inactive". Still its a NAT configuration you dont want to have at the very highest priority.

- Jouni

jerry.henzel · ‎08-01-2013

Hi

I removed it. Does not seem to have effect. I was playing with it from another example and then made it inactive. Do you think the order of some of the NAT rules might be causing this?

Thanks

Jerry

ASA1(config)# packet-tracer input outside tcp 1.1.1.1 12345 66.208.204.49 443

Phase: 1

Type: CAPTURE

Subtype:

Result: ALLOW

Config:

Additional Information:

MAC Access list

Phase: 2

Type: ACCESS-LIST

Subtype:

Result: ALLOW

Config:

Implicit Rule

Additional Information:

MAC Access list

Phase: 3

Type: ROUTE-LOOKUP

Subtype: input

Result: ALLOW

Config:

Additional Information:

in 66.208.204.49 255.255.255.255 identity

Phase: 4

Type: NAT

Subtype: per-session

Result: ALLOW

Config:

Additional Information:

Phase: 5

Type: ACCESS-LIST

Subtype:

Result: DROP

Config:

Implicit Rule

Additional Information:

Result:

input-interface: outside

input-status: up

input-line-status: up

output-interface: NP Identity Ifc

output-status: up

output-line-status: up

Action: drop

Drop-reason: (acl-drop) Flow is denied by configured rule

Jouni Forss · ‎08-02-2013

Hi,

The traffic simply aint matching any NAT rule on the way.

Looking at your Manual NAT rules in the configuration it seems to me that most are "static" configurations with "destination" parameters so I dont see how any of them could override this NAT

I guess we can determine if any other NAT rule is causing this by configuring the NAT in the following way. Remove the previous Network Object NAT for the TCP/443 before configuring this

object network SERVER

host 192.168.16.32

object service HTTPS

service tcp source eq 443

nat (inside,outside) 1 source static SERVER interface service HTTPS HTTPS

This should also do Static PAT configuration even though I personally dont use this configuration format.

- Jouni

jerry.henzel · ‎08-02-2013

Hi

That did the trick.

Thanks

Jerry