topic It definitely looks like the in Network Security

Port Forwarding ASA5505 packets dropping

staylor2112 — Tue, 12 Mar 2019 08:47:48 GMT

Outbound traffic works but inbound does not. I have checked my ACL and it looks correct, but I am still not seeing what I missed. I have attached our current running-config as well as a screenshot of the Packet Trace utility. Below is the detailed packet-tracer results.

Result of the command: "packet-tracer input outside tcp 8.8.8.8 888 192.168.1.88 888 detailed"

Phase: 1
Type: ROUTE-LOOKUP
Subtype: input
Result: ALLOW
Config:
Additional Information:
in   192.168.1.0     255.255.255.0   inside

Phase: 2
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group outside_access_in in interface outside
access-list outside_access_in extended permit object FrontelPort1 any object FRONTELSVR1
Additional Information:
Forward Flow based lookup yields rule:
in id=0xd003f6b8, priority=13, domain=permit, deny=false
   hits=7, user_data=0xca0e8a80, cs_id=0x0, use_real_addr, flags=0x0, protocol=6
   src ip/id=0.0.0.0, mask=0.0.0.0, port=888, tag=0
   dst ip/id=192.168.1.88, mask=255.255.255.255, port=888, tag=0 dscp=0x0
   input_ifc=outside, output_ifc=any

Phase: 3
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0xcba010a0, priority=0, domain=nat-per-session, deny=false
   hits=1193646144, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=6
   src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0
   dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0 dscp=0x0
   input_ifc=any, output_ifc=any

Phase: 4
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0xcc08d5a0, priority=0, domain=inspect-ip-options, deny=true
   hits=612753499, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
   src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0
   dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0 dscp=0x0
   input_ifc=outside, output_ifc=any

Phase: 5
Type: VPN
Subtype: ipsec-tunnel-flow
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0xcca5dc28, priority=13, domain=ipsec-tunnel-flow, deny=true
   hits=373410, user_data=0x0, cs_id=0x0, flags=0x0, protocol=0
   src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0
   dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0 dscp=0x0
   input_ifc=outside, output_ifc=any

Phase: 6
Type: NAT
Subtype: rpf-check
Result: DROP
Config:
object network FRONTELSVR1
nat (inside,outside) static interface service tcp 888 888
Additional Information:
Forward Flow based lookup yields rule:
out id=0xd06d4bb0, priority=6, domain=nat-reverse, deny=false
   hits=2, user_data=0xd04266d8, cs_id=0x0, use_real_addr, flags=0x0, protocol=6
   src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0
   dst ip/id=192.168.1.88, mask=255.255.255.255, port=888, tag=0 dscp=0x0
   input_ifc=outside, output_ifc=inside

Result:
input-interface: outside
input-status: up
input-line-status: up
output-interface: inside
output-status: up
output-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule

According to this statement,

Rahul Govindan — Wed, 18 Jan 2017 13:36:46 GMT

According to this statement,

object network FRONTELSVR1
 nat (inside,outside) static interface service tcp 888 888

Your server at 192.168.1.88 is translated to your outside interface ip address when tcp port 888 is used. Your packet tracer is failing because the ASA determines that there are asymmetric NAT rules being hit in the return path (hence reverse path failure or RPF). Your packet tracer should ideally be the below statement as outside users will hit the public ip address to access the server:

packet-tracer input outside tcp 8.8.8.8 888 <outside-interface-ip> 888 detailed

Thank you for the quick reply

staylor2112 — Wed, 18 Jan 2017 13:57:08 GMT

Thank you for the quick reply.

Its been a while since I worked in this environment, yes I can see the un-NAT and NAT when using the packet-tracer input outside tcp 8.8.8.8 888 <outside-interface-ip> 888 detailed statement.

Why would this stop us from being able to telnet into the server using the public address on port 888? (telnet xx.xx.xx.xx 888)

Telnet should work if the

Rahul Govindan — Wed, 18 Jan 2017 14:41:37 GMT

Telnet should work if the packet-tracer showed allow all the way through to the inside interface. Can you paste the output here?

You may want to apply a capture on the outside and inside interface to capture the entire transaction through the ASA. I would apply a capture like this:

cap capo interface outside match ip host <source ip address> any

cap capi interface inside match ip host <source ip address> any

Source ip address is the client's ip address on the internet.

There's quite a bit of nat in

emily00001 — Wed, 18 Jan 2017 14:45:04 GMT

There's quite a bit of nat in there but I believe you need to set it up with after-auto parameter as in the example below.

nat (inside,outside) after-auto source dynamic Inside-Network interface

That shows up after your specific nat rules for the port forwarding.

Hopefully I did this right:

staylor2112 — Wed, 18 Jan 2017 14:50:00 GMT

Hopefully I did this right:

Result of the command: "packet-tracer input outside tcp 8.8.8.8 888 xx.xx.xx.xx 888 detailed"

Phase: 1
Type: CAPTURE
Subtype:
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0xd0712e80, priority=13, domain=capture, deny=false
   hits=15533, user_data=0xd0712d40, cs_id=0x0, l3_type=0x0
   src mac=0000.0000.0000, mask=0000.0000.0000
   dst mac=0000.0000.0000, mask=0000.0000.0000
   input_ifc=outside, output_ifc=any

Phase: 2
Type: ACCESS-LIST
Subtype:
Result: ALLOW
Config:
Implicit Rule
Additional Information:
Forward Flow based lookup yields rule:
in id=0xcc087cc8, priority=1, domain=permit, deny=false
   hits=10642429583, user_data=0x0, cs_id=0x0, l3_type=0x8
   src mac=0000.0000.0000, mask=0000.0000.0000
   dst mac=0000.0000.0000, mask=0100.0000.0000
   input_ifc=outside, output_ifc=any

Phase: 3
Type: UN-NAT
Subtype: static
Result: ALLOW
Config:
object network FRONTELSVR1
nat (inside,outside) static interface service tcp 888 888
Additional Information:
NAT divert to egress interface inside
Untranslate xx.xx.xx.xx/888 to 192.168.1.88/888

Phase: 4
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group outside_access_in in interface outside
access-list outside_access_in extended permit object-group DM_INLINE_SERVICE_8 any object FRONTELSVR1
object-group service DM_INLINE_SERVICE_8
service-object object FrontelPort1
service-object tcp destination eq telnet
Additional Information:
Forward Flow based lookup yields rule:
in id=0xd003f6b8, priority=13, domain=permit, deny=false
   hits=0, user_data=0xca0e8a80, cs_id=0x0, use_real_addr, flags=0x0, protocol=6
   src ip/id=0.0.0.0, mask=0.0.0.0, port=888, tag=0
   dst ip/id=192.168.1.88, mask=255.255.255.255, port=888, tag=0 dscp=0x0
   input_ifc=outside, output_ifc=any

Phase: 5
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0xcba010a0, priority=0, domain=nat-per-session, deny=false
   hits=1197004907, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=6
   src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0
   dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0 dscp=0x0
   input_ifc=any, output_ifc=any

Phase: 6
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0xcc08d5a0, priority=0, domain=inspect-ip-options, deny=true
   hits=614473460, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
   src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0
   dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0 dscp=0x0
   input_ifc=outside, output_ifc=any

Phase: 7
Type: VPN
Subtype: ipsec-tunnel-flow
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0xcca5dc28, priority=13, domain=ipsec-tunnel-flow, deny=true
   hits=373713, user_data=0x0, cs_id=0x0, flags=0x0, protocol=0
   src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0
   dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0 dscp=0x0
   input_ifc=outside, output_ifc=any

Phase: 8
Type: NAT
Subtype: rpf-check
Result: ALLOW
Config:
object network FRONTELSVR1
nat (inside,outside) static interface service tcp 888 888
Additional Information:
Forward Flow based lookup yields rule:
out id=0xd06d4bb0, priority=6, domain=nat-reverse, deny=false
   hits=6, user_data=0xd04266d8, cs_id=0x0, use_real_addr, flags=0x0, protocol=6
   src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0
   dst ip/id=192.168.1.88, mask=255.255.255.255, port=888, tag=0 dscp=0x0
   input_ifc=outside, output_ifc=inside

Phase: 9
Type: USER-STATISTICS
Subtype: user-statistics
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
out id=0xcf077c90, priority=0, domain=user-statistics, deny=false
   hits=613755692, user_data=0xcf06e370, cs_id=0x0, reverse, flags=0x0, protocol=0
   src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0
   dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0 dscp=0x0
   input_ifc=any, output_ifc=inside

Phase: 10
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:
Reverse Flow based lookup yields rule:
in id=0xcba010a0, priority=0, domain=nat-per-session, deny=false
   hits=1197004909, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=6
   src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0
   dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0 dscp=0x0
   input_ifc=any, output_ifc=any

Phase: 11
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Reverse Flow based lookup yields rule:
in id=0xcc064370, priority=0, domain=inspect-ip-options, deny=true
   hits=614036763, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
   src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0
   dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0 dscp=0x0
   input_ifc=inside, output_ifc=any

Phase: 12
Type: USER-STATISTICS
Subtype: user-statistics
Result: ALLOW
Config:
Additional Information:
Reverse Flow based lookup yields rule:
out id=0xcf077ef8, priority=0, domain=user-statistics, deny=false
   hits=614737321, user_data=0xcf06e370, cs_id=0x0, reverse, flags=0x0, protocol=0
   src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0
   dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0 dscp=0x0
   input_ifc=any, output_ifc=outside

Phase: 13
Type: FLOW-CREATION
Subtype:
Result: ALLOW
Config:
Additional Information:
New flow created with id 614835307, packet dispatched to next module
Module information for forward flow ...
snp_fp_tracer_drop
snp_fp_inspect_ip_options
snp_fp_tcp_normalizer
snp_fp_translate
snp_fp_adjacency
snp_fp_fragment
snp_ifc_stat

Module information for reverse flow ...
snp_fp_tracer_drop
snp_fp_inspect_ip_options
snp_fp_translate
snp_fp_tcp_normalizer
snp_fp_adjacency
snp_fp_fragment
snp_ifc_stat

Result:
input-interface: outside
input-status: up
input-line-status: up
output-interface: inside
output-status: up
output-line-status: up
Action: allow

Packet-tracer looks good. Can

Rahul Govindan — Wed, 18 Jan 2017 15:32:54 GMT

Packet-tracer looks good. Can you check if the capture on inside and outside interface show 2 way traffic?

How can I check?

staylor2112 — Wed, 18 Jan 2017 15:34:10 GMT

How can I check?

Mentioned it in an earlier

Rahul Govindan — Wed, 18 Jan 2017 15:40:39 GMT

Mentioned it in an earlier post:

You may want to apply a capture on the outside and inside interface to capture the entire transaction through the ASA. I would apply a capture like this:

cap capo interface outside match ip host <source ip address> any

cap capi interface inside match ip host <source ip address> any

Source ip address is the client's ip address on the internet. After sending traffic, check the captures using:

show capture capo
show capture capi

More info on capture here:

http://www.cisco.com/c/en/us/support/docs/security/asa-5500-x-series-next-generation-firewalls/118097-configure-asa-00.html

I get 3 capi responses on

staylor2112 — Wed, 18 Jan 2017 16:43:53 GMT

I get 3 capi responses on port 888 from an internal telnet to out outside address, but nothing on the capo for port 888

Result of the command: "show capture capi"

3 packets captured

1: 08:15:29.396631       802.1Q vlan#1 P0 192.168.1.148.64584 > xx.xx.xx.xx.888: S 820795542:820795542(0) win 8192 <mss 1460,nop,wscale 8,nop,nop,sackOK>
2: 08:15:32.395853       802.1Q vlan#1 P0 192.168.1.148.64584 > xx.xx.xx.xx.888: S 820795542:820795542(0) win 8192 <mss 1460,nop,wscale 8,nop,nop,sackOK>
3: 08:15:38.395990       802.1Q vlan#1 P0 192.168.1.148.64584 > xx.xx.xx.xx.888: S 820795542:820795542(0) win 8192 <mss 1460,nop,nop,sackOK>
3 packets shown

Are you trying to access the

Rahul Govindan — Wed, 18 Jan 2017 20:52:28 GMT

Are you trying to access the server from outside or from inside? From your captures looks like the source is an internal ip address (192.168.1.148) and destination is the ASA public ip address on the outside (xx.xx.xx.xx). This wont work, you cannot access resources on another interface of the ASA while coming in from a different interface.

Maybe I have mistaken your topology, but you had mentioned that access to the server was not working from the outside correct. Can you send some traffic from a host on the internet to the ASA's public ip address on port 888? Clear the captures using command "clear cap /all" before you test it.

From the outside. I was

staylor2112 — Wed, 18 Jan 2017 23:19:27 GMT

From the outside. I was testing from the inside thinking it would work. However, we cant connect even connect from another public address.

Could you capture the packets

Rahul Govindan — Thu, 19 Jan 2017 07:05:34 GMT

Could you capture the packets when testing from outside? Your config looks ok and packet-tracer allows the packet to go all the way from outside to inside. We need to see if packets are actually being blocked by the ASA or if there is another reason it is getting dropped.

To make sure I set up the

staylor2112 — Thu, 19 Jan 2017 14:27:58 GMT

To make sure I set up the capture correctly:

no capture capo

no capture capi

cap capo interface outside match ip host 104.137.90.176 any

cap capi interface inside match ip host 104.137.90.176 any

clear capture /all

Where 104.137.90.176 is the external address I will be testing from. (Should I put anything in the any place?)

Then I would be sending telnet xx.xx.xx.xx 888 from 104.137.90.176 to my public IP address xx.xx.xx.xx. Is this correct?

I am not seeing any traffic

staylor2112 — Thu, 19 Jan 2017 17:08:51 GMT

I am not seeing any traffic on port 888. It looks like only traffic for Teamviewer from my home PC to my work PC.

I am confused as to why I am not even seeing the telnet attempt on port 888. Any additional thoughts?

Not sure, we should see at

Rahul Govindan — Fri, 20 Jan 2017 01:38:17 GMT

Not sure, we should see at least the TCP syn packet reaching the ASA from your ip address. Could it be possible that your ISP or gateway router is not allowing the port 888 across?

I can check in the morning. I

staylor2112 — Fri, 20 Jan 2017 02:04:12 GMT

I can check in the morning. I know at the office end (outside interface) is open. I will check from a different ISP/Location. Also, was my capture setup correct? (see above)

Yes, capture was correct. I

Rahul Govindan — Fri, 20 Jan 2017 03:14:13 GMT

Yes, capture was correct. I would recommend the following capture so that you do not get your teamviewer packets in the capture also:

no capture capo

no capture capi

cap capo interface outside match tcp host <your public ip> host <ASA public ip> eq 888

cap capi interface inside match tcp host <your public ip> host 192.168.1.88 eq 888

clear capture /all

This will only capture the traffic to the server from your PC.

Looks like the ISP is not

staylor2112 — Fri, 20 Jan 2017 13:28:23 GMT

Looks like the ISP is not blocking port 888, but the packet is not showing on the inside capture.

Result of the command: "show capture capo"

1: 07:22:29.641064       802.1Q vlan#2 P0 104.137.90.176.60744 > xx.xx.xx.xx.888: S 3035968005:3035968005(0) win 8192 <mss 1260,nop,wscale 8,nop,nop,sackOK>
   2: 07:22:32.641202       802.1Q vlan#2 P0 104.137.90.176.60744 > xx.xx.xx.xx.888: S 3035968005:3035968005(0) win 8192 <mss 1260,nop,wscale 8,nop,nop,sackOK>
   3: 07:22:38.641904       802.1Q vlan#2 P0 104.137.90.176.60744 > xx.xx.xx.xx.888: S 3035968005:3035968005(0) win 8192 <mss 1260,nop,nop,sackOK>
   4: 07:22:50.692942       802.1Q vlan#2 P0 104.137.90.176.60744 > xx.xx.xx.xx.888: R 0:0(0) win 0
4 packets shown

Result of the command: "show capture capi"

0 packet captured

0 packet shown

Strange. Considering both the

Rahul Govindan — Fri, 20 Jan 2017 23:40:47 GMT

Strange. Considering both the captures are correctly configured, looks like the ASA is not forwarding the packet to the inside interface. Can you attach the "show capture" and packet-tracer output to confirm that nothing has changed from before.

A few other troubleshooting steps you can try:

1) check the local buffer syslog for anything regarding this transaction that could indicate that the ASA dropped it.

2) Apply an ASP drop capture - this will capture all the packets the ASA drops. Steps to do this are:

a) Command "cap capasp type asp all"

b) Send traffic

c) show cap capasp | in <your public ip address>

If you are seeing this, you may get a reason for the drop in the last output. This would help in investigation on the packet drop.