CIsco ASA Firewall NAT services not working

[vitumbiko nkhwazi]

Hello.

We have a Cisco ASA firewall that is running  ASA version14(4)23. This firewall is meant to replace our current firewall, we have done all the configurations including NAT and Access lists, when we connect the ASA into production, all outgoing traffic is working properly (We can browse the Internet), however, on the incoming services that we have NATed to internal private address, only one service is working and the rest are not able to connect. Our NAT and Access-list configurations look to be good but there is something preventing these services from connecting, we have tried upgrading the firmware but it did not solve the problem. What could be the issue and how can we troubleshoot why the incoming NATed services are not woking?

Regards.

[MHM Cisco World]

Can yoh share the packet tracer of other asa' I need to compare.

Thanks 

MHM

[vitumbiko nkhwazi]

Note that this one is using an older ASA version 8.2

NBS-ASA-OUTSIDE# packet-tracer input outside tcp 41.21.36.145 1234 102.36.145.$

Phase: 1
Type: FLOW-LOOKUP
Subtype:
Result: ALLOW
Config:
Additional Information:
Found no matching flow, creating a new flow

Phase: 2
Type: UN-NAT
Subtype: static
Result: ALLOW
Config:
static (DMZ,Outside) tcp 102.36.145.8 6443 10.40.129.212 6443 netmask 255.255.255.255
match tcp DMZ host 10.40.129.212 eq 6443 Outside any
static translation to 102.36.145.8/6443
translate_hits = 1887, untranslate_hits = 455852
Additional Information:
NAT divert to egress interface DMZ
Untranslate 102.36.145.8/6443 to 10.40.129.212/6443 using netmask 255.255.255.255

Phase: 3
Type: ROUTE-LOOKUP
Subtype: input
Result: ALLOW
Config:
Additional Information:
in 0.0.0.0 0.0.0.0 Outside

Phase: 4
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group OUTSIDE-IN in interface Outside
access-list OUTSIDE-IN extended permit tcp any host 102.36.145.8 eq 6443
Additional Information:
Forward Flow based lookup yields rule:
in id=0xabb08770, priority=12, domain=permit, deny=false
hits=453026, user_data=0xa89f2bc0, cs_id=0x0, flags=0x0, protocol=6
src ip=0.0.0.0, mask=0.0.0.0, port=0
dst ip=102.36.145.8, mask=255.255.255.255, port=6443, dscp=0x0

Phase: 5
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0xab7de220, priority=0, domain=permit-ip-option, deny=true
hits=13962431, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip=0.0.0.0, mask=0.0.0.0, port=0
dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 6
Type:
Subtype:
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0xaeb835b8, priority=17, domain=flow-export, deny=false
hits=2163800, user_data=0xac7201b0, cs_id=0x0, flags=0x0, protocol=0
src ip=0.0.0.0, mask=0.0.0.0, port=0
dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 7
Type: VPN
Subtype: ipsec-tunnel-flow
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0xac5531c0, priority=12, domain=ipsec-tunnel-flow, deny=true
hits=1263350, user_data=0x0, cs_id=0x0, flags=0x0, protocol=0
src ip=0.0.0.0, mask=0.0.0.0, port=0
dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 8
Type: NAT
Subtype: rpf-check
Result: ALLOW
Config:
static (DMZ,Outside) tcp 102.36.145.8 6443 10.40.129.212 6443 netmask 255.255.255.255
match tcp DMZ host 10.40.129.212 eq 6443 Outside any
static translation to 102.36.145.8/6443
translate_hits = 1887, untranslate_hits = 455875
Additional Information:
Forward Flow based lookup yields rule:
out id=0xaba9d7c8, priority=5, domain=nat-reverse, deny=false
hits=465138, user_data=0xaba9d330, cs_id=0x0, flags=0x0, protocol=6
src ip=0.0.0.0, mask=0.0.0.0, port=0
dst ip=10.40.129.212, mask=255.255.255.255, port=6443, dscp=0x0

Phase: 9
Type: NAT
Subtype: host-limits
Result: ALLOW
Config:
static (DMZ,Outside) tcp 102.36.145.8 6443 10.40.129.212 6443 netmask 255.255.255.255
match tcp DMZ host 10.40.129.212 eq 6443 Outside any
static translation to 102.36.145.8/6443
translate_hits = 1887, untranslate_hits = 455880
Additional Information:
Reverse Flow based lookup yields rule:
in id=0xaba9d970, priority=5, domain=host, deny=false
hits=697255, user_data=0xaba9d330, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip=10.40.129.212, mask=255.255.255.255, port=0
dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 10
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Reverse Flow based lookup yields rule:
in id=0xab8645f0, priority=0, domain=permit-ip-option, deny=true
hits=13250111, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip=0.0.0.0, mask=0.0.0.0, port=0
dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 11
Type: FLOW-CREATION
Subtype:
Result: ALLOW
Config:
Additional Information:
New flow created with id 13696903, packet dispatched to next module
Module information for forward flow ...
snp_fp_tracer_drop
snp_fp_inspect_ip_options
snp_fp_tcp_normalizer
snp_fp_translate
snp_fp_adjacency
snp_fp_fragment
snp_ifc_stat

Module information for reverse flow ...
snp_fp_tracer_drop
snp_fp_inspect_ip_options
snp_fp_translate
snp_fp_tcp_normalizer
snp_fp_adjacency
snp_fp_fragment
snp_ifc_stat

Result:
input-interface: Outside
input-status: up
input-line-status: up
output-interface: Outside
output-status: up
output-line-status: up
Action: allow

[Flavio Miranda]

Hi

 I dont know how did you build the NAT but if one service is working chances are the problem is not NAT.  If you telnet from outside to the TCP port what do you see on the firewall logs? 

[MHM Cisco World]

I test by my self and this packet tracer from my lab
packet tracer input OUTSIDE tcp <select any IP from the outside subnet> 1234 <mapped IP of server> 7412 detail

select any IP from the outside subnet <<- the IP must not be ASA IP interface, if you use ASA IP use other and share the packter tracer 

ciscoasa# packet-tracer input OUT tcp 100.0.0.100 1234 100.0.0.50 23 detailed

Phase: 1
Type: UN-NAT
Subtype: static
Result: ALLOW
Config:
nat (IN,OUT) source static telnet-ip telnet-map
Additional Information:
NAT divert to egress interface IN
Untranslate 100.0.0.50/23 to 10.0.0.50/23

Phase: 2
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group telnet in interface OUT
access-list telnet extended permit tcp any host 10.0.0.50 eq telnet
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7fe08d9b6e30, priority=13, domain=permit, deny=false
hits=0, user_data=0x7fe086496d00, cs_id=0x0, use_real_addr, flags=0x0, protocol=6
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=10.0.0.50, mask=255.255.255.255, port=23, tag=any, dscp=0x0
input_ifc=OUT, output_ifc=any

Phase: 3
Type: NAT
Subtype:
Result: ALLOW
Config:
nat (IN,OUT) source static telnet-ip telnet-map
Additional Information:
Static translate 100.0.0.100/1234 to 100.0.0.100/1234
Forward Flow based lookup yields rule:
in id=0x7fe08d9b5fd0, priority=6, domain=nat, deny=false
hits=0, user_data=0x7fe08d9b4de0, cs_id=0x0, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=100.0.0.50, mask=255.255.255.255, port=0, tag=any, dscp=0x0
input_ifc=OUT, output_ifc=IN

Phase: 4
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7fe08d5df380, priority=1, domain=nat-per-session, deny=true
hits=2, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=6
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=any, output_ifc=any

Phase: 5
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7fe08d93d540, priority=0, domain=inspect-ip-options, deny=true
hits=0, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=OUT, output_ifc=any

Phase: 6
Type: QOS
Subtype:
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7fe08da6bcc0, priority=70, domain=qos-per-class, deny=false
hits=1, user_data=0x7fe08d80e660, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=any, output_ifc=any

Phase: 7
Type: VPN
Subtype: ipsec-tunnel-flow
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7fe08d9e4ec0, priority=13, domain=ipsec-tunnel-flow, deny=true
hits=1, user_data=0x0, cs_id=0x0, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=OUT, output_ifc=any

Phase: 8
Type: NAT
Subtype: rpf-check
Result: ALLOW
Config:
nat (IN,OUT) source static telnet-ip telnet-map
Additional Information:
Forward Flow based lookup yields rule:
out id=0x7fe08d9b6400, priority=6, domain=nat-reverse, deny=false
hits=1, user_data=0x7fe08d9b4ce0, cs_id=0x0, use_real_addr, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=10.0.0.50, mask=255.255.255.255, port=0, tag=any, dscp=0x0
input_ifc=OUT, output_ifc=IN

Phase: 9
Type: QOS
Subtype:
Result: ALLOW
Config:
Additional Information:
Reverse Flow based lookup yields rule:
in id=0x7fe08da6bcc0, priority=70, domain=qos-per-class, deny=false
hits=2, user_data=0x7fe08d80e660, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=any, output_ifc=any

Phase: 10
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:
Reverse Flow based lookup yields rule:
in id=0x7fe08d5df380, priority=1, domain=nat-per-session, deny=true
hits=4, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=6
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=any, output_ifc=any

Phase: 11
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Reverse Flow based lookup yields rule:
in id=0x7fe08d9a91c0, priority=0, domain=inspect-ip-options, deny=true
hits=3, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=IN, output_ifc=any

Phase: 12
Type: FLOW-CREATION
Subtype:
Result: ALLOW
Config:
Additional Information:
New flow created with id 2, packet dispatched to next module
Module information for forward flow ...
snp_fp_tracer_drop
snp_fp_inspect_ip_options
snp_fp_tcp_normalizer
snp_fp_translate
snp_fp_adjacency
snp_fp_fragment
snp_ifc_stat

Module information for reverse flow ...
snp_fp_tracer_drop
snp_fp_inspect_ip_options
snp_fp_translate
snp_fp_tcp_normalizer
snp_fp_adjacency
snp_fp_fragment
snp_ifc_stat

Result:
input-interface: OUT
input-status: up
input-line-status: up
output-interface: IN
output-status: up
output-line-status: up
Action: allow

[vitumbiko nkhwazi]

NBS-BT-INTERNET-ASA5525# packet-tracer input OUTSIDE tcp 102.36.145.100 1234 1$

Phase: 1
Type: UN-NAT
Subtype: static
Result: ALLOW
Config:
nat (INSIDE,OUTSIDE) source static AMEYO_SERVER 102.36.145.13 service tcp_7412 tcp_7412
Additional Information:
NAT divert to egress interface INSIDE
Untranslate 102.36.145.13/7412 to 10.40.129.50/7412

Phase: 2
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group OUTSIDE in interface OUTSIDE
access-list OUTSIDE remark AMEYO CALL CENTER
access-list OUTSIDE extended permit tcp any object AMEYO_SERVER object-group AMEYO_PORTS
object-group service AMEYO_PORTS tcp
port-object eq 7412
port-object eq 8443
port-object eq 8080
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7f5cdc41fc50, priority=13, domain=permit, deny=false
hits=0, user_data=0x7f5cd13b2740, cs_id=0x0, use_real_addr, flags=0x0, protocol=6
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=10.40.129.50, mask=255.255.255.255, port=7412, tag=any, dscp=0x0
input_ifc=OUTSIDE, output_ifc=any

Phase: 3
Type: NAT
Subtype:
Result: ALLOW
Config:
nat (INSIDE,OUTSIDE) source static AMEYO_SERVER 102.36.145.13 service tcp_7412 tcp_7412
Additional Information:
Static translate 102.36.145.100/1234 to 102.36.145.100/1234
Forward Flow based lookup yields rule:
in id=0x7f5cdc6789c0, priority=6, domain=nat, deny=false
hits=0, user_data=0x7f5cdc6779b0, cs_id=0x0, flags=0x0, protocol=6
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=102.36.145.13, mask=255.255.255.255, port=7412, tag=any, dscp=0x0
input_ifc=OUTSIDE, output_ifc=INSIDE

Phase: 4
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7f5cdb5891a0, priority=0, domain=nat-per-session, deny=false
hits=93852, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=6
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=any, output_ifc=any

Phase: 5
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7f5cdc439a80, priority=0, domain=inspect-ip-options, deny=true
hits=69568, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=OUTSIDE, output_ifc=any

Phase: 6
Type: VPN
Subtype: ipsec-tunnel-flow
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7f5cdeccc400, priority=13, domain=ipsec-tunnel-flow, deny=true
hits=9703, user_data=0x0, cs_id=0x0, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=OUTSIDE, output_ifc=any

Phase: 7
Type: NAT
Subtype: rpf-check
Result: ALLOW
Config:
nat (INSIDE,OUTSIDE) source static AMEYO_SERVER 102.36.145.13 service tcp_7412 tcp_7412
Additional Information:
Forward Flow based lookup yields rule:
out id=0x7f5cdc678da0, priority=6, domain=nat-reverse, deny=false
hits=1, user_data=0x7f5cdc670270, cs_id=0x0, use_real_addr, flags=0x0, protocol=6
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=10.40.129.50, mask=255.255.255.255, port=7412, tag=any, dscp=0x0
input_ifc=OUTSIDE, output_ifc=INSIDE

Phase: 8
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:
Reverse Flow based lookup yields rule:
in id=0x7f5cdb5891a0, priority=0, domain=nat-per-session, deny=false
hits=93854, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=6
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=any, output_ifc=any

Phase: 9
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Reverse Flow based lookup yields rule:
in id=0x7f5cdc3d1ba0, priority=0, domain=inspect-ip-options, deny=true
hits=66129, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=INSIDE, output_ifc=any

Phase: 10
Type: FLOW-CREATION
Subtype:
Result: ALLOW
Config:
Additional Information:
New flow created with id 73637, packet dispatched to next module
Module information for forward flow ...
snp_fp_inspect_ip_options
snp_fp_tcp_normalizer
snp_fp_translate
snp_fp_adjacency
snp_fp_fragment
snp_fp_tracer_drop
snp_ifc_stat

Module information for reverse flow ...
snp_fp_inspect_ip_options
snp_fp_translate
snp_fp_tcp_normalizer
snp_fp_adjacency
snp_fp_fragment
snp_fp_tracer_drop
snp_ifc_stat

Phase: 11
Type: INPUT-ROUTE-LOOKUP-FROM-OUTPUT-ROUTE-LOOKUP
Subtype: Resolve Preferred Egress interface
Result: ALLOW
Config:
Additional Information:
Found next-hop 10.40.139.81 using egress ifc INSIDE

Phase: 12
Type: ADJACENCY-LOOKUP
Subtype: Resolve Nexthop IP address to MAC
Result: ALLOW
Config:
Additional Information:
Found adjacency entry for Next-hop 10.40.139.81 on interface INSIDE
Adjacency :Active
MAC address 0000.0c07.ac82 hits 10478 reference 2

Result:
input-interface: OUTSIDE
input-status: up
input-line-status: up
output-interface: INSIDE
output-status: up
output-line-status: up
Action: allow