Unable to ping DMZ from Outside

AFlack20 · ‎04-06-2021

For some reason I cannot ping my DMZ IP from outside through static NAT. Packet capture shows the success ping from dmz out but failure on outside in. Packet trace shows that it should be successful. Included is all the relevant running config.

# show cap ICMP

18 packets captured

1: 15:30:52.301299 X.X.X.70 > 8.8.8.8 icmp: echo request
2: 15:30:52.348065 8.8.8.8 > X.X.X.70 icmp: echo reply
3: 15:30:52.349133 X.X.X.70 > 8.8.8.8 icmp: echo request
4: 15:30:52.396006 8.8.8.8 > X.X.X.70 icmp: echo reply
5: 15:30:52.397013 X.X.X.70 > 8.8.8.8 icmp: echo request
6: 15:30:52.444038 8.8.8.8 > X.X.X.70 icmp: echo reply
7: 15:30:52.445014 X.X.X.70 > 8.8.8.8 icmp: echo request
8: 15:30:52.492009 8.8.8.8 > X.X.X.70 icmp: echo reply
9: 15:30:52.492955 X.X.X.70 > 8.8.8.8 icmp: echo request
10: 15:30:52.540026 8.8.8.8 > X.X.X.70 icmp: echo reply
11: 15:33:16.6X481 X.X.X.73 > X.X.X.70 icmp: echo request
12: 15:33:16.673671 X.X.X.70 > X.X.X.73 icmp: host X.X.X.70 unreachable - admin prohibited filter
13: 15:33:21.521625 X.X.X.73 > X.X.X.70 icmp: echo request
14: 15:33:21.522662 X.X.X.70 > X.X.X.73 icmp: host X.X.X.70 unreachable - admin prohibited filter
15: 15:33:26.517490 X.X.X.73 > X.X.X.70 icmp: echo request
16: 15:33:26.518X6 X.X.X.70 > X.X.X.73 icmp: host X.X.X.70 unreachable - admin prohibited filter
17: 15:33:31.515979 X.X.X.73 > X.X.X.70 icmp: echo request
18: 15:33:31.517093 X.X.X.70 > X.X.X.73 icmp: host X.X.X.70 unreachable - admin prohibited filter
18 packets shown

# packet-tracer input OUTSIDE icmp X.X.X.73 0 0 X.X.X.70

Phase: 1
Type: CAPTURE
Subtype:
Result: ALLOW
Config:
Additional Information:
MAC Access list

Phase: 2
Type: ACCESS-LIST
Subtype:
Result: ALLOW
Config:
Implicit Rule
Additional Information:
MAC Access list

Phase: 3
Type: UN-NAT
Subtype: static
Result: ALLOW
Config:
object network DMZ
nat (DMZ,OUTSIDE) static X.X.X.70
Additional Information:
NAT divert to egress interface DMZ
Untranslate X.X.X.70/0 to 192.168.244.2/0

Phase: 4
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group OUTSIDE_access_in in interface OUTSIDE
access-list OUTSIDE_access_in extended permit icmp any4 object DMZ
Additional Information:

Phase: 5
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:

Phase: 6
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:

Phase: 7
Type: SFR
Subtype:
Result: ALLOW
Config:
class-map firePOWER-class
description class to send all traffic to the Firepower module
match any
policy-map global_policy
class firePOWER-class
sfr fail-open
service-policy global_policy global
Additional Information:

Phase: 8
Type: INSPECT
Subtype: np-inspect
Result: ALLOW
Config:
class-map inspection_default
match default-inspection-traffic
policy-map global_policy
class inspection_default
inspect icmp
service-policy global_policy global
Additional Information:

Phase: 9
Type: INSPECT
Subtype: np-inspect
Result: ALLOW
Config:
Additional Information:

Phase: 10
Type: VPN
Subtype: ipsec-tunnel-flow
Result: ALLOW
Config:
Additional Information:

Phase: 11
Type: NAT
Subtype: rpf-check
Result: ALLOW
Config:
object network DMZ
nat (DMZ,OUTSIDE) static X.X.X.70
Additional Information:

Phase: 12
Type: FLOW-CREATION
Subtype:
Result: ALLOW
Config:
Additional Information:
New flow created with id 355283, packet dispatched to next module

Result:
input-interface: OUTSIDE
input-status: up
input-line-status: up
output-interface: DMZ
output-status: up
output-line-status: up
Action: allow

# show run

object network DMZ
host 192.168.244.2

access-list OUTSIDE_access_in extended permit icmp any4 object DMZ
access-list OUTSIDE_access_in extended deny ip any any

access-group OUTSIDE_access_in in interface OUTSIDE

object network DMZ
nat (DMZ,OUTSIDE) static X.X.X.70

Sheraz.Salim · ‎04-07-2021

you nat rules look good also capture show the data too. I just tune your rule and added the echo command at the end.

object network DMZ
host 192.168.244.2
nat (DMZ,OUTSIDE) static X.X.X.70
!
access-list OUTSIDE_access_in extended permit icmp any4 object DMZ eq echo
access-list OUTSIDE_access_in extended deny ip any any
!
access-group OUTSIDE_access_in in interface OUTSIDE

could you confirm if youu can ping the DMZ server from the ASA?

could you do a packet-tracer and show us the output

packet-tracer input OUTSIDE icmp 8.8.8 8 0 X.X.X.70 detail x.x.x.x.70 this has to be the outside interface ip address.

please do not forget to rate.

AFlack20 · ‎04-07-2021

Couple of things to note, interface Gi0/0 is OUTSIDE, interface Gi0/1 is DMZ, interface Gi0/2 is INSIDE. Outside interface is configured for PAT on the X.X.X.145 for the rest of the production network. This is why there's a static NAT for the X.X.X.70 to the OUTSIDE interface as seen in my original post. Both the .70 and .145 are in the same /24 subnet using .1 as the gateway.

For the access-list I've added the permit echo, and echo-reply above the deny any any. This did not resolve the issues, as I still can't ping the box from the outside using its NAT'd external IP.

Pinging from the DMZ IP 192.168.244.1 to the 192.168.244.2 didn't work either. A packet capture from the DMZ interface is showing the failure of the ping. I've also included a show route to show that they're directly connected, and from a layer one perspective I can personally confirm they're directly connected as well, i.e. no switch in-between. The strange thing is, that I can ping from the box at 192.168.244.2 to the 244.1 successfully (shown separately below).

Lastly I've included the output of the packet-tracer as requested with detailed output.

# show int ip bri
Interface                  IP-Address      OK? Method Status                Protocol
GigabitEthernet0/0         XXX.XXX.XXX.145 YES manual up                    up  
GigabitEthernet0/1         192.168.244.1   YES CONFIG up                    up  
GigabitEthernet0/2         192.168.244.5   YES manual up                    up  

access-list OUTSIDE_access_in extended permit icmp any4 object DMZ-CUBE_SIP 
access-list OUTSIDE_access_in extended permit icmp any4 object DMZ-CUBE_SIP echo 
access-list OUTSIDE_access_in extended permit icmp any4 object DMZ-CUBE_SIP echo-reply 
access-list OUTSIDE_access_in extended deny ip any any log 

# ping DMZ 192.168.244.2    
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.244.2, timeout is 2 seconds:
?????
Success rate is 0 percent (0/5)

# show cap ICMP_DMZ     

10 packets captured

   1: 13:11:26.946377       192.168.244.1 > 192.168.244.2 icmp: echo request 
   2: 13:11:26.946926       192.168.244.2 > 192.168.244.1 icmp: host 192.168.244.2 unreachable - admin prohibited filter 
   3: 13:11:28.941601       192.168.244.1 > 192.168.244.2 icmp: echo request 
   4: 13:11:28.942059       192.168.244.2 > 192.168.244.1 icmp: host 192.168.244.2 unreachable - admin prohibited filter 
   5: 13:11:30.941647       192.168.244.1 > 192.168.244.2 icmp: echo request 
   6: 13:11:30.942166       192.168.244.2 > 192.168.244.1 icmp: host 192.168.244.2 unreachable - admin prohibited filter 
   7: 13:11:32.941769       192.168.244.1 > 192.168.244.2 icmp: echo request 
   8: 13:11:32.942257       192.168.244.2 > 192.168.244.1 icmp: host 192.168.244.2 unreachable - admin prohibited filter 
   9: 13:11:34.941967       192.168.244.1 > 192.168.244.2 icmp: echo request 
  10: 13:11:34.942486       192.168.244.2 > 192.168.244.1 icmp: host 192.168.244.2 unreachable - admin prohibited filter 
10 packets shown

# show route

Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2, V - VPN
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, + - replicated route
Gateway of last resort is X.X.X.1 to network 0.0.0.0

C        192.168.244.0 255.255.255.252 is directly connected, DMZ
L        192.168.244.1 255.255.255.255 is directly connected, DMZ
C        192.168.244.4 255.255.255.252 is directly connected, INSIDE
L        192.168.244.5 255.255.255.255 is directly connected, INSIDE

# packet-tracer input OUTSIDE icmp 8.8.8.8 0 0 X.X.X.70 detailed 

Phase: 1
Type: CAPTURE
Subtype: 
Result: ALLOW
Config:
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x7fee753e5bc0, priority=13, domain=capture, deny=false
        hits=27679233, user_data=0x7fee68121220, cs_id=0x0, l3_type=0x0
        src mac=0000.0000.0000, mask=0000.0000.0000
        dst mac=0000.0000.0000, mask=0000.0000.0000
        input_ifc=OUTSIDE, output_ifc=any

Phase: 2
Type: ACCESS-LIST
Subtype: 
Result: ALLOW
Config:
Implicit Rule
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x7fee6685ccb0, priority=1, domain=permit, deny=false
        hits=42445934, user_data=0x0, cs_id=0x0, l3_type=0x8
        src mac=0000.0000.0000, mask=0000.0000.0000
        dst mac=0000.0000.0000, mask=0100.0000.0000
        input_ifc=OUTSIDE, output_ifc=any

Phase: 3
Type: UN-NAT
Subtype: static
Result: ALLOW
Config:
object network DMZ
 nat (DMZ,OUTSIDE) static X.X.X.70
Additional Information:
NAT divert to egress interface DMZ
Untranslate X.X.X.70/0 to 192.168.244.2/0

Phase: 4
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group OUTSIDE_access_in in interface OUTSIDE
access-list OUTSIDE_access_in extended permit icmp any4 object DMZ 
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x7fee67e46360, priority=13, domain=permit, deny=false
        hits=60, user_data=0x7fee5c7f8100, cs_id=0x0, use_real_addr, flags=0x0, protocol=1
        src ip/id=0.0.0.0, mask=0.0.0.0, icmp-type=0, tag=any
        dst ip/id=192.168.244.2, mask=255.255.255.255, icmp-code=0, tag=any, dscp=0x0
        input_ifc=OUTSIDE, output_ifc=any

Phase: 5
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x7fee65b97780, priority=0, domain=nat-per-session, deny=true
        hits=495150, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=0
        src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
        dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
        input_ifc=any, output_ifc=any

Phase: 6
Type: IP-OPTIONS
Subtype: 
Result: ALLOW
Config:
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x7fee66864fd0, priority=0, domain=inspect-ip-options, deny=true
        hits=642181, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
        src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
        dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
        input_ifc=OUTSIDE, output_ifc=any

Phase: 7
Type: SFR
Subtype: 
Result: ALLOW
Config:
class-map firePOWER-class
 description class to send all traffic to the Firepower module
 match any
policy-map global_policy
 class firePOWER-class
  sfr fail-open
service-policy global_policy global
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x7fee74f78ff0, priority=71, domain=sfr, deny=false
        hits=167249, user_data=0x7fee6a5c8aa0, cs_id=0x0, use_real_addr, flags=0x0, protocol=0
        src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
        dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
        input_ifc=OUTSIDE, output_ifc=any

Phase: 8
Type: INSPECT
Subtype: np-inspect
Result: ALLOW
Config:
class-map inspection_default
 match default-inspection-traffic
policy-map global_policy
 class inspection_default
  inspect icmp 
service-policy global_policy global
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x7fee74f75cf0, priority=70, domain=inspect-icmp, deny=false
        hits=3121, user_data=0x7fee74f3cb70, cs_id=0x0, use_real_addr, flags=0x0, protocol=1
        src ip/id=0.0.0.0, mask=0.0.0.0, icmp-type=0, tag=any
        dst ip/id=0.0.0.0, mask=0.0.0.0, icmp-code=0, tag=any, dscp=0x0
        input_ifc=OUTSIDE, output_ifc=any

Phase: 9
Type: INSPECT
Subtype: np-inspect
Result: ALLOW
Config:
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x7fee668647e0, priority=66, domain=inspect-icmp-error, deny=false
        hits=4036, user_data=0x7fee66864490, cs_id=0x0, use_real_addr, flags=0x0, protocol=1
        src ip/id=0.0.0.0, mask=0.0.0.0, icmp-type=0, tag=any
        dst ip/id=0.0.0.0, mask=0.0.0.0, icmp-code=0, tag=any, dscp=0x0
        input_ifc=OUTSIDE, output_ifc=any

Phase: 10
Type: VPN
Subtype: ipsec-tunnel-flow
Result: ALLOW
Config:
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x7fee680e05c0, priority=13, domain=ipsec-tunnel-flow, deny=true
        hits=41849, user_data=0x0, cs_id=0x0, flags=0x0, protocol=0
        src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
        dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
        input_ifc=OUTSIDE, output_ifc=any

Phase: 11     
Type: NAT
Subtype: rpf-check
Result: ALLOW
Config:
object network DMZ
 nat (DMZ,OUTSIDE) static X.X.X.70
Additional Information:
 Forward Flow based lookup yields rule:
 out id=0x7fee669815e0, priority=6, domain=nat-reverse, deny=false
        hits=206, user_data=0x7fee66980420, cs_id=0x0, use_real_addr, flags=0x0, protocol=0
        src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
        dst ip/id=192.168.244.2, mask=255.255.255.255, port=0, tag=any, dscp=0x0
        input_ifc=OUTSIDE, output_ifc=DMZ

Phase: 12
Type: FLOW-CREATION
Subtype: 
Result: ALLOW
Config:
Additional Information:
New flow created with id 631055, packet dispatched to next module
Module information for forward flow ...
snp_fp_tracer_drop
snp_fp_inspect_ip_options
snp_sfr
snp_fp_inspect_icmp
snp_fp_translate
snp_fp_adjacency
snp_fp_fragment
snp_ifc_stat

Module information for reverse flow ...

Result:
input-interface: OUTSIDE
input-status: up
input-line-status: up
output-interface: DMZ
output-status: up
output-line-status: up
Action: allow

Config and ping from the NAT'd box @192.168.244.2

#show ip int bri
Interface              IP-Address      OK? Method Status                Protocol
GigabitEthernet0/0/0   192.168.244.2   YES NVRAM  up                    up      

#ping 192.168.244.1 source gi0/0/0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.244.1, timeout is 2 seconds:
Packet sent with a source address of 192.168.244.2 
!!!!!

#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       a - application route
       + - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is 192.168.244.1 to network 0.0.0.0

S*    0.0.0.0/0 [1/0] via 192.168.244.1
      192.168.244.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.244.0/30 is directly connected, GigabitEthernet0/0/0
L        192.168.244.2/32 is directly connected, GigabitEthernet0/0/0

#show run

interface GigabitEthernet0/0/0
 ip address 192.168.244.2 255.255.255.252
 ip access-group XX in

# show access-lists

Extended IP access list XX
    10 permit ip host X.X.X.X any (74 matches)
    20 permit ip host X.X.X.X any
    30 permit ip host X.X.X.X any (110407 matches)
    40 permit ip host X.X.X.X any
    50 permit icmp any any echo-reply (20 matches)
    60 deny ip any any log (18468 matches)

AFlack20 · ‎04-07-2021

I just saw it, it was the ACL on the NAT'd box. I needed to permit echo, as soon as I did that, the ping started to work. My apologies for the wasted time @Sheraz.Salim and thank you for the help.

Extended IP access list XX
    10 permit ip host X.X.X.X any (74 matches)
    20 permit ip host X.X.X.X any
    30 permit ip host X.X.X.X any (110407 matches)
    40 permit ip host X.X.X.X any
    50 permit icmp any any echo-reply (20 matches)
    55 permit icmp any any echo
    60 deny ip any any log (18520 matches)