Another "(acl-drop) Flow is denied by configured rule" Issue

[Elijah Conn]

I have a Nexus 5k with a 10k sfp configured for vlan 800 along with another port also configured for 800.  This goes into an edge router which then goes into the outside interface of an  asa 5545 (had to do it this way temporarily because I did not have a transceiver and ISP has fiber).  I can ping outside and inside from the edge router but can't even ping the ISP interface on the router from the FW.  I think its a nat problem, but I can't figure it out.

packet-tracer in outside icmp 10.200.1.2 8 0 8.8.8.8 de

Phase: 1
Type: ROUTE-LOOKUP
Subtype: input
Result: ALLOW
Config:
Additional Information:
in   0.0.0.0         0.0.0.0         Outside

Phase: 2
Type: ACCESS-LIST
Subtype:
Result: DROP
Config:
Implicit Rule
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x7ffec8c31690, priority=500, domain=permit, deny=true
        hits=9617, user_data=0x6, cs_id=0x0, reverse, flags=0x0, protocol=0
        src ip/id=10.200.1.2, mask=255.255.255.255, port=0, tag=0
        dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0, dscp=0x0
        input_ifc=Outside, output_ifc=any

Result:
input-interface: Outside
input-status: up
input-line-status: up
output-interface: Outside
output-status: up
output-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule

access-list Outside_access_in extended permit ip any any

nat (inside,Outside) source static obj_inside obj_inside destination static obj-ANYCONNECT obj-ANYCONNECT

!
object network obj_inside
 nat (inside,Outside) dynamic interface

object network obj_outside
 subnet 10.200.1.0 255.255.255.0

object network obj_inside
 subnet 10.200.0.0 255.255.0.0

object network obj-ANYCONNECT
 subnet 10.200.0.0 255.255.255.0

Any help would be appreciated, thanks

[Marvin Rhoads]

Your packet-tracer says you are initiating from an outside host 10.200.1.2 and expect to reach 8.8.8.8. Would you expect that to be out some other interface? The logic seems opposite what we would normally see.

Your NAT is setup consistent with more standard logical configuration - trusted hosts on inside being NATted to the outside interface address unless they are going to VPN pool addresses.

A diagram would help here.

[Elijah Conn]

Thank you so much for your response.  Yes this is an unorthodox setup.  The only sfp port available to us was on the Nexus 5k so we had to use that as a L2 edge switch.  I can ping the outside from the router, but cannot from the ASA.  I am also able to ping the inside from the router as well

[Marvin Rhoads · Accepted solution]

The packet-tracer you started the thread with should be simulating a flow THROUGH the ASA - not from it as your example shows. So try something like:

packet-tracer in inside icmp 10.200.0.2 8 0 8.8.8.8 de

That should be allowed assuming your default (or learned) route from the ASA to all things Internet-based is on the outside interface.

When a packet leaves the ASA outside interface you diagram shows it hitting your router. Since your ASA outside interface has a private IP address, your NAT must be taking place on the router - correct?

The router's far side interface is connected to your ISP via a switched layer 2 interface on the Nexus 5k. The VLAN for that interface should be unique to the router and ISP connection.

[Elijah Conn]

That is correct  (see router below).  I have been scratching my head on this for two days now and cant figure it out.  But I think it has something to do with the NAT statements on the FW or Router.

cLAB-EF1# packet-tracer in inside icmp 10.200.0.2 8 0 8.8.8.8 de

Phase: 1
Type: ROUTE-LOOKUP
Subtype: input
Result: ALLOW
Config:
Additional Information:
in   0.0.0.0         0.0.0.0         Outside

Phase: 2
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group acl-inside in interface inside
access-list acl-inside extended permit ip any4 any4
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x7ffec8c561e0, priority=13, domain=permit, deny=false
        hits=799, user_data=0x7ffec0997b00, cs_id=0x0, use_real_addr, flags=0x0, protocol=0
        src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0
        dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0, dscp=0x0
        input_ifc=inside, output_ifc=any

Phase: 3
Type: NAT     
Subtype:
Result: ALLOW
Config:
object network obj_inside
 nat (inside,Outside) dynamic interface
Additional Information:
Dynamic translate 10.200.0.2/0 to 10.200.1.2/53778
 Forward Flow based lookup yields rule:
 in  id=0x7ffebcff3240, priority=6, domain=nat, deny=false
        hits=136, user_data=0x7ffebcfe9260, cs_id=0x0, flags=0x0, protocol=0
        src ip/id=10.200.0.0, mask=255.255.0.0, port=0, tag=0
        dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0, dscp=0x0
        input_ifc=inside, output_ifc=Outside

Phase: 4
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x7ffec7da5b80, priority=0, domain=nat-per-session, deny=true
        hits=14031, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=0
        src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0
        dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0, dscp=0x0
        input_ifc=any, output_ifc=any

Phase: 5
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x7ffec8a4d790, priority=0, domain=inspect-ip-options, deny=true
        hits=2161, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
        src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0
        dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0, dscp=0x0
        input_ifc=inside, output_ifc=any

Phase: 6
Type: INSPECT
Subtype: np-inspect
Result: ALLOW
Config:
class-map inspection_default
 match default-inspection-traffic
policy-map global_policy
 class inspection_default
  inspect icmp
service-policy global_policy global
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x7ffec9982c40, priority=70, domain=inspect-icmp, deny=false
        hits=56, user_data=0x7ffec997ef20, cs_id=0x0, use_real_addr, flags=0x0, protocol=1
        src ip/id=0.0.0.0, mask=0.0.0.0, icmp-type=0, tag=0
        dst ip/id=0.0.0.0, mask=0.0.0.0, icmp-code=0, tag=0, dscp=0x0
        input_ifc=inside, output_ifc=any

Phase: 7
Type: INSPECT
Subtype: np-inspect
Result: ALLOW
Config:
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x7ffec8a4d0c0, priority=66, domain=inspect-icmp-error, deny=false
        hits=80, user_data=0x7ffec8a4c630, cs_id=0x0, use_real_addr, flags=0x0, protocol=1
        src ip/id=0.0.0.0, mask=0.0.0.0, icmp-type=0, tag=0
        dst ip/id=0.0.0.0, mask=0.0.0.0, icmp-code=0, tag=0, dscp=0x0
        input_ifc=inside, output_ifc=any
              
Phase: 8
Type: DEBUG-ICMP
Subtype:
Result: ALLOW
Config:
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x7ffeca439af0, priority=13, domain=debug-icmp-trace, deny=false
        hits=15, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=1
        src ip/id=0.0.0.0, mask=0.0.0.0, icmp-type=0, tag=0
        dst ip/id=0.0.0.0, mask=0.0.0.0, icmp-code=0, tag=0, dscp=0x0
        input_ifc=inside, output_ifc=any

Phase: 9
Type: DEBUG-ICMP
Subtype:
Result: ALLOW
Config:
Additional Information:
 Reverse Flow based lookup yields rule:
 in  id=0x7ffeca439550, priority=13, domain=debug-icmp-trace, deny=false
        hits=71, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=1
        src ip/id=0.0.0.0, mask=0.0.0.0, icmp-type=0, tag=0
        dst ip/id=0.0.0.0, mask=0.0.0.0, icmp-code=0, tag=0, dscp=0x0
        input_ifc=Outside, output_ifc=any

Phase: 10
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:
 Reverse Flow based lookup yields rule:
 in  id=0x7ffec7da5b80, priority=0, domain=nat-per-session, deny=true
        hits=14033, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=0
        src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0
        dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0, dscp=0x0
        input_ifc=any, output_ifc=any

Phase: 11
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
 Reverse Flow based lookup yields rule:
 in  id=0x7ffec8912730, priority=0, domain=inspect-ip-options, deny=true
        hits=12013, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
        src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0
        dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=0, dscp=0x0
        input_ifc=Outside, output_ifc=any

Phase: 12
Type: FLOW-CREATION
Subtype:
Result: ALLOW
Config:
Additional Information:
New flow created with id 13217, packet dispatched to next module
Module information for forward flow ...
snp_fp_tracer_drop
snp_fp_inspect_ip_options
snp_fp_inspect_icmp
snp_fp_translate
snp_fp_dbg_icmp
snp_fp_adjacency
snp_fp_fragment
snp_ifc_stat

Module information for reverse flow ...
snp_fp_tracer_drop
snp_fp_inspect_ip_options
snp_fp_translate
snp_fp_inspect_icmp
snp_fp_dbg_icmp
snp_fp_adjacency
snp_fp_fragment
snp_ifc_stat

Result:
input-interface: inside
input-status: up
input-line-status: up
output-interface: Outside
output-status: up
output-line-status: up
Action: allow

cLAB-EF1(config)# ping 8.8.8.8
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 8.8.8.8, timeout is 2 seconds:
?????
Success rate is 0 percent (0/5)

cLAB-EF1(config)# ping 10.200.1.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.200.1.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms

cLAB-EF1# ping 10.200.0.4
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.200.0.4, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/10 ms
cLAB-EF1#

cLAB-EF1(config)# ping 63.128.68.13
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 63.128.68.13, timeout is 2 seconds:
?????
Success rate is 0 percent (0/5)

cLAB-EF1(config)# sh run route
route Outside 0.0.0.0 0.0.0.0 10.200.1.1 1
route inside 10.200.0.0 255.255.255.0 10.200.0.4 1

Gateway of last resort is 10.200.1.1 to network 0.0.0.0

S    10.200.0.0 255.255.255.0 [1/0] via 10.200.0.4, inside
C    10.200.0.0 255.255.255.248 is directly connected, inside
C    10.200.1.0 255.255.255.0 is directly connected, Outside
O    63.128.68.12 255.255.255.252 [110/11] via 10.200.1.1, 24:01:54, Outside
S*   0.0.0.0 0.0.0.0 [1/0] via 10.200.1.1, Outside
cLAB-EF1#

cLAB-EF1(config)# sh run nat
nat (inside,Outside) source static cLAB_NETWORK cLAB_NETWORK destination static
nat (inside,Outside) source static obj_inside obj_inside destination static obj-ANYCONNECT obj-ANYCONNECT
!
object network obj_inside
 nat (inside,Outside) dynamic interface
object network obj_management
 nat (Management,Outside) dynamic interface

Gateway of last resort is 63.128.68.13 to network 0.0.0.0

S*    0.0.0.0/0 [1/0] via 63.128.68.13
      10.0.0.0/8 is variably subnetted, 6 subnets, 4 masks
S        10.0.0.0/8 is directly connected, GigabitEthernet0/1
C        10.1.200.254/32 is directly connected, Loopback0
O E1     10.200.0.0/24 [110/31] via 10.200.1.2, 23:01:57, GigabitEthernet0/1
O        10.200.0.0/29 [110/11] via 10.200.1.2, 23:48:35, GigabitEthernet0/1
C        10.200.1.0/24 is directly connected, GigabitEthernet0/1
L        10.200.1.1/32 is directly connected, GigabitEthernet0/1
      63.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C        63.128.68.12/30 is directly connected, GigabitEthernet0/0
L        63.128.68.14/32 is directly connected, GigabitEthernet0/0
cLAB-ER1#

cLAB-ER1(config)#do sh run | in ip nat
ip nat inside source static udp 10.200.1.2 500 interface GigabitEthernet0/0 500
ip nat inside source static udp 10.200.1.2 4500 interface GigabitEthernet0/0 4500
ip nat inside source static esp 10.200.1.2 interface GigabitEthernet0/0
ip nat inside source static tcp 10.200.1.2 443 interface GigabitEthernet0/0 443
ip nat inside source list 1 interface GigabitEthernet0/0 overload

cLAB-ER1#sh  ip nat translations
Pro Inside global      Inside local       Outside local      Outside global
esp 63.128.68.14:0     10.200.1.2:0       ---                ---
tcp 63.128.68.14:443   10.200.1.2:443     ---                ---
udp 63.128.68.14:500   10.200.1.2:500     ---                ---
udp 63.128.68.14:4500  10.200.1.2:4500    ---                ---

cLAB-ER1(config)#do ping 10.200.0.4                                 
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.200.0.4, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

cLAB-ER1(config)#do ping 8.8.8.8
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 8.8.8.8, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms

cLAB-ER1#ping 63.128.68.13
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 63.128.68.13, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms

cLAB-ER1#sh arp         
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  10.1.255.1            214   0006.f6e6.90b9  ARPA   GigabitEthernet0/1
Internet  10.200.1.1              -   e02f.6ddf.b6e1  ARPA   GigabitEthernet0/1
Internet  10.200.1.2             15   0006.f6e6.90b9  ARPA   GigabitEthernet0/1
Internet  63.128.68.13          231   001c.b147.ec00  ARPA   GigabitEthernet0/0
Internet  63.128.68.14            -   e02f.6ddf.b6e0  ARPA   GigabitEthernet0/0

[Elijah Conn]

Ok, I figured it out.  One word "overload", smh

ip nat inside source list NAT int g 0/0 over was added to the router and presto.

as far as pinging the inside interface from the router, this will not be possible because of the translations on the fw.  This is not really needed since the router acts as a passthrough.

Thanks for your help

[Marvin Rhoads]

Glad it's working for you. Thanks for the rating.