Re: ASA 5585-x - Unreachable gateway between interfaces

geoff · ‎04-12-2019

Hi All,

All devices within both subnets are reachable across both subnets but each subnet can only reach it's own interface gateway, we require to have reachable gateways across subnets.

For example;

1) subnet-1 can reach subnet-1 gateway but is unable to reach subnet-2 gateway although can reach all devices on subnet-1

2) subnet-2 can reach subnet-2 gateway but is unable to reach subnet-1 gateway although can reach all devices on subnet-2

We have configured the following;

same-security-traffic permit inter-interface

same-security-traffic permit intra-interface

Many thanks in advance for your kind assistance.

Cheers

Geoff

joseph.h.nguyen · ‎04-12-2019

It could be a routing issue or it could be a firewall policy issue. Have you already apply your firewall rules?

Try to use packet-tracer to help diagnose your traffic flow. To learn how, see https://www.cisco.com/c/en/us/td/docs/security/asa/asa-command-reference/I-R/cmdref2/p1.html. You can use this technique either on the CLI or ASDM GUI.

geoff · ‎04-12-2019

Hi Joseph,

Many thanks for your response.

Below 2 x packet trace results, 1 to gateway (unsuccessful) and the other to another device on same subnet as failed first packet trace;

Cheers

Geoff

packet-tracer input vmtg icmp 172.16.40.203 8 0 172.16.254.1 detailed

Phase: 1

Type: ACCESS-LIST

Subtype:

Result: ALLOW

Config:

Implicit Rule

Additional Information:

Forward Flow based lookup yields rule:

in id=0x7f632648b700, priority=1, domain=permit, deny=false

hits=501962414, user_data=0x0, cs_id=0x0, l3_type=0x8

src mac=0000.0000.0000, mask=0000.0000.0000

dst mac=0000.0000.0000, mask=0100.0000.0000

input_ifc=vmtg, output_ifc=any

Result:

input-interface: vmtg

input-status: up

input-line-status: up

Action: drop

Drop-reason: (no-route) No route to host

packet-tracer input vmtg icmp 172.16.40.203 8 0 172.16.254.4 detailed

Phase: 1

Type: ROUTE-LOOKUP

Subtype: Resolve Egress Interface

Result: ALLOW

Config:

Additional Information:

found next-hop 172.16.254.4 using egress ifc imm-s1

Phase: 2

Type: ACCESS-LIST

Subtype:

Result: ALLOW

Config:

Implicit Rule

Additional Information:

Forward Flow based lookup yields rule:

in id=0x7f633b555ff0, priority=2, domain=permit, deny=false

hits=163498, user_data=0x0, cs_id=0x0, flags=0x3000, protocol=0

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0

input_ifc=vmtg, output_ifc=any

Phase: 3

Type: NAT

Subtype: per-session

Result: ALLOW

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0x7f63258cf490, priority=0, domain=nat-per-session, deny=true

hits=8786357, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=0

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0

input_ifc=any, output_ifc=any

Phase: 4

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0x7f6326493b50, priority=0, domain=inspect-ip-options, deny=true

hits=15473298, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0

input_ifc=vmtg, output_ifc=any

Phase: 5

Type: INSPECT

Subtype: np-inspect

Result: ALLOW

Config:

class-map inspection_default

match default-inspection-traffic

policy-map global_policy

class inspection_default

inspect icmp

service-policy global_policy global

Additional Information:

Forward Flow based lookup yields rule:

in id=0x7f6338ddcda0, priority=70, domain=inspect-icmp, deny=false

hits=188094, user_data=0x7f6338e5be90, cs_id=0x0, use_real_addr, flags=0x0, protocol=1

src ip/id=0.0.0.0, mask=0.0.0.0, icmp-type=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, icmp-code=0, tag=any, dscp=0x0

input_ifc=vmtg, output_ifc=any

Phase: 6

Type: INSPECT

Subtype: np-inspect

Result: ALLOW

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0x7f6326493360, priority=66, domain=inspect-icmp-error, deny=false

hits=359650, user_data=0x7f63264928e0, cs_id=0x0, use_real_addr, flags=0x0, protocol=1

src ip/id=0.0.0.0, mask=0.0.0.0, icmp-type=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, icmp-code=0, tag=any, dscp=0x0

input_ifc=vmtg, output_ifc=any

Phase: 7

Type: NAT

Subtype: per-session

Result: ALLOW

Config:

Additional Information:

Reverse Flow based lookup yields rule:

in id=0x7f63258cf490, priority=0, domain=nat-per-session, deny=true

hits=8786359, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=0

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0

input_ifc=any, output_ifc=any

Phase: 8

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Config:

Additional Information:

Reverse Flow based lookup yields rule:

in id=0x7f633a12d380, priority=0, domain=inspect-ip-options, deny=true

hits=2227479, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0

input_ifc=imm-s1, output_ifc=any

Phase: 9

Type: FLOW-CREATION

Subtype:

Result: ALLOW

Config:

Additional Information:

New flow created with id 18485794, packet dispatched to next module

Module information for forward flow ...

snp_fp_tracer_drop

snp_fp_inspect_ip_options

snp_fp_inspect_icmp

snp_fp_adjacency

snp_fp_fragment

snp_ifc_stat

Module information for reverse flow ...

snp_fp_tracer_drop

snp_fp_inspect_ip_options

snp_fp_inspect_icmp

snp_fp_adjacency

snp_fp_fragment

snp_ifc_stat

Result:

input-interface: vmtg

input-status: up

input-line-status: up

output-interface: imm-s1

output-status: up

output-line-status: up

Action: allow

joseph.h.nguyen · ‎04-12-2019

Your first packet-tracer output shows "no route" so I would double check your route table. Second packet-tracer output shows it has passed which conflicts the first output. So I don't know you are using NAT to manipulate the packet switching process. Can you post your configuration and output of "show route"?

geoff · ‎04-12-2019

Hi Joseph,

We have 2 ASA's at 2 different DC's, one works perfectly correctly and the does not.

We have compared configs on both ASA's and they're exactly the same.

We do not use any NAT's and/or Routes between interfaces on same ASA.

Below is show route output;

Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP

D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area

N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2

E1 - OSPF external type 1, E2 - OSPF external type 2, V - VPN

i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2

ia - IS-IS inter area, * - candidate default, U - per-user static route

o - ODR, P - periodic downloaded static route, + - replicated route

Gateway of last resort is n1.n1.n1.161 to network 0.0.0.0

S* 0.0.0.0 0.0.0.0 [1/0] via n1.n1.n1.161, outside

C 10.0.10.0 255.255.255.0 is directly connected, rcg-vlan

L 10.0.10.1 255.255.255.255 is directly connected, rcg-vlan

C 10.0.11.0 255.255.255.0 is directly connected, rlt-vlan

L 10.0.11.1 255.255.255.255 is directly connected, rlt-vlan

C 10.0.12.0 255.255.255.0 is directly connected, tif-vlan

L 10.0.12.1 255.255.255.255 is directly connected, tif-vlan

C 10.0.100.0 255.255.255.0 is directly connected, pdc-vlan

L 10.0.100.1 255.255.255.255 is directly connected, pdc-vlan

C n1.n1.n1.160 255.255.255.240 is directly connected, outside

L n1.n1.n1.166 255.255.255.255 is directly connected, outside

C n2.n2.n2.0 255.255.255.128 is directly connected, DMZ

L n2.n2.n2.2 255.255.255.255 is directly connected, DMZ

C 172.16.40.0 255.255.255.0 is directly connected, vmtg

L 172.16.40.1 255.255.255.255 is directly connected, vmtg

C 172.16.254.0 255.255.255.0 is directly connected, imm-s1

L 172.16.254.1 255.255.255.255 is directly connected, imm-s1

Cheers

Geoff

joseph.h.nguyen · ‎04-12-2019

I think I understand your problem set. Earlier I was visualizing an external gateway but based on your recent routing table, you are trying to ping a different interface IP from the source subnet. ASA has a built-in security mechanism by default to drop ICMP request to ASA interface IP address from different subnet. You can override the default configuration. Follow this link: https://community.cisco.com/t5/firewalls/ping-through-asa/td-p/1324977.

Try executing "icmp permit 172.16.40.0 255.255.255.0 echo vmtg" or try the policy configuration from the link above.

geoff · ‎04-12-2019

Hi Joseph,

Sorry unfortunately still not working, config below;

Pinging between /24 subnet's is working, just unable to interface ip address across subnet.

Cheers

Geoff

sh run icmp

icmp unreachable rate-limit 1 burst-size 1

icmp permit 172.16.254.0 255.255.255.0 echo vmtg

icmp permit 172.16.40.0 255.255.255.0 echo vmtg

icmp permit 172.16.254.0 255.255.255.0 echo imm-s1

icmp permit 172.16.40.0 255.255.255.0 echo imm-s1

joseph.h.nguyen · ‎04-12-2019

Have you already configured the policy?

policy-map global_policy

class inspection_default

inspect icmp

geoff · ‎04-12-2019

Hi Joseph,

Yes, config below;

sh run policy-map

!

policy-map type inspect dns preset_dns_map

parameters

message-length maximum client auto

message-length maximum 512

no tcp-inspection

policy-map global_policy

class inspection_default

inspect dns preset_dns_map

inspect ftp

inspect h323 h225

inspect h323 ras

inspect rsh

inspect rtsp

inspect esmtp

inspect sqlnet

inspect skinny

inspect sunrpc

inspect xdmcp

inspect sip

inspect netbios

inspect tftp

inspect ip-options

inspect icmp

!

We need application to connect to 172.16.254.1 from 172.16.40.205, the application first ping's the ip address then connects via tcp port.

ICMP settings are not blocking us, something else in ASA is though.

We get following ambiguous error message;

Failed to locate egress interface for ICMP from vmtg: 172.16.40.205/27688 to 172.16.254.1/0

Cheers

Geoff

geoff · ‎04-12-2019

Here is a successful log to another device on same subnet;

6|Apr 13 2019|14:46:52|302021|172.16.40.205|28221|172.16.254.5|0|Teardown ICMP connection for faddr 172.16.40.205/28221 gaddr 172.16.254.5/0 laddr 172.16.254.5/0 type 8 code 0
6|Apr 13 2019|14:46:52|302020|172.16.40.205|28221|172.16.254.5|0|Built inbound ICMP connection for faddr 172.16.40.205/28221 gaddr 172.16.254.5/0 laddr 172.16.254.5/0 type 8 code 0

joseph.h.nguyen · ‎04-12-2019

Is the policy-map applied? show service-policy?

geoff · ‎04-12-2019

Hi Joseph,

I believe so, refer output below;

Cheers

Geoff

show service-policy

Global policy:

Service-policy: global_policy

Class-map: inspection_default

Inspect: dns preset_dns_map, packet 7342690, lock fail 0, drop 500, reset-drop 0, 5-min-pkt-rate 0 pkts/sec, v6-fail-close 0 sctp-drop-override 0

Inspect: ftp, packet 536034, lock fail 0, drop 0, reset-drop 0, 5-min-pkt-rate 0 pkts/sec, v6-fail-close 0 sctp-drop-override 0

Inspect: h323 h225 _default_h323_map, packet 0, lock fail 0, drop 0, reset-drop 0, 5-min-pkt-rate 0 pkts/sec, v6-fail-close 0 sctp-drop-override 0

tcp-proxy: bytes in buffer 0, bytes dropped 0

Inspect: h323 ras _default_h323_map, packet 0, lock fail 0, drop 0, reset-drop 0, 5-min-pkt-rate 0 pkts/sec, v6-fail-close 0 sctp-drop-override 0

Inspect: rsh, packet 1, lock fail 0, drop 0, reset-drop 0, 5-min-pkt-rate 0 pkts/sec, v6-fail-close 0 sctp-drop-override 0

Inspect: rtsp, packet 1, lock fail 0, drop 0, reset-drop 0, 5-min-pkt-rate 0 pkts/sec, v6-fail-close 0 sctp-drop-override 0

tcp-proxy: bytes in buffer 0, bytes dropped 0

Inspect: esmtp _default_esmtp_map, packet 2865713, lock fail 2, drop 0, reset-drop 0, 5-min-pkt-rate 0 pkts/sec, v6-fail-close 0 sctp-drop-override 0

Inspect: sqlnet, packet 293, lock fail 0, drop 0, reset-drop 0, 5-min-pkt-rate 0 pkts/sec, v6-fail-close 0 sctp-drop-override 0

Inspect: skinny , packet 1, lock fail 0, drop 0, reset-drop 0, 5-min-pkt-rate 0 pkts/sec, v6-fail-close 0 sctp-drop-override 0

tcp-proxy: bytes in buffer 0, bytes dropped 0

Inspect: sunrpc, packet 611, lock fail 0, drop 0, reset-drop 0, 5-min-pkt-rate 0 pkts/sec, v6-fail-close 0 sctp-drop-override 0

tcp-proxy: bytes in buffer 0, bytes dropped 0

Inspect: xdmcp, packet 0, lock fail 0, drop 0, reset-drop 0, 5-min-pkt-rate 0 pkts/sec, v6-fail-close 0 sctp-drop-override 0

Inspect: sip , packet 4, lock fail 0, drop 0, reset-drop 0, 5-min-pkt-rate 0 pkts/sec, v6-fail-close 0 sctp-drop-override 0

tcp-proxy: bytes in buffer 0, bytes dropped 0

Inspect: netbios, packet 324, lock fail 0, drop 0, reset-drop 0, 5-min-pkt-rate 0 pkts/sec, v6-fail-close 0 sctp-drop-override 0

Inspect: tftp, packet 0, lock fail 0, drop 0, reset-drop 0, 5-min-pkt-rate 0 pkts/sec, v6-fail-close 0 sctp-drop-override 0

Inspect: ip-options _default_ip_options_map, packet 0, lock fail 0, drop 0, reset-drop 0, 5-min-pkt-rate 0 pkts/sec, v6-fail-close 0 sctp-drop-override 0

Inspect: icmp, packet 381825, lock fail 0, drop 503, reset-drop 0, 5-min-pkt-rate 0 pkts/sec, v6-fail-close 0 sctp-drop-override 0

Georg Pauwen · ‎04-13-2019

Hello,

post the full running configuration of your ASA, there might be something else we are overlooking...

geoff · ‎04-13-2019

Hi George,

Please kindly find attached a running config, obviously I have changed public ip's and other sensitivity information.

Cheers

Geoff

GRANT3779 · ‎04-13-2019

I don't think this is possibe and by design. See the following lines from the link provided.

The ASA only responds to ICMP traffic sent to the interface that traffic comes in on; you cannot send ICMP traffic through an interface to a far interface.

https://www.cisco.com/c/en/us/td/docs/security/asa/asa98/configuration/firewall/asa-98-firewall-config/access-rules.html