Azure Site-to-Site VPN: Can ping from on-prem to Azure VM but not visa versa; all other traffic flows

[mark.bell]

Can someone please help me figure out why I can't ping from my Azure VM to any on-premises devices? I've added exclusions for ICMP in Windows Firewalls on both sides and can successfully ping from on-prem (i.e. 192.168.1.90) to an Azure VM (i.e. 10.0.0.1), but not the other way around. I've also added exclusions in Azure NSGs. All other traffic flows successfully. We are currently utilizing a Cisco ASA 5506-X with Firepower. We could ping in both directions when we were using the older ASA 5505, however the config for the two are a bit different (no VLANs on the 5506). 

Here is my config (anything marked [removed] I deemed sensitive):

Result of the command: "show run"

: Saved

:
: Serial Number: [removed]
: Hardware:   ASA5506, 4096 MB RAM, CPU Atom C2000 series 1250 MHz, 1 CPU (4 cores)
:
ASA Version 9.8(2)
!
hostname ciscoasa
domain-name [removed]
enable password [removed]
xlate per-session deny tcp any4 any4
names

!
interface GigabitEthernet1/1
 description Comcast
 nameif outside
 security-level 0
 ip address [removed] 255.255.255.248
!
interface GigabitEthernet1/2
 bridge-group 1
 nameif inside_1
 security-level 100
!
interface GigabitEthernet1/3
 bridge-group 1
 nameif inside_2
 security-level 100
!
interface GigabitEthernet1/4
 bridge-group 1
 nameif inside_3
 security-level 100
!
interface GigabitEthernet1/5
 bridge-group 1
 nameif inside_4
 security-level 100
!
interface GigabitEthernet1/6
 bridge-group 1
 nameif inside_5
 security-level 100
!
interface GigabitEthernet1/7
 bridge-group 1
 nameif inside_6
 security-level 100
!
interface GigabitEthernet1/8
 bridge-group 1
 nameif inside_7
 security-level 100
!
interface Management1/1
 management-only
 no nameif
 no security-level
 no ip address
!
interface BVI1
 description [Removed] Network
 nameif inside
 security-level 100
 ip address 192.168.1.254 255.255.255.0
!
boot system disk0:/asa982-lfbff-k8.SPA
ftp mode passive
clock timezone CST -6
clock summer-time CDT recurring
dns domain-lookup outside
dns server-group DefaultDNS
 name-server 75.75.75.75
 name-server 8.8.8.8
 name-server 8.8.4.4
 domain-name [removed]
same-security-traffic permit inter-interface
same-security-traffic permit intra-interface
object network obj_any
 subnet 0.0.0.0 0.0.0.0
object network inside-network
 subnet 192.168.1.0 255.255.255.0
object network azure
 subnet 10.0.0.0 255.255.0.0
object network DHCP_server
 host 192.168.1.10
object service tcp44434
 service tcp destination eq 44434
 description RDP
object network OutsideInterface
 host [removed]
object service RDP-Service
 service tcp source eq [removed]
object network AzureDC1
 host 10.0.0.4
object service ICMPv4
 service icmp echo 0
object-group network azure-networks
 network-object object azure
object-group network onprem-networks
 network-object 192.168.1.0 255.255.255.0
object-group service rdp tcp
 port-object eq [removed]
access-list outside_access_in extended permit icmp any any
access-list outside_access_in extended permit ip host [removed] host [removed]
access-list outside_access_in remark Ping
access-list outside_cryptomap extended permit ip object-group onprem-networks object-group azure-networks
access-list outside_nat extended permit tcp any host [removed] eq [removed]
access-list inside_nat0_outbound extended permit ip 192.168.1.0 255.255.255.0 object azure
access-list OUTSIDE extended permit icmp any any
access-list inside_6_access_in extended permit ip any any
access-list inside_6_access_in extended permit icmp any any
pager lines 24
logging enable
logging buffer-size 1000000
logging buffered informational
logging asdm informational
mtu outside 1500
mtu inside_1 1500
mtu inside_2 1500
mtu inside_3 1500
mtu inside_4 1500
mtu inside_5 1500
mtu inside_6 1500
mtu inside_7 1500
icmp unreachable rate-limit 1 burst-size 1
icmp permit any outside
icmp permit any inside_2
icmp permit any inside_5
icmp permit any inside_6
icmp permit any inside
asdm image disk0:/asdm-782-151.bin
no asdm history enable
arp timeout 14400
no arp permit-nonconnected
arp rate-limit 16384
nat (inside_1,outside) source static onprem-networks onprem-networks destination static azure-networks azure-networks
nat (inside_2,outside) source static onprem-networks onprem-networks destination static azure-networks azure-networks
nat (inside_3,outside) source static onprem-networks onprem-networks destination static azure-networks azure-networks
nat (inside_4,outside) source static onprem-networks onprem-networks destination static azure-networks azure-networks
nat (inside_5,outside) source static onprem-networks onprem-networks destination static azure-networks azure-networks
nat (inside_6,outside) source static onprem-networks onprem-networks destination static azure-networks azure-networks
nat (inside_7,outside) source static onprem-networks onprem-networks destination static azure-networks azure-networks
!
object network obj_any
 nat (any,outside) dynamic interface
access-group outside_access_in in interface outside
access-group inside_6_access_in in interface inside_6
route outside 0.0.0.0 0.0.0.0 [removed] 1
timeout xlate 3:00:00
timeout pat-xlate 0:00:30
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 sctp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute
timeout tcp-proxy-reassembly 0:01:00
timeout floating-conn 0:00:00
timeout conn-holddown 0:00:15
timeout igp stale-route 0:01:10
user-identity default-domain LOCAL
user-identity ad-agent active-user-database on-demand
aaa authentication http console LOCAL
aaa authentication ssh console LOCAL
aaa authentication telnet console LOCAL
aaa authentication login-history
http server enable
http 192.168.1.0 255.255.255.0 inside_1
http 192.168.1.0 255.255.255.0 inside_2
http 192.168.1.0 255.255.255.0 inside_3
http 192.168.1.0 255.255.255.0 inside_4
http 192.168.1.0 255.255.255.0 inside_5
http 192.168.1.0 255.255.255.0 inside_6
http 192.168.1.0 255.255.255.0 inside_7
no snmp-server location
no snmp-server contact
sysopt connection tcpmss 1318
sysopt connection preserve-vpn-flows
service sw-reset-button
crypto ipsec ikev2 ipsec-proposal AES256Azure
 protocol esp encryption aes-256
 protocol esp integrity sha-1
crypto ipsec ikev2 ipsec-proposal DES
 protocol esp encryption des
 protocol esp integrity sha-1 md5
crypto ipsec ikev2 ipsec-proposal 3DES
 protocol esp encryption 3des
 protocol esp integrity sha-1 md5
crypto ipsec ikev2 ipsec-proposal AES
 protocol esp encryption aes
 protocol esp integrity sha-1 md5
crypto ipsec ikev2 ipsec-proposal AES192
 protocol esp encryption aes-192
 protocol esp integrity sha-1 md5
crypto ipsec ikev2 ipsec-proposal AES256
 protocol esp encryption aes-256
 protocol esp integrity sha-1 md5
crypto ipsec security-association lifetime seconds 3600
crypto ipsec security-association lifetime kilobytes 102400000
crypto ipsec security-association pmtu-aging infinite
crypto map outside_map1 1 match address outside_cryptomap
crypto map outside_map1 1 set pfs group24
crypto map outside_map1 1 set peer [removed]
crypto map outside_map1 1 set ikev2 ipsec-proposal AES256Azure
crypto map outside_map1 1 set security-association lifetime seconds 3600
crypto map outside_map1 1 set security-association lifetime kilobytes 102400000
crypto map outside_map1 interface outside
crypto ca trustpoint _SmartCallHome_ServerCA
 no validation-usage
 crl configure
crypto ca trustpool policy
crypto ca certificate chain _SmartCallHome_ServerCA
 certificate ca [removed]
  quit
crypto ikev2 policy 1
 encryption aes-256
 integrity sha
 group 24
 prf sha
 lifetime seconds 28800
crypto ikev2 enable outside
crypto ikev2 enable inside_1
crypto ikev2 enable inside_2
crypto ikev2 enable inside_3
crypto ikev2 enable inside_4
crypto ikev2 enable inside_5
crypto ikev2 enable inside_6
crypto ikev2 enable inside_7
crypto ikev2 enable inside
crypto ikev1 enable outside
crypto ikev1 enable inside_1
crypto ikev1 enable inside_2
crypto ikev1 enable inside_3
crypto ikev1 enable inside_4
crypto ikev1 enable inside_5
crypto ikev1 enable inside_6
crypto ikev1 enable inside_7
crypto ikev1 enable inside
crypto ikev1 policy 20
 authentication rsa-sig
 encryption aes-256
 hash sha
 group 2
 lifetime 86400
crypto ikev1 policy 30
 authentication pre-share
 encryption aes-256
 hash sha
 group 2
 lifetime 86400
crypto ikev1 policy 50
 authentication rsa-sig
 encryption aes-192
 hash sha
 group 2
 lifetime 86400
crypto ikev1 policy 60
 authentication pre-share
 encryption aes-192
 hash sha
 group 2
 lifetime 86400
crypto ikev1 policy 80
 authentication rsa-sig
 encryption aes
 hash sha
 group 2
 lifetime 86400
crypto ikev1 policy 90
 authentication pre-share
 encryption aes
 hash sha
 group 2
 lifetime 86400
crypto ikev1 policy 110
 authentication rsa-sig
 encryption 3des
 hash sha
 group 2
 lifetime 86400
crypto ikev1 policy 120
 authentication pre-share
 encryption 3des
 hash sha
 group 2
 lifetime 86400
crypto ikev1 policy 140
 authentication rsa-sig
 encryption des
 hash sha
 group 2
 lifetime 86400
crypto ikev1 policy 150
 authentication pre-share
 encryption des
 hash sha
 group 2
 lifetime 86400
telnet 192.168.1.0 255.255.255.0 inside_1
telnet 192.168.1.0 255.255.255.0 inside_2
telnet 192.168.1.0 255.255.255.0 inside_3
telnet 192.168.1.0 255.255.255.0 inside_4
telnet 192.168.1.0 255.255.255.0 inside_5
telnet 192.168.1.0 255.255.255.0 inside_6
telnet 192.168.1.0 255.255.255.0 inside_7
telnet timeout 5
no ssh stricthostkeycheck
ssh 192.168.1.0 255.255.255.0 inside_1
ssh 192.168.1.0 255.255.255.255 inside_2
ssh 192.168.1.0 255.255.255.255 inside_3
ssh 192.168.1.0 255.255.255.255 inside_4
ssh 192.168.1.0 255.255.255.255 inside_5
ssh 192.168.1.0 255.255.255.255 inside_6
ssh 192.168.1.0 255.255.255.255 inside_7
ssh timeout 5
ssh key-exchange group dh-group1-sha1
console timeout 0

dhcpd dns 75.75.75.75 75.75.76.76
dhcpd domain [removed]
dhcpd auto_config outside
!
threat-detection basic-threat
threat-detection statistics
threat-detection statistics tcp-intercept rate-interval 30 burst-rate 400 average-rate 200
group-policy GroupPolicy1 internal
group-policy GroupPolicy1 attributes
 vpn-tunnel-protocol ikev2
dynamic-access-policy-record DfltAccessPolicy
username [removed]
username [removed]
tunnel-group [removed] type ipsec-l2l
tunnel-group [removed] general-attributes
 default-group-policy GroupPolicy1
tunnel-group [removed] ipsec-attributes
 ikev2 remote-authentication pre-shared-key [removed]
 ikev2 local-authentication pre-shared-key [removed]
no tunnel-group-map enable ou
!
class-map class-default-settings
 match default-inspection-traffic
class-map global-class
 match any
class-map inspection_default
 match default-inspection-traffic
!
!
policy-map type inspect dns preset_dns_map
 parameters
  message-length maximum client auto
  message-length maximum 512
  no tcp-inspection
policy-map global_policy
 class inspection_default
  inspect icmp
policy-map global-policy
 class class-default-settings
  inspect icmp
policy-map class-default
 class global-class
  inspect icmp
!
service-policy global_policy global
prompt hostname context
service call-home
call-home reporting anonymous
call-home
 contact-email-addr [removed]
 profile CiscoTAC-1
  destination address http https://tools.cisco.com/its/service/oddce/services/DDCEService
  destination address email callhome@cisco.com
  destination transport-method http
  subscribe-to-alert-group diagnostic
  subscribe-to-alert-group environment
  subscribe-to-alert-group inventory periodic monthly
  subscribe-to-alert-group configuration periodic monthly
  subscribe-to-alert-group telemetry periodic daily
hpm topN enable
Cryptochecksum:426c1042e5da8735c3d46dba6a161aec
: end

[Georg Pauwen]

Hello,

can you ping the internal hosts from the ASA ?

Post the output of:

ciscoasa# packet-tracer input outside icmp 8.8.8.8 8 0 192.168.1.1 --> this needs to be one of your inside host IP addresses

[mark.bell]

Result of the command: "packet-tracer input outside icmp 8.8.8.8 8 0 192.168.1.90"

Phase: 1
Type: ACCESS-LIST
Subtype:
Result: ALLOW
Config:
Implicit Rule
Additional Information:
MAC Access list

Phase: 2
Type: ROUTE-LOOKUP
Subtype: Resolve Egress Interface
Result: ALLOW
Config:
Additional Information:
found next-hop 192.168.1.90 using egress ifc  inside

Phase: 3
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group outside_access_in in interface outside
access-list outside_access_in extended permit icmp any any
Additional Information:

Phase: 4
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:

Phase: 5
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:

Phase: 6
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group outside_access_in in interface outside
access-list outside_access_in extended permit icmp any any
Additional Information:

Phase: 7
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:

Phase: 8
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:

Phase: 9
Type: INSPECT
Subtype: np-inspect
Result: ALLOW
Config:
class-map inspection_default
 match default-inspection-traffic
policy-map global_policy
 class inspection_default
  inspect icmp
service-policy global_policy global
Additional Information:

Phase: 10
Type: INSPECT
Subtype: np-inspect
Result: ALLOW
Config:
Additional Information:

Phase: 11
Type: VPN
Subtype: ipsec-tunnel-flow
Result: ALLOW
Config:
Additional Information:

Phase: 12
Type: NAT
Subtype: rpf-check
Result: DROP
Config:
object network obj_any
 nat (any,outside) dynamic interface
Additional Information:

Result:
input-interface: outside
input-status: up
input-line-status: up
output-interface: inside
output-status: up
output-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule

[Georg Pauwen]

Hello,

PING is denied for addresses originating from the Internet (8.8.8.8 in this case) and dropped by NAT. 

Can you do the same packet tracer using only internal addresses ?

e.g.:

ciscoasa#packet-tracer input outside icmp 10.0.1.1 8 0 192.168.1.90

[mark.bell]

Result of the command: "packet-tracer input outside icmp 10.0.1.1 8 0 192.168.1.90"

Phase: 1
Type: UN-NAT
Subtype: static
Result: ALLOW
Config:
nat (inside_1,outside) source static onprem-networks onprem-networks destination static azure-networks azure-networks
Additional Information:
NAT divert to egress interface inside_1
Untranslate 192.168.1.90/0 to 192.168.1.90/0

Phase: 2
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group outside_access_in in interface outside
access-list outside_access_in extended permit icmp any any
Additional Information:

Phase: 3
Type: NAT
Subtype:
Result: ALLOW
Config:
nat (inside_1,outside) source static onprem-networks onprem-networks destination static azure-networks azure-networks
Additional Information:
Static translate 10.0.1.1/0 to 10.0.1.1/0

Phase: 4
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:

Phase: 5
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:

Phase: 6
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group outside_access_in in interface outside
access-list outside_access_in extended permit icmp any any
Additional Information:

Phase: 7
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:

Phase: 8
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:

Phase: 9
Type: VPN
Subtype: ipsec-tunnel-flow
Result: DROP
Config:
Additional Information:

Result:
input-interface: outside
input-status: up
input-line-status: up
output-interface: inside_1
output-status: down
output-line-status: down
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule

[Georg Pauwen]

Hello,

somehow the tunnel is dropping the ICMP traffic, I cannot really see anything wrong with your ASA config...

For the sake of testing can you change:

access-list outside_access_in extended permit icmp any any

to

access-list outside_access_in extended permit ip any any

and try the ping again ?

[mark.bell]

Made the change. Still not able to ping. I wonder if this has anything to do with this site-to-site VPN using policy-based traffic selectors?

Result of the command: "packet-tracer input outside icmp 10.0.1.1 8 0 192.168.1.90"

Phase: 1

Type: UN-NAT

Subtype: static

Result: ALLOW

Config:

nat (inside_1,outside) source static onprem-networks onprem-networks destination static azure-networks azure-networks

Additional Information:

NAT divert to egress interface inside_1

Untranslate 192.168.1.90/0 to 192.168.1.90/0

Phase: 2

Type: ACCESS-LIST

Subtype: log

Result: ALLOW

Config:

access-group outside_access_in in interface outside

access-list outside_access_in extended permit ip any any

Additional Information:

Phase: 3

Type: NAT

Subtype:

Result: ALLOW

Config:

nat (inside_1,outside) source static onprem-networks onprem-networks destination static azure-networks azure-networks

Additional Information:

Static translate 10.0.1.1/0 to 10.0.1.1/0

Phase: 4

Type: NAT

Subtype: per-session

Result: ALLOW

Config:

Additional Information:

Phase: 5

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Config:

Additional Information:

Phase: 6

Type: ACCESS-LIST

Subtype: log

Result: ALLOW

Config:

access-group outside_access_in in interface outside

access-list outside_access_in extended permit ip any any

Additional Information:

Phase: 7

Type: NAT

Subtype: per-session

Result: ALLOW

Config:

Additional Information:

Phase: 8

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Config:

Additional Information:

Phase: 9

Type: VPN

Subtype: ipsec-tunnel-flow

Result: DROP

Config:

Additional Information:

Result:

input-interface: outside

input-status: up

input-line-status: up

output-interface: inside_1

output-status: down

output-line-status: down

Action: drop

Drop-reason: (acl-drop) Flow is denied by configured rule

##- Please type your reply above this line -##