ASA AnyConnect client reachability

cheeseburger · ‎05-02-2019

Good Morning,

I'm working on a POC for Cisco Phones to VPN for remote agents. I've never done anything like this before but I think I'm close. I am able to get a laptop to connect to the ASA via AnyConnect and obtain an IP from a local pool. However, I cannot ping the laptop from the ASA. I have posted my config below. I'm wondering if there is some sort of NAT or routing issue but the config looks the same as an AnyConnect VPN we currently have that is working.

B6A-DC-THOMPSON-VOIP-ASA# sh run
: Saved

:
: Serial Number: 9A0MVFHS3WT
: Hardware: ASAv, 1536 MB RAM, CPU Xeon 8100 series 2200 MHz
:
ASA Version 9.10(1)17
!
hostname B6A-DC-THOMPSON-VOIP-ASA
enable password ***** pbkdf2
!
license smart
feature tier standard
throughput level 100M
names
no mac-address auto
ip local pool POOLCRAP 10.69.69.10-10.69.69.254 mask 255.255.255.0

!
interface GigabitEthernet0/0
nameif Outside
security-level 100
ip address 10.176.69.69 255.255.0.0
!
interface GigabitEthernet0/1
nameif Inside
security-level 100
ip address 10.177.69.69 255.255.0.0
!
interface GigabitEthernet0/2
shutdown
no nameif
no security-level
no ip address
!
interface GigabitEthernet0/3
shutdown
no nameif
no security-level
no ip address
!
interface GigabitEthernet0/4
shutdown
no nameif
no security-level
no ip address
!
interface GigabitEthernet0/5
shutdown
no nameif
no security-level
no ip address
!
interface GigabitEthernet0/6
shutdown
no nameif
no security-level
no ip address
!
interface GigabitEthernet0/7
shutdown
no nameif
no security-level
no ip address
!
interface GigabitEthernet0/8
shutdown
no nameif
no security-level
no ip address
!
interface Management0/0
nameif management
security-level 0
ip address 10.175.69.69 255.255.0.0
!
ftp mode passive
dns domain-lookup management
dns server-group DefaultDNS
name-server 10.175.254.10
access-list NO_NAT extended permit ip 10.69.69.0 255.255.255.0 10.0.0.0 255.0.0.0
access-list Inside_access_in extended permit ip any any
access-list Outside_access_in extended permit ip any any
pager lines 23
mtu management 1500
mtu Outside 1500
mtu Inside 1500
no failover
no monitor-interface service-module
icmp unreachable rate-limit 1 burst-size 1
no asdm history enable
arp timeout 14400
no arp permit-nonconnected
arp rate-limit 8192
access-group Outside_access_in in interface Outside
access-group Inside_access_in in interface Inside
route Outside 0.0.0.0 0.0.0.0 10.176.0.1 1
route Inside 10.0.0.0 255.0.0.0 10.177.0.1 1
route management 10.201.0.0 255.255.0.0 10.175.0.1 1
route Outside 10.201.22.0 255.255.255.0 10.176.0.1 1
route management 192.168.0.0 255.255.0.0 10.175.0.1 1
timeout xlate 3:00:00
timeout pat-xlate 0:00:30
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 sctp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute
timeout tcp-proxy-reassembly 0:01:00
timeout floating-conn 0:00:00
timeout conn-holddown 0:00:15
timeout igp stale-route 0:01:10
user-identity default-domain LOCAL
aaa authentication ssh console LOCAL
aaa authentication login-history
http server enable
http 0.0.0.0 0.0.0.0 management
no snmp-server location
no snmp-server contact
crypto ipsec security-association pmtu-aging infinite
crypto ca trustpoint _SmartCallHome_ServerCA
no validation-usage
crl configure
crypto ca trustpoint VOIPCRAP
enrollment self
fqdn voipcrap.mydomain.com
subject-name CN=voipcrap.mydomain.com
keypair PHONEVPNCRAP
crl configure
crypto ca trustpool policy
auto-import

crypto ca certificate chain _SmartCallHome_ServerCA
certificate ca 0509
quit

crypto ca certificate chain VOIPCRAP
certificate 0ecfc95c

quit
telnet timeout 5
ssh stricthostkeycheck
ssh 0.0.0.0 0.0.0.0 management
ssh 0.0.0.0 0.0.0.0 Inside
ssh timeout 60
ssh version 2
ssh key-exchange group dh-group1-sha1
console timeout 0
threat-detection basic-threat
threat-detection statistics access-list
no threat-detection statistics tcp-intercept
ssl trust-point VOIPCRAP Outside
webvpn
enable Outside
anyconnect image disk0:/anyconnect-win-3.1.00495-k9.pkg 1
anyconnect enable
tunnel-group-list enable
cache
disable
error-recovery disable
group-policy POLICYCRAP internal
group-policy POLICYCRAP attributes
dns-server value 10.175.254.10
vpn-tunnel-protocol ssl-client
default-domain value mydomain.com
address-pools value POOLCRAP
webvpn
anyconnect ask enable
dynamic-access-policy-record DfltAccessPolicy
username TEST password ***** pbkdf2
username TEST attributes
service-type remote-access
username admin password ***** pbkdf2 privilege 15
tunnel-group TUNNELGROUPCRAP type remote-access
tunnel-group TUNNELGROUPCRAP general-attributes
default-group-policy POLICYCRAP
tunnel-group TUNNELGROUPCRAP webvpn-attributes
group-alias TUNNELGROUPCRAP$ enable
!
class-map inspection_default
match default-inspection-traffic
!
!
policy-map type inspect dns preset_dns_map
parameters
message-length maximum client auto
message-length maximum 512
no tcp-inspection
policy-map global_policy
class inspection_default
inspect ip-options
inspect netbios
inspect rtsp
inspect sunrpc
inspect tftp
inspect xdmcp
inspect dns preset_dns_map
inspect ftp
inspect h323 h225
inspect h323 ras
inspect rsh
inspect esmtp
inspect sqlnet
inspect sip
inspect skinny
policy-map type inspect dns migrated_dns_map_2
parameters
message-length maximum client auto
message-length maximum 512
no tcp-inspection
policy-map type inspect dns migrated_dns_map_1
parameters
message-length maximum client auto
message-length maximum 512
no tcp-inspection
!
service-policy global_policy global
prompt hostname context
no call-home reporting anonymous
call-home
profile CiscoTAC-1
no active
destination address http https://tools.cisco.com/its/service/oddce/services/DDCEService
destination address email callhome@cisco.com
destination transport-method http
subscribe-to-alert-group diagnostic
subscribe-to-alert-group environment
subscribe-to-alert-group inventory periodic monthly
subscribe-to-alert-group configuration periodic monthly
subscribe-to-alert-group telemetry periodic daily
profile License
destination address http https://tools.cisco.com/its/service/oddce/services/DDCEService
destination transport-method http
Cryptochecksum:38e44706c3d8d2a82
: end

I've looked at some online configuration guides and I believe I have everything correct. Obviously, I'm missing something though.

cheeseburger · ‎05-02-2019

I'm including some troubleshooting steps I've done. debug on ICMP:

B6A-DC-THOMPSON-VOIP-ASA# ping 10.69.69.10
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.69.69.10, timeout is 2 seconds:
ICMP echo request from 10.176.69.69 to 10.69.69.10 ID=4373 seq=51841 len=72
?ICMP echo request from 10.176.69.69 to 10.69.69.10 ID=4374 seq=51841 len=72
?ICMP echo request from 10.176.69.69 to 10.69.69.10 ID=4375 seq=51841 len=72
?ICMP echo request from 10.176.69.69 to 10.69.69.10 ID=4376 seq=51841 len=72
?ICMP echo request from 10.176.69.69 to 10.69.69.10 ID=4377 seq=51841 len=72
?
Success rate is 0 percent (0/5)

Host route in RIB:

V 10.69.69.10 255.255.255.255 connected by VPN (advertised), Outside

cheeseburger · ‎05-06-2019

172 people and not one person has an idea? I make 173 then! :) Working on this more today, will keep an eye on this in case I get lucky with a response.

Rahul Govindan · ‎05-06-2019

Do a "ping inside 10.69.69.10" instead. You would also need the following command to reach a VPN IP from the ASA:

management-access inside

Also, your inside and outside interfaces are on the same security level. By default, traffic between same security levels is not allowed. You would need to add:

same-security permit inter-interface

cheeseburger · ‎05-06-2019

Thanks for the reply Rahul. I have applied those commands and attempted ping inside as you suggested. A few things to add:
The route to 10.69.69.10 is a host route and shows Outside interface. I assume you meant ping Outside but I tried both and neither worked.
I applied the same-security command and it did not help.
I found this Cisco ASDM Packet Tracer on accident. 😃 I thought I’d try it out and I find that the route lookup is successful, the access list lookup is successful but the Action is DROP. However, I have an allow any any for a FW rule. Hopefully the screen shots work with email replies. I guess I’m about to find out.

Okay, pictures uploaded. We see hits on the Inside interface but I'm not seeing hits on the implicit deny all rule. Confused.

Rahul Govindan · ‎05-06-2019

Outside is where the VPN client exists with respect to the ASA. When you initiate a ping from the outside, it will take the outside ip address and send the echo packet outside of the VPN tunnel. When you initiate it from the inside, the echo request will match against the VPN rules to send it via the tunnel. In a normal setup, your resources lie behind the inside interface, while the user connects via the outside interface.

Try your packet tracer from the inside interface with source as your ASA's inside interface ip address.

cheeseburger · ‎05-06-2019

Thank you for the explanation, Rahul. Running the packet tracer still shows that the ASA will drop the traffic due to the ACCESS-LIST while using the Inside interface IP.

B6A-DC-THOMPSON-VOIP-ASA# packet-tracer input Inside icmp 10.177.69.69 8 0 10.$

Phase: 1
Type: ROUTE-LOOKUP
Subtype: Resolve Egress Interface
Result: ALLOW
Config:
Additional Information:
found next-hop 10.69.69.10 using egress ifc Outside

Phase: 2
Type: ACCESS-LIST
Subtype:
Result: DROP
Config:
Implicit Rule
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7faa4eb8e880, priority=501, domain=permit, deny=true
hits=0, user_data=0x7, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip/id=10.177.69.69, mask=255.255.255.255, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=Inside, output_ifc=any

Result:
input-interface: Inside
input-status: up
input-line-status: up
output-interface: Outside
output-status: up
output-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule

B6A-DC-THOMPSON-VOIP-ASA#

If I use a different IP, the resulting action says allow. I still cannot successfully ping the VPN client though.

B6A-DC-THOMPSON-VOIP-ASA# packet-tracer input Inside icmp 10.201.10.10 8 0 10.$

(OUTPUT SHORTENED)

Result:
input-interface: Inside
input-status: up
input-line-status: up
output-interface: Outside
output-status: up
output-line-status: up
Action: allow

B6A-DC-THOMPSON-VOIP-ASA#

However, I am able to successfully ping from the VPN client into the internal network but I'm not able to ping from the internal network to the VPN client. I have zero NAT rules and an allow ANY ANY firewall rule. I'm quite confused now.

Rahul Govindan · ‎05-07-2019

"If I use a different IP, the resulting action says allow. I still cannot successfully ping the VPN client though."

This is expected, I forgot about that. The packet-tracer sourced from the ASA interface does not behave really well. If you are seeing the ping allowed from another IP address, this is good. Can you share this output?

Any chance that the PC has some kind of firewall turned on? If you are able to ping one way but not in the other direction, this could indicate your PC blocking icmp echo request, but not a response.

Marvin Rhoads · ‎05-06-2019

It appears you've put an explicit "permit ip any any" ACL on your inside interface. That adds an implicit deny for all other protocols (including icmp which is used by the ping utility).

cheeseburger · ‎05-07-2019

Hi Marvin,

I believe that ICMP falls under IP but just to check your hypothesis, I added rules on Inside and Outside interfaces stating allow any any ICMP both ingress and egress and I still cannot ping the VPN client. I've also created a global rule allow any any IP and ICMP. I believe that should allow on any interface ingress or egress.

This is strange because I can send ICMP echo FROM the client and I get REPLY, but, I cannot send ICMP echo to the client, the ASA drops it.

cheeseburger · ‎05-07-2019

My next step was a Packet Tracer using IP and the Result shows Input Interface: UNKNOWN and Output Interface: UNKNOWN

Any idea why that is? It passes all the Phases and the result says RESULT - The packet is allowed. Route lookup says it knows to take next hop Outside interface which is expected behavior.

cheeseburger · ‎05-08-2019

Windows Firewall was off.

About a week after installing this virtual ASA, we started having some issues on a couple of our hosts not being able to access storage. We eventually deleted the ASA VM and reinstalled it and now things work. I'm able to connect and ping to the laptop and from the laptop to the network. I have no idea why the original ASA was possessed but it seems all my config was correct because I didn't change any of it.

I appreciate everyones help and marked all your posts helpful, thank you for trying to work through this with me. I can now get the phone part up tomorrow and finally complete this POC and move on with life.

Thanks all!!!