11-25-2013 06:56 AM
Hi,
I'm experiencing strange problem.
I can't establish SSL VPN connection from 1 IP address, but I don't have problem establishing SSL VPN from any other IP address.
Remote IP address: 10.0.0.1
ASA's public IP address: 192.168.1.1
Output of packet-tracer:
1. with problematic source IP address:
packet-tracer input wan tcp 10.0.0.1 50601 192.168.1.1 443 detailed
Phase: 1
Type: ROUTE-LOOKUP
Subtype: input
Result: ALLOW
Config:
Additional Information:
in 192.168.1.1 255.255.255.255 identity
Phase: 2
Type: ACCESS-LIST
Subtype:
Result: ALLOW
Config:
Implicit Rule
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7fff37573f00, priority=119, domain=permit, deny=false
hits=861, user_data=0x0, cs_id=0x0, flags=0x0, protocol=6
src ip/id=0.0.0.0, mask=0.0.0.0, port=0
dst ip/id=0.0.0.0, mask=0.0.0.0, port=443, dscp=0x0
input_ifc=wan, output_ifc=identity
Phase: 3
Type: CONN-SETTINGS
Subtype:
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7fff38a10a50, priority=8, domain=conn-set, deny=false
hits=4069, user_data=0x7fff38770910, cs_id=0x0, reverse, flags=0x0, protocol=6
src ip/id=0.0.0.0, mask=0.0.0.0, port=0
dst ip/id=192.168.1.1, mask=255.255.255.255, port=443, dscp=0x0
input_ifc=wan, output_ifc=identity
Phase: 4
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7fff395c7d70, priority=0, domain=inspect-ip-options, deny=true
hits=4044934, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0
input_ifc=wan, output_ifc=any
Phase: 5
Type: VPN
Subtype: ipsec-tunnel-flow
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7fff37560700, priority=13, domain=ipsec-tunnel-flow, deny=true
hits=2268518, user_data=0x0, cs_id=0x0, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0
input_ifc=wan, output_ifc=any
Phase: 6
Type: TCP-MODULE
Subtype: webvpn
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7fff38a10cc0, priority=13, domain=soft-np-tcp-module, deny=false
hits=4627, user_data=0x7fff38c14300, cs_id=0x0, reverse, flags=0x0, protocol=6
src ip/id=0.0.0.0, mask=0.0.0.0, port=0
dst ip/id=192.168.1.1, mask=255.255.255.255, port=443, dscp=0x0
input_ifc=wan, output_ifc=identity
Phase: 7
Type: VPN
Subtype: encrypt
Result: DROP
Config:
Additional Information:
Reverse Flow based lookup yields rule:
out id=0x7fff375504a0, priority=69, domain=encrypt, deny=false
hits=40747, user_data=0x0, cs_id=0x7fff3754fa40, reverse, flags=0x0, protocol=0
src ip/id=192.168.1.1, mask=255.255.255.255, port=0
dst ip/id=10.0.0.1, mask=255.255.255.255, port=0, dscp=0x0
input_ifc=any, output_ifc=wan
Result:
input-interface: wan
input-status: up
input-line-status: up
output-interface: NP Identity Ifc
output-status: up
output-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule
If I run packet-tracer with any other source IP address, let's say 10.0.0.2, everything is OK:
packet-tracer input wan tcp 10.0.0.2 50601 192.168.1.1 443 de
Phase: 1
Type: ROUTE-LOOKUP
Subtype: input
Result: ALLOW
Config:
Additional Information:
in 192.168.1.1 255.255.255.255 identity
Phase: 2
Type: ACCESS-LIST
Subtype:
Result: ALLOW
Config:
Implicit Rule
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7fff37573f00, priority=119, domain=permit, deny=false
hits=862, user_data=0x0, cs_id=0x0, flags=0x0, protocol=6
src ip/id=0.0.0.0, mask=0.0.0.0, port=0
dst ip/id=0.0.0.0, mask=0.0.0.0, port=443, dscp=0x0
input_ifc=wan, output_ifc=identity
Phase: 3
Type: CONN-SETTINGS
Subtype:
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7fff38a10a50, priority=8, domain=conn-set, deny=false
hits=4090, user_data=0x7fff38770910, cs_id=0x0, reverse, flags=0x0, protocol=6
src ip/id=0.0.0.0, mask=0.0.0.0, port=0
dst ip/id=192.168.1.1, mask=255.255.255.255, port=443, dscp=0x0
input_ifc=wan, output_ifc=identity
Phase: 4
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7fff395c7d70, priority=0, domain=inspect-ip-options, deny=true
hits=4047886, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0
input_ifc=wan, output_ifc=any
Phase: 5
Type: VPN
Subtype: ipsec-tunnel-flow
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7fff37560700, priority=13, domain=ipsec-tunnel-flow, deny=true
hits=2270040, user_data=0x0, cs_id=0x0, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0
input_ifc=wan, output_ifc=any
Phase: 6
Type: TCP-MODULE
Subtype: webvpn
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7fff38a10cc0, priority=13, domain=soft-np-tcp-module, deny=false
hits=4648, user_data=0x7fff38c14300, cs_id=0x0, reverse, flags=0x0, protocol=6
src ip/id=0.0.0.0, mask=0.0.0.0, port=0
dst ip/id=192.168.1.1, mask=255.255.255.255, port=443, dscp=0x0
input_ifc=wan, output_ifc=identity
Phase: 7
Type: USER-STATISTICS
Subtype: user-statistics
Result: ALLOW
Config:
Additional Information:
Reverse Flow based lookup yields rule:
out id=0x7fff3a1cc320, priority=0, domain=user-statistics, deny=false
hits=4902651, user_data=0x7fff3a0043c0, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0
input_ifc=any, output_ifc=wan
Phase: 8
Type: FLOW-CREATION
Subtype:
Result: ALLOW
Config:
Additional Information:
New flow created with id 4384689, packet dispatched to next module
Module information for forward flow ...
snp_fp_tracer_drop
snp_fp_inspect_ip_options
snp_fp_tcp_normalizer
snp_fp_tcp_mod
snp_fp_adjacency
snp_fp_fragment
snp_fp_drop
Module information for reverse flow ...
snp_fp_tracer_drop
snp_fp_inspect_ip_options
snp_fp_tcp_normalizer
snp_fp_adjacency
snp_fp_fragment
snp_ifc_stat
Result:
input-interface: wan
input-status: up
input-line-status: up
output-interface: NP Identity Ifc
output-status: up
output-line-status: up
Action: allow
I run packet capture on WAN interface - and I can only see incoming packets (SYN) with destination to tcp/443 but there isn't any outgoing packet (SYN/ACK).
I even can't open web page from internet browser (url https://192.168.1.1) when source IP is 10.0.0.1, but I can open "SSL VPN Service" web page from any other source IP address.
The only thing different with this IP address is that there's configured site-to-site (IPsec) vpn tunnel from same source to same destination IP address.
Here is the configuration of the tunnel:
group-policy GroupPolicy_10.0.0.1 internal
group-policy GroupPolicy_10.0.0.1 attributes
vpn-filter value VPN-ACL
vpn-tunnel-protocol ikev1 ssl-client
access-list VPN-ACL:
access-list VPN-ACL extended permit ip object-group DM_INLINE_NETWORK_83 object-group DM_INLINE_NETWORK_84
object-group network DM_INLINE_NETWORK_83
network-object 10.11.217.0 255.255.255.0
network-object 192.168.201.0 255.255.255.0
object-group network DM_INLINE_NETWORK_84
network-object 10.11.217.0 255.255.255.0
network-object 192.168.201.0 255.255.255.0
tunnel local & remote networks:
access-list wan_cryptomap_5 extended permit ip 10.11.217.0 255.255.255.0 192.168.201.0 255.255.255.0
crypto map wan_map 5 match address wan_cryptomap_5
crypto map wan_map 5 set connection-type answer-only
crypto map wan_map 5 set peer 10.0.0.1
crypto map wan_map 5 set ikev1 transform-set ESP-3DES-SHA
I've configured the same setup in my lab and I can't reproduce the error.
The SW version running on ASA is asa861-12.
I'm out of ideas.
11-28-2013 06:23 AM
Hi,
I checked that there isn't any NAT translation happening for this traffic.
When running traffic capture on WAN interface I can only see 4 packets coming
- 3x SYN from remote IP
- 1x RST from remote IP
There wasn't any packet sent back from ASA to remote host.
Traffic filter (acl) for capturing this traffic is defined:
- IP; remote host 10.0.0.1, any
- ICMP; remot host 10.0.0.1, any
- IP; any, remote host 10.0.0.1
- ICMP; any, remote host 10.0.0.1
I can find this line in capture asp-drop type asp-drop: 14:26:49.865693 000b.dee6.4b00 30f7.8d47.9db4 0x0800 62: 10.0.0.1.51285 > 192.168.1.1.443: S [tcp sum ok] 3204589742:3204589742(0) win 8192
Nothing else.
This is the first ACE in WAN's access list:
access-list wan_access_in line 1 extended permit ip host 10.0.0.1 any log emergencies interval 300
Still this is happening with only 1 remote host.
Does anyone have any idea how to further diagnose the problem?
Regards,
Jernej
11-28-2013 08:59 PM
Just collected some other information:
1. traceroute shows that traffic is not leaving ASA at all
1 * * *
2 * * *
3 * * *
.......
I double checked that there is no "strange" entry for remote public IP in routing. Traffic with destination to remote IP should be sent via default gateway like all other traffic.
2. debug crypto ipsec shows this information when I ping public IP address of the remote host (with VPN
IPSEC(crypto_map_check)-3: Looking for crypto map matching 5-tuple: Prot=1, saddr=192.168.1.1, sport=30647, daddr=10.0.0.1, dport=30647
IPSEC(crypto_map_check)-5: Checking crypto map wan_map 1: skipping because 5-tuple does not match ACL wan_cryptomap_1.
IPSEC(crypto_map_check)-5: Checking crypto map wan_map 2: skipping because 5-tuple does not match ACL wan_cryptomap_2.
IPSEC(crypto_map_check)-5: Checking crypto map wan_map 3: skipping because 5-tuple does not match ACL wan_cryptomap_3.
IPSEC(crypto_map_check)-5: Checking crypto map wan_map 4: skipping because 5-tuple does not match ACL wan_cryptomap_4.
IPSEC(crypto_map_check)-5: Checking crypto map wan_map 5: skipping dormant map.
IPSEC(crypto_map_check)-5: Checking crypto map wan_map 5: skipping dormant map.
IPSEC(crypto_map_check)-5: Checking crypto map wan_map 6: skipping because 5-tuple does not match ACL wan_cryptomap_6.
IPSEC(crypto_map_check)-5: Checking crypto map wan_map 7: skipping because 5-tuple does not match ACL wan_cryptomap_7.
IPSEC(crypto_map_check)-5: Checking crypto map wan_map 8: skipping because 5-tuple does not match ACL wan_cryptomap_8.
IPSEC(crypto_map_check)-5: Checking crypto map wan_map 9: skipping because 5-tuple does not match ACL wan_cryptomap_9.
IPSEC(crypto_map_check)-5: Checking crypto map wan_map 10: skipping because 5-tuple does not match ACL wan_cryptomap_10.
IPSEC(crypto_map_check)-5: Checking crypto map wan_map 11: skipping because 5-tuple does not match ACL wan_cryptomap_11.
IPSEC(crypto_map_check)-5: Checking crypto map wan_map 13: skipping because 5-tuple does not match ACL wan_cryptomap_13.
IPSEC(crypto_map_check)-5: Checking crypto map wan_map 65535: skipping dynamic_link.
IPSEC(crypto_map_check)-1: Error: No crypto map matched.
It really seems that the whole problem is that ASA is trying to encrypt traffic sent from public IP address of one VPN endpoint and targeted to public IP address of another VPN endpoint and send it to remote VPN endpoint via IPcec tunel.
There is indeed VPN tunnel established between both VPN endpoints, but there are just local and remote networks defined with private IP address space for this tunnel, VPN endpoint's public IP addresses are not included in the definition of this IPsec VPN tunnel.
And there are at least two more IPsec VPN tunnels configured the same way and I can't reprodure this error on there two VPN tunnels.
Any idea?
11-29-2013 01:55 AM
I don't understand why you have a crypto map entry (IPsec) for 10.0.0.1 if this client should connect via SSL. Also: according to documentation the Anyconnect client doesn't support IKEv1, only IKEv2 can be used for Anyconnect.
11-29-2013 02:14 AM
Hi,
I have crypto map entry for 10.0.0.1 because there is site-to-site VPN tunnel established between both locations: for servers to communicate with each other in both data centers.
But additionaly to this site-to-site IPsec VPN tunnel there are some people at remote location that want to establish SSL VPN remote connection that grants them additional access.
I would also like to mention: this setup was working fine until the problem occured about one week ago. The only thing that I'm aware of is that one week ago there was ASA's firmware updated from 8.6.1-2 to 8.6.1-12. But even when I downgrade firmware back to 8.6.1-2 the issue is still here.
ASA is encrypting traffic targeted to remote WAN IP address, even if it is TCP or ICMP traffic.
ikve1 debug shows: [IKE COMMON DEBUG]Tunnel Manager failed to dispatch a KEY_ACQUIRE message. Probable mis-configuration of the crypto map or tunnel-group. Map Tag = Unknown. Map Sequence Number = 0.
But the tunnel group for IPsec tunnel only includes remote and local network.
11-29-2013 05:47 AM
The problem was in ipsec vpn tunel type: when it was changed from respond only to bidirectional, it started working.
Is this a bug or am I missing something?
11-29-2013 06:00 AM
Honestly, I can't give an answer whether its a bug or by design.
To my understaning it shouldn't have an effect for incoming connections.
you can read on
"This document provides information on how to configure a VPN gateway device to always act as a responder in an IKE negotiation. The device will respond to any crypto negotiations initiated by its peers."
So it seems to me a software bug...
11-30-2013 04:55 AM
Is there any way to submit bug report to help Cisco create patch for this bug in next ASA software versions (ASA is not covered by Smartnet but I don't need any service from TAC center, just want to help improve software)?
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide