I'm not seeing anything

nshoe18 · ‎10-05-2015

We have a remote site with a 5505 connected back to a 5525 in the data center. The L2L connection is good from an IPSEC standpoint, however the users at the remote site cannot access internal resources and we cannot get tot he inside interface on the unit.

Below is a config and a packet trace. Please let me know if you see something strange.

Vertical-BA-5505# packet-tracer input inside icmp 10.100.1.1 10 8 192.168.49.1

Phase: 1
Type: ACCESS-LIST
Subtype:
Result: ALLOW
Config:
Implicit Rule
Additional Information:
MAC Access list

Phase: 2
Type: ROUTE-LOOKUP
Subtype: input
Result: ALLOW
Config:
Additional Information:
in 0.0.0.0 0.0.0.0 outside

Phase: 3
Type: ACCESS-LIST
Subtype:
Result: DROP
Config:
Implicit Rule
Additional Information:

Result:
input-interface: inside
input-status: up
input-line-status: up
output-interface: outside
output-status: up
output-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule

Type help or '?' for a list of available commands.
Vertical-BA-5505> en
Password: ********
Vertical-BA-5505#
Vertical-BA-5505#
Vertical-BA-5505#
Vertical-BA-5505# sh run
: Saved
:
ASA Version 8.2(5)
!
hostname Vertical-BA-5505
domain-name bh.com
enable password /fmyR87pbRr2Ntpz encrypted
passwd /fmyR87pbRr2Ntpz encrypted
names
!
interface Ethernet0/0
switchport access vlan 2
!
interface Ethernet0/1
!
interface Ethernet0/2
!
interface Ethernet0/3
!
interface Ethernet0/4
!
interface Ethernet0/5
!
interface Ethernet0/6
!
interface Ethernet0/7
!
interface Vlan1
nameif inside
security-level 100
ip address 10.100.1.1 255.255.255.0
!
interface Vlan2
nameif outside
security-level 0
ip address 207.140.243.222 255.255.255.252
!
ftp mode passive
clock timezone PST -8
clock summer-time PDT recurring
dns server-group DefaultDNS
domain-name bh.com
same-security-traffic permit inter-interface
same-security-traffic permit intra-interface
access-list nonat extended permit ip 10.100.1.0 255.255.255.0 192.168.49.0 255.255.255.0
access-list nonat extended permit ip 10.100.1.0 255.255.255.0 172.16.81.0 255.255.255.0
access-list nonat extended permit ip 10.100.1.0 255.255.255.0 192.168.50.0 255.255.255.0
access-list nonat extended permit ip 10.100.1.0 255.255.255.0 172.16.50.0 255.255.255.0
access-list nonat extended permit ip 10.100.1.0 255.255.255.0 10.1.3.0 255.255.255.0
access-list nonat extended permit ip 10.100.1.0 255.255.255.0 192.168.47.0 255.255.255.0
access-list nonat extended permit ip 10.100.1.0 255.255.255.0 192.168.48.0 255.255.255.0
access-list nonat extended permit ip 10.100.1.0 255.255.255.0 192.168.57.0 255.255.255.0
access-list nonat extended permit ip 10.100.1.0 255.255.255.0 192.168.58.0 255.255.255.0
access-list outside_access_in extended permit ip 192.168.50.0 255.255.255.0 any
access-list outside_access_in extended permit icmp 192.168.50.0 255.255.255.0 any
access-list outside_access_in extended permit ip 192.168.49.0 255.255.255.0 any
access-list outside_access_in extended permit icmp 192.168.49.0 255.255.255.0 any
access-list outside_access_in extended permit ip 10.1.3.0 255.255.255.0 any
access-list outside_access_in extended permit icmp 10.1.3.0 255.255.255.0 any
access-list outside_access_in extended permit ip any any
access-list outside_access_in extended permit ip 10.100.1.0 255.255.255.0 192.168.49.0 255.255.255.0
access-list outside_access_in extended permit ip 10.100.1.0 255.255.255.0 10.1.3.0 255.255.255.0
access-list outside_access_in extended permit ip 10.100.1.0 255.255.255.0 192.168.50.0 255.255.255.0
access-list outside_access_in extended permit ip 10.100.1.0 255.255.255.0 172.16.50.0 255.255.255.0
access-list outside_access_in extended permit ip 192.168.48.0 255.255.255.0 any
access-list outside_access_in extended permit ip 192.168.47.0 255.255.255.0 any
access-list outside_access_in extended permit icmp 192.168.47.0 255.255.255.0 any
access-list outside_access_in extended permit icmp 192.168.48.0 255.255.255.0 any
access-list outside_access_in extended permit icmp 192.168.57.0 255.255.255.0 any
access-list outside_access_in extended permit icmp 192.168.58.0 255.255.255.0 any
access-list outside_access_in extended permit ip 10.100.1.0 255.255.255.0 192.168.47.0 255.255.255.0
access-list outside_access_in extended permit ip 10.100.1.0 255.255.255.0 192.168.48.0 255.255.255.0
access-list outside_access_in extended permit ip 10.100.1.0 255.255.255.0 192.168.57.0 255.255.255.0
access-list outside_access_in extended permit ip 10.100.1.0 255.255.255.0 192.168.58.0 255.255.255.0
access-list inside_access_in extended permit ip any 192.168.50.0 255.255.255.0
access-list inside_access_in extended permit ip any 192.168.49.0 255.255.255.0
access-list inside_access_in extended permit icmp any any
access-list inside_access_in extended permit ip any 10.1.3.0 255.255.255.0
access-list inside_access_in extended permit ip any any
access-list inside_access_in extended permit icmp 192.168.50.0 255.255.255.0 any
access-list inside_access_in extended permit ip any 192.168.47.0 255.255.255.0
access-list inside_access_in extended permit ip any 192.168.48.0 255.255.255.0
access-list inside_access_in extended permit ip any 192.168.57.0 255.255.255.0
access-list inside_access_in extended permit ip any 192.168.58.0 255.255.255.0
access-list inside_access_in extended permit icmp 192.168.47.0 255.255.255.0 any
access-list inside_access_in extended permit icmp 192.168.48.0 255.255.255.0 any
access-list inside_access_in extended permit icmp 192.168.49.0 255.255.255.0 any
access-list inside_access_in extended permit icmp 192.168.57.0 255.255.255.0 any
access-list inside_access_in extended permit icmp 192.168.58.0 255.255.255.0 any
access-list dallas_l2l_vpn extended permit ip 10.100.1.0 255.255.255.0 192.168.49.0 255.255.255.0
access-list dallas_l2l_vpn extended permit ip 10.100.1.0 255.255.255.0 192.168.50.0 255.255.255.0
access-list dallas_l2l_vpn extended permit ip 10.100.1.0 255.255.255.0 172.16.81.0 255.255.255.0
access-list dallas_l2l_vpn extended permit ip 10.100.1.0 255.255.255.0 10.1.3.0 255.255.255.0
access-list dallas_l2l_vpn extended permit ip 10.100.1.0 255.255.255.0 172.16.50.0 255.255.255.0
access-list dallas_l2l_vpn extended permit ip 10.100.1.0 255.255.255.0 192.168.47.0 255.255.255.0
access-list dallas_l2l_vpn extended permit ip 10.100.1.0 255.255.255.0 192.168.48.0 255.255.255.0
access-list dallas_l2l_vpn extended permit ip 10.100.1.0 255.255.255.0 192.168.57.0 255.255.255.0
access-list dallas_l2l_vpn extended permit ip 10.100.1.0 255.255.255.0 192.168.58.0 255.255.255.0
access-list inside_nat0_outbound extended permit ip 10.100.1.0 255.255.255.0 192.168.50.0 255.255.255.0
access-list inside_nat0_outbound extended permit ip 10.100.1.0 255.255.255.0 192.168.49.0 255.255.255.0
access-list inside_nat0_outbound extended permit ip 10.100.1.0 255.255.255.0 10.1.3.0 255.255.255.0
access-list inside_nat0_outbound extended permit ip 10.100.1.0 255.255.255.0 172.16.50.0 255.255.255.0
access-list inside_nat0_outbound extended permit ip 10.100.1.0 255.255.255.0 192.168.47.0 255.255.255.0
access-list inside_nat0_outbound extended permit ip 10.100.1.0 255.255.255.0 192.168.48.0 255.255.255.0
access-list inside_nat0_outbound extended permit ip 10.100.1.0 255.255.255.0 192.168.57.0 255.255.255.0
access-list inside_nat0_outbound extended permit ip 10.100.1.0 255.255.255.0 192.168.58.0 255.255.255.0
pager lines 24
logging enable
logging asdm informational
mtu inside 1500
mtu outside 1500
icmp unreachable rate-limit 1 burst-size 1
no asdm history enable
arp timeout 14400
global (outside) 1 interface
nat (inside) 0 access-list nonat
nat (inside) 1 0.0.0.0 0.0.0.0
access-group inside_access_in in interface inside
access-group outside_access_in in interface outside
route outside 0.0.0.0 0.0.0.0 207.140.243.221 1
timeout xlate 3:00:00
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute
timeout tcp-proxy-reassembly 0:01:00
timeout floating-conn 0:00:00
dynamic-access-policy-record DfltAccessPolicy
aaa-server TACACS+ protocol tacacs+
aaa-server RADIUS protocol radius
aaa authentication http console LOCAL
aaa authentication telnet console LOCAL
aaa authentication ssh console LOCAL
http server enable
http 192.168.1.0 255.255.255.0 inside
http 10.100.1.0 255.255.255.0 inside
http 0.0.0.0 0.0.0.0 outside
http 192.168.49.0 255.255.255.0 inside
http 192.168.50.0 255.255.255.0 inside
http 192.168.47.0 255.255.255.0 inside
http 192.168.48.0 255.255.255.0 inside
http 10.10.45.0 255.255.255.0 inside
http 192.168.50.0 255.255.255.0 outside
snmp-server host inside 192.168.50.42 community *****
snmp-server location Vertical-BA-5505
snmp-server contact Matt Shelton
snmp-server community *****
snmp-server enable traps snmp authentication linkup linkdown coldstart
crypto ipsec transform-set ESP-3DES-MD5 esp-3des esp-md5-hmac
crypto ipsec security-association lifetime seconds 28800
crypto ipsec security-association lifetime kilobytes 4608000
crypto map outside_map 100 match address dallas_l2l_vpn
crypto map outside_map 100 set peer 162.223.244.101
crypto map outside_map 100 set transform-set ESP-3DES-MD5
crypto map outside_map interface outside
crypto isakmp identity address
crypto isakmp enable outside
crypto isakmp policy 10
authentication pre-share
encryption 3des
hash md5
group 1
lifetime 86400
crypto isakmp policy 30
authentication pre-share
encryption 3des
hash md5
group 2
lifetime 86400
crypto isakmp policy 65535
authentication pre-share
encryption 3des
hash sha
group 2
lifetime 86400
telnet 0.0.0.0 0.0.0.0 inside
telnet 0.0.0.0 0.0.0.0 outside
telnet timeout 60
ssh 0.0.0.0 0.0.0.0 inside
ssh 0.0.0.0 0.0.0.0 outside
ssh timeout 5
console timeout 0
management-access inside
dhcpd dns 207.140.243.194
dhcpd ping_timeout 750
dhcpd auto_config outside
!
dhcpd address 10.100.1.100-10.100.1.200 inside
dhcpd dns 192.168.50.4 207.140.243.194 interface inside
dhcpd domain bh.com interface inside
dhcpd enable inside
!

threat-detection basic-threat
threat-detection statistics access-list
no threat-detection statistics tcp-intercept
webvpn
username ciscoadmin password UybCG/fz9x/XXzg5 encrypted privilege 15
tunnel-group 162.223.244.101 type ipsec-l2l
tunnel-group 162.223.244.101 ipsec-attributes
pre-shared-key *****
isakmp keepalive threshold 20 retry 5
!
class-map inspection_default
match default-inspection-traffic
!
!
policy-map type inspect dns preset_dns_map
parameters
message-length maximum client auto
message-length maximum 512
policy-map global_policy
class inspection_default
inspect dns preset_dns_map
inspect ftp
inspect h323 h225
inspect h323 ras
inspect rsh
inspect rtsp
inspect esmtp
inspect sqlnet
inspect skinny
inspect sunrpc
inspect xdmcp
inspect sip
inspect netbios
inspect tftp
inspect ip-options
!
service-policy global_policy global
prompt hostname context
no call-home reporting anonymous
Cryptochecksum:02b89a17c36d9a32b6a0ac0602144670
: end
Vertical-BA-5505# exit

Logoff

Kyle Fine · ‎10-05-2015

Your no nat ACL is only performing the operation on IP. ICMP is not IP, which is why I believe your packet tracer is failing. Are things working with TCP?

The following traffic defined in your crypto map is being blocked by the access list you have applied to your inside interface:

access-list dallas_l2l_vpn extended permit ip 10.100.1.0 255.255.255.0 172.16.81.0 255.255.255.0
access-list dallas_l2l_vpn extended permit ip 10.100.1.0 255.255.255.0 10.1.3.0 255.255.255.0
access-list dallas_l2l_vpn extended permit ip 10.100.1.0 255.255.255.0 172.16.50.0 255.255.255.0

The command show crypto ipsec sa can show you if packets are being encrypted and decrypted.

Thanks,
Kyle

nshoe18 · ‎10-05-2015

It does not look like we are able to pass TCP at this point either.

So do I need to remove the nonat ACL?

Kyle Fine · ‎10-05-2015

I would add lines to the nonat ACL permitting ICMP in addition to IP.

Before you do that, run the packet tracer command again with TCP instead of ICMP. Does it still drop on Phase 3?

What did you see from the show crypto ipsec sa command? Was there an SA? if so, were packets encrypted and decrypt? or maybe only one or the other? You should see something like the following from that output if the tunnel is up.

      #pkts encaps: 5, #pkts encrypt: 5, #pkts digest: 0
      #pkts decaps: 5, #pkts decrypt: 5, #pkts verify: 5
      #pkts compressed: 0, #pkts decompressed: 0
      #pkts not compressed: 0, #pkts comp failed: 0, #pkts decomp failed: 0
      #pre-frag successes: 0, #pre-frag failures: 0, #fragments created: 0
      #PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0

nshoe18 · ‎10-05-2015

Here is the sh crypto ipsec sa output

Vertical-BA-5505# sh crypto ipsec sa
interface: outside
Crypto map tag: outside_map, seq num: 100, local addr: 207.140.243.222

access-list dallas_l2l_vpn extended permit ip 10.100.1.0 255.255.255.0 192.168.49.0 255.255.255.0
local ident (addr/mask/prot/port): (10.100.1.0/255.255.255.0/0/0)
remote ident (addr/mask/prot/port): (192.168.49.0/255.255.255.0/0/0)
current_peer: 162.223.244.101

#pkts encaps: 18, #pkts encrypt: 18, #pkts digest: 18
#pkts decaps: 24, #pkts decrypt: 24, #pkts verify: 24
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 18, #pkts comp failed: 0, #pkts decomp failed: 0
#pre-frag successes: 0, #pre-frag failures: 0, #fragments created: 0
#PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0
#send errors: 0, #recv errors: 0

local crypto endpt.: 207.140.243.222, remote crypto endpt.: 162.223.244.101

path mtu 1500, ipsec overhead 58, media mtu 1500
current outbound spi: 4D1FB1CF
current inbound spi : 736420CE

inbound esp sas:
spi: 0x736420CE (1935941838)
transform: esp-3des esp-md5-hmac no compression
in use settings ={L2L, Tunnel, }
slot: 0, conn_id: 12288, crypto-map: outside_map
sa timing: remaining key lifetime (kB/sec): (4373997/1527)
IV size: 8 bytes
replay detection support: Y
Anti replay bitmap:
0x00000000 0x01FFFFFF
outbound esp sas:
spi: 0x4D1FB1CF (1293922767)
transform: esp-3des esp-md5-hmac no compression
in use settings ={L2L, Tunnel, }
slot: 0, conn_id: 12288, crypto-map: outside_map
sa timing: remaining key lifetime (kB/sec): (4373995/1527)
IV size: 8 bytes
replay detection support: Y
Anti replay bitmap:
0x00000000 0x00000001

Crypto map tag: outside_map, seq num: 100, local addr: 207.140.243.222

access-list dallas_l2l_vpn extended permit ip 10.100.1.0 255.255.255.0 192.168.50.0 255.255.255.0
local ident (addr/mask/prot/port): (10.100.1.0/255.255.255.0/0/0)
remote ident (addr/mask/prot/port): (192.168.50.0/255.255.255.0/0/0)
current_peer: 162.223.244.101

#pkts encaps: 8334337, #pkts encrypt: 8334337, #pkts digest: 8334337
#pkts decaps: 9119743, #pkts decrypt: 9119740, #pkts verify: 9119740
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 8334337, #pkts comp failed: 0, #pkts decomp failed: 0
#pre-frag successes: 0, #pre-frag failures: 0, #fragments created: 0
#PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0
#send errors: 0, #recv errors: 1079

local crypto endpt.: 207.140.243.222, remote crypto endpt.: 162.223.244.101

path mtu 1500, ipsec overhead 58, media mtu 1500
current outbound spi: 1BCDDB10
current inbound spi : D4560267

inbound esp sas:
spi: 0xD4560267 (3562406503)
transform: esp-3des esp-md5-hmac no compression
in use settings ={L2L, Tunnel, }
slot: 0, conn_id: 12288, crypto-map: outside_map
sa timing: remaining key lifetime (kB/sec): (4373289/2357)
IV size: 8 bytes
replay detection support: Y
Anti replay bitmap:
0xFFFFFFFF 0xFFFFFFFF
outbound esp sas:
spi: 0x1BCDDB10 (466475792)
transform: esp-3des esp-md5-hmac no compression
in use settings ={L2L, Tunnel, }
slot: 0, conn_id: 12288, crypto-map: outside_map
sa timing: remaining key lifetime (kB/sec): (4373202/2356)
IV size: 8 bytes
replay detection support: Y
Anti replay bitmap:
0x00000000 0x00000001

If I do tcp on packet tracer it works.

What prompted this whole issue is that if we try to ssh to the inside interface address on that ASA we are not able to connect to it. We can however connect to the outside interface with no issues.

Kyle Fine · ‎10-06-2015

On the ASA to which you're trying to to connect. Try to issue the command management access inside

nshoe18 · ‎10-06-2015

That command is already active on the device.

Kyle Fine · ‎10-06-2015

I'm not seeing anything jumping out at me from the config. At this point I'd configure logging and check the log for what is happening when you attempt the connection. I like to look at the log from the CLI but ASDM would work too.

logging buffer-size 1048576

logging buffered debugging

logging enable

clear log buff

Then check the log for the ip of the host from which you're attempting to connect.

sh log | in x.x.x.x

I'm not sure what network you're connecting from but make sure that is correctly exempted from NAT on both sides of the tunnel.

-Kyle

S2S VPN Not passing internal trafffic