Cisco ASA 5505 - SYN Timeout

[Tarran]

Cisco ASA 5505

Cisco Adaptive Security Appliance Software Version 7.2(4)

Device Manager Version 5.2(4)

Hi all,

I have and vpn tunnel between a pix network (192.168.200.0/24) and an asa network (192.168.100.0/24); it's been running fine for awhile now but this morning i've come in an i can not access anything on the pix network, (mail, file & web servers). Each attempt to access results in a SYN timeout.

6 Nov 30 2012 14:24:01 302014 192.168.200.9 192.168.100.115  Teardown TCP connection 6014 for outside:192.168.200.9/135 to inside:192.168.100.115/51240 duration 0:00:30 bytes 0 SYN Timeout

Can anyone suggest what the issue is... and a fix please?

Many thanks,

Tarran

[Jouni Forss]

Hi,

• Is the L2L VPN up even though the connections dont go through?
• When attempting the connections, does the VPN connections counters go up?
  • show crypto ipsec sa peer x.x.x.x
• Are you managing both Firewalls? Can you access the remote sides firewall to check the situation there?
• Have any changes been made?
• Are you testing connections only to single host on the remote side or is there any other host to which you can test connections to?

- Jouni

[Tarran]

Hi Jouni,

In reply:

- The VPN is up.

- The counters also go up per connection attempt

- i manage both sides of the vpn and i haven't made changes in atleast a few days.... the asa is a very simple config setup.

- i am testing multiple hosts and all seem to come back with syn timeout.

I have also restarted both sides of the vpn.

Thanks,

Tarran

[flyntransam]

delete me

[Jack Leung]

I'm assuming this is your basic ipsec site to site. Check and make sure the IPSEC tunnel is being built on either side with the

show crypto ipsec sa

and make sure the two networks are talking.

Or, you could simply try restarting the VPN if nothing is working at all?

clear crypto ipsec sa peer IPADDRESS

If the restart doesnt work then I would verify if there have been any changes on either end.  Some of the things to look for includes, NO NAT statements removed or changed.

[Tarran]

Hello Jack,

I can confirm the vpn tunnel is being built on both sides. I have also tried restarting the vpn but with no luck.

Thanks,

Tarran

[Jouni Forss]

Hi,

Regarding the TCP connection attempts, can you confirm that they are being "Built" on the remote side firewall? As in that the remote firewall can see the SYN from the connection initiator?

If you can see the connections being formed on the remote firewall it would seem more logical that there is either some configurations change made on that side (since no host can bring up a TCP connection) or perhaps is something wrong with the actual firewall.

Have you tried booting up the firewalls?

Can you see any TCP connections from the hosts on the remote network? Are they connecting to anything, even to Internet?

Is there any routers behind the remote firewall or just flat switch network? Can you see anythng behind the remote firewall with "show arp" command? (Clear the arp with "clear arp" if they are old markings)

- Jouni

[Tarran]

Hi Jouni,

Hmmm, how can i check that? This is an extract fro "sh conn" on the remote pix (192.168.200.253) -  is that what i should be looking at?

TCP out 192.168.100.5:135 in 192.168.200.122:4733 idle 0:01:28 Bytes 0 flags saA
TCP out 192.168.100.2:135 in 192.168.200.3:4242 idle 0:00:30 Bytes 0 flags saA
TCP out 192.168.100.2:135 in 192.168.200.3:4243 idle 0:00:09 Bytes 0 flags saA

this is an sh conn extrct from the asa (192.168.100.253)

TCP outside 192.168.200.55:135 inside 192.168.100.115:53181, idle 0:00:03, bytes 0, flags saA
TCP outside 192.168.200.66:135 inside 192.168.100.115:53180, idle 0:00:02, bytes 0, flags saA

Both devices have internet access and don't have any routers behind the firewalls.

I believe the problem is to be on the asa not the remote pix (but may be wrong)..... i am the only person with access to the pix and i haven't made a change for around six months but i did make a change to the asa a few days back... ill post the config but i think is a simple one so wonder if it is the hardware...

: Saved
:
ASA Version 7.2(4)
!
hostname bosfw
enable password 1nbBm6rSIvdi3SGH encrypted
passwd 2KFQnbNIdI.2KYOU encrypted
names
name 192.168.100.0 Network-Boston
name 192.168.200.0 Network-London
name 192.168.40.0 Network-HongKong
name 192.168.101.0 Network-Boston-VPN
!
interface Vlan1
 nameif inside
 security-level 100
 ip address 192.168.100.253 255.255.255.0
!
interface Vlan2
 nameif outside
 security-level 0
 ip address 10.1.10.2 255.255.255.0
!
interface Ethernet0/0
 switchport access vlan 2
!
interface Ethernet0/1
!
interface Ethernet0/2
!
interface Ethernet0/3
!
interface Ethernet0/4
!
interface Ethernet0/5
!
interface Ethernet0/6
!
interface Ethernet0/7
!
ftp mode passive
dns server-group DefaultDNS
 domain-name domain.com
access-list outside_1_cryptomap extended permit ip Network-Boston 255.255.255.0 Network-London 255.255.255.0
access-list inside_nat0_outbound extended permit ip Network-Boston 255.255.255.0 Network-London 255.255.255.0
access-list inside_nat0_outbound extended permit ip Network-Boston 255.255.255.0 Network-HongKong 255.255.255.0
access-list inside_nat0_outbound extended permit ip Network-Boston 255.255.255.0 Network-Boston-VPN 255.255.255.0
access-list outside_2_cryptomap extended permit ip Network-Boston 255.255.255.0 Network-HongKong 255.255.255.0
access-list vpnuser_splitTunnelAcl standard permit Network-Boston 255.255.255.0
pager lines 24
logging enable
logging asdm informational
mtu inside 1500
mtu outside 1500
ip local pool VPNDHCP 192.168.101.100-192.168.101.200 mask 255.255.255.0
ip verify reverse-path interface outside
icmp unreachable rate-limit 1 burst-size 1
icmp permit any inside
asdm image disk0:/asdm-524.bin
asdm history enable
arp timeout 14400
global (outside) 1 interface
nat (inside) 0 access-list inside_nat0_outbound
nat (inside) 1 0.0.0.0 0.0.0.0
route outside 0.0.0.0 0.0.0.0 10.1.10.1 1
timeout xlate 3:00:00
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute
aaa-server CyptoCard protocol radius
aaa-server CyptoCard (inside) host 192.168.100.1
 key password
 radius-common-pw password
no eou allow clientless
http server enable
http Network-Boston 255.255.255.0 inside
http Network-HongKong 255.255.255.0 inside
http Network-London 255.255.255.0 inside
http Network-Boston-VPN 255.255.255.0 outside
no snmp-server location
no snmp-server contact
snmp-server enable traps snmp authentication linkup linkdown coldstart
crypto ipsec transform-set ESP-DES-MD5 esp-des esp-md5-hmac
crypto ipsec transform-set ESP-3DES-SHA esp-3des esp-sha-hmac
crypto dynamic-map outside_dyn_map 20 set transform-set ESP-3DES-SHA
crypto map outside_map 1 match address outside_1_cryptomap
crypto map outside_map 1 set peer 62.X.X.X
crypto map outside_map 1 set transform-set ESP-DES-MD5
crypto map outside_map 2 match address outside_2_cryptomap
crypto map outside_map 2 set peer 116.X.X.X
crypto map outside_map 2 set transform-set ESP-DES-MD5
crypto map outside_map 65535 ipsec-isakmp dynamic outside_dyn_map
crypto map outside_map interface outside
crypto isakmp enable outside
crypto isakmp policy 10
 authentication pre-share
 encryption des
 hash md5
 group 2
 lifetime 86400
crypto isakmp policy 30
 authentication pre-share
 encryption 3des
 hash sha
 group 2
 lifetime 86400
telnet Network-London 255.255.255.0 inside
telnet Network-HongKong 255.255.255.0 inside
telnet Network-Boston 255.255.255.0 inside
telnet timeout 5
ssh Network-London 255.255.255.0 inside
ssh Network-HongKong 255.255.255.0 inside
ssh Network-Boston 255.255.255.0 inside
ssh Network-Boston-VPN 255.255.255.0 inside
ssh timeout 5
console timeout 0
management-access inside
dhcpd auto_config outside
!
group-policy vpnuser internal
group-policy vpnuser attributes
 dns-server value 192.168.100.1 8.8.8.8
 vpn-tunnel-protocol IPSec
 split-tunnel-policy tunnelspecified
 split-tunnel-network-list value vpnuser_splitTunnelAcl
 default-domain value arete.net
tunnel-group 62.X.X.X type ipsec-l2l
tunnel-group 62.X.X.X ipsec-attributes
 pre-shared-key *
tunnel-group 116.X.X.X type ipsec-l2l
tunnel-group 116.X.X.X ipsec-attributes
 pre-shared-key *
tunnel-group vpnuser type ipsec-ra
tunnel-group vpnuser general-attributes
 address-pool VPNDHCP
 authentication-server-group CyptoCard
 default-group-policy vpnuser
tunnel-group vpnuser ipsec-attributes
 pre-shared-key *
!
class-map inspection_default
 match default-inspection-traffic
!
!
policy-map type inspect dns preset_dns_map
 parameters
  message-length maximum 512
policy-map global_policy
 class inspection_default
  inspect dns preset_dns_map
  inspect ftp
  inspect h323 h225
  inspect h323 ras
  inspect rsh
  inspect rtsp
  inspect esmtp
  inspect sqlnet
  inspect skinny
  inspect sunrpc
  inspect xdmcp
  inspect sip
  inspect netbios
  inspect tftp
!
service-policy global_policy global
prompt hostname context
Cryptochecksum:cbb7494c7b6b48db5d01a8fc7f1ce042
: end

[Tarran]

Just a thought but could the asa shun the remote network?

[Jouni Forss]

Hi,

Are those ALL connections showing on the PIX side? Can you see any other connections on the PIX?

What does the "show arp" show on the PIX side? Does it show anything for the LAN interface? If it does can you clear the arp with "clear arp" command and see if any of the previous IP/MAC appear on the PIX arp table (could naturally ping the previous ones also to see if they appear)

If I am reading the TCP Flags correctly

• PIX has seen the SYN from the host behind ASA initiating the TCP connection
  • Therefore connection attempt has come through the L2L VPN
• But this is where it stops for PIX also
  • To my understanding it sees the TCP SYN from the Client/host on attempting the connection from behind the ASA but doesnt get ANY reply from the host behind the PIX. So even the PIX doesnt see any return traffic for the TCP connection forming from hosts behind it.
• Usually the above means that
  • Either the hosts are blocking the connection
  • They have incorrect default gateway
  • There is some other routing problem related to the return traffic that is forwarding the traffic to a wrong place
  • Some device in between broken down or some connection is down

I would really check ARP on the PIX. If you say there is no routers behind the PIX, the PIX should see every single host behind it on its ARP table. Are you sure there is not some kind of problem with the network behind PIX (broken LAN switch)? Have you tried to connect with several different ports?

I presume this network is a production one (is in real use)? Have you changed the ASA outside IP address to private for the purpose of this forum post? Or do you have a NAT device in front of the ASA doing static NAT for ASAs outside interface?

- Jouni

[Tarran]

Hi Jouni,

This is strange as is working now after just restarting the comcast router and asa (i had already done this). I think either the ASA or the comcast router is on the way out.

Just to reply to some of the things you mention as really don't want this kind of thing happening where i don't have a clue to what went wrong or why is started working again:

- I know (well, seems most likely) the hosts had the correct details on the PIX side as other sites can access them (and there are multiple hosts also and would be strange that they all had died) as can also the clients at the pix site.

- the pix can see all hosts behind it; there are no routers. it is possible a switch is on it's way out on the pix site though.

I presume this network is a production one (is in real use)? Have you changed the ASA outside IP address to private for the purpose of this forum post? Or do you have a NAT device in front of the ASA doing static NAT for ASAs outside interface?

This is a production network; i didn't change the outside interface ip address to a private one - it has a router doing static nat; the router is provided by comcast and this is how they provide the router and address range, not allowing you to access or change.

With the above info, does it seem likely that it is the asa or comcast router that is having the issues?

[Julio Carvajal]

Hello Tarran,

I would recommend next time to run captures on both side,

This will allow you to make sure the ASA and the PIX are receiving the traffic and ofcourse that all the host are acting as they should, then we will be able to determine where is the problem...

At this time I agree with Jouni about what the flags are letting us know is that the SYN is being sent and no SYN-ACK is received but again captures next time will help us

Regards,

Julio