Solved: Can't access server on different subnet (ASA issue I believe)

mitchell helton · ‎09-08-2011

First off, let me preface this by saying that I'm a novice when it comes to firewalls and more specifically, the ASA. I do however, have an above average understanding of switches/routers.

We have an ASA 5510 running 8.3 and recently I've decided to clean up the last admin's mess. All hosts and servers are on the same subnet, multiple subnets on the same VLAN... and a slew of other problems. Anyway, I recently placed the IT department on another subnet to test some things out before I migrated other departments to different networks. Everything seems to be working as it should be with the exception of one of our servers. The IT subnet is 192.168.150.0/24 and the problem server is on the 192.168.10.xxx network. I'm guessing the issue lies somewhere in the fact this server does have a static NAT and is accessible from the public. Let me give you an overview of what our network looks like:

ISP ---->ASA----->3750----->2960

My workstation is directly plugged into the 3750 switch, and the server is plugged into the 2960. I'm able to ping this server by both IP and hostname. However, I cannot access port 80 by IP or hostname. The users that are on the 192.168.10 and 192.168.11 (sadly both of those are on the same VLAN) network are able to access this server without a problem. Thinking logically, I thought I would send a packet from my workstation, it would head to the layer 3 switch's VLAN interface corresponding to my subnet, realize the .10 network is directly connected and then forward the packet straight to the server. However, it doesn't seem to be working that way. It look like it's being routed to the ASA then being dropped. I guess there's an access rule or firewall rule preventing me from getting to the server. Is there a specific part of my config you will need to see... or do I need to post all of it? Thanks for your time.

varrao · ‎09-08-2011

Yes, the traffic isnt even reaching the firewall so we might need to troubleshoot why? We might need to check the routing and trace the packets on the switches.

-Varun

Thanks,
Varun Rao

View solution in original post

varrao · ‎09-08-2011

Hi Mitch,

Yes, you might need to post the config in order to nail the situation. What you might be doing is u-turning on the firewall and we might need to configure it so that the ASA allows the packets back into the same interface. What I would like to know is:

Are you accessing the server on public ip?

Are both the source and destination on the same interface of the ASA?

Are you able to access the server on private ip?

I guess this info along with config should be good enough to get started on it.

Thanks,

Varun

Thanks,
Varun Rao

mitchell helton · ‎09-08-2011

Thank you for the quick response sir!

I'm trying to access the private IP which is 192.168.10.59. I cannot access the public IP either.

Our ASA connects to our internal network on the 192.168.15.xxx subnet which is connected to the 3750 switch. The server resides on the 2960 switch which is directly connected to the 3750.

I cannot access the server, via http or https, with any name or IP address. I can however ping the server by name AND IP. The hosts on the .11 and .10 networks are able to access the server without a problem. My machine is on the .150 network.

ASA Version 8.3(2)

!

hostname ciscoasa

domain-name xxxxxxxxx

enable password 5UAWulVGFDL9UTag encrypted

passwd 5UAWulVGFDL9UTag encrypted

names

!

interface Ethernet0/0

description WAN connection to Internet

nameif WAN

security-level 0

ip address xxx.xxx.xxx.xxx 255.255.255.0

!

interface Ethernet0/1

description LAN connection to internal network

nameif LAN

security-level 100

ip address 192.168.15.2 255.255.255.0

!

interface Ethernet0/2

description DMZ

nameif DMZ

security-level 50

ip address 192.168.50.254 255.255.255.0

!

interface Ethernet0/3

shutdown

no nameif

no security-level

no ip address

!

interface Management0/0

nameif management

security-level 100

ip address 192.168.1.1 255.255.255.0

management-only

boot system disk0:/asa832-k8.bin

ftp mode passive

clock timezone EST -5

clock summer-time EDT recurring

dns domain-lookup WAN

dns domain-lookup LAN

dns domain-lookup management

dns server-group DefaultDNS

name-server 192.168.10.200

domain-name xxxxxxx

object network 192.168.10.59

host 192.168.10.59

object network All_Inside_Networks

subnet 0.0.0.0 0.0.0.0

object network NAT-Pool

range xxx.xxx.xxx.xxx xxx.xxx.xxx.xxx

object network Nat_pool

range xxx.xxx.xxx.xxx xxx.xxx.xxx.xxx

object network NETWORK_OBJ_192.168.15.0_24

subnet 192.168.15.0 255.255.255.0

object network NETWORK_OBJ_192.168.250.0_25

subnet 192.168.250.0 255.255.255.128

object network 192.168.10.0

subnet 192.168.10.0 255.255.255.0

object network 10.1.5.0

subnet 10.1.5.0 255.255.255.0

object network 192.168.11.0

subnet 192.168.11.0 255.255.255.0

object network 192.168.150.0

subnet 192.168.150.0 255.255.255.0

object-group network DM_INLINE_NETWORK_1

network-object host 10.1.5.0

network-object host 10.1.50.0

network-object host 10.1.51.0

object-group protocol DM_INLINE_PROTOCOL_1

protocol-object icmp

protocol-object udp

protocol-object tcp

object-group service DM_INLINE_SERVICE_1

service-object tcp destination eq 1755

service-object tcp destination eq www

service-object tcp destination eq https

service-object udp destination eq 1755

object-group service DM_INLINE_SERVICE_2

service-object gre

service-object tcp destination eq pptp

object-group service DM_INLINE_SERVICE_3

service-object tcp destination eq 5222

service-object tcp destination eq www

service-object tcp destination eq ssh

service-object udp destination range 10000 20000

service-object udp destination eq 4569

service-object udp destination eq sip

service-object udp destination range 3000 3200

service-object tcp destination eq sip

object-group service DM_INLINE_SERVICE_4

service-object icmp

service-object udp

service-object tcp destination eq https

object-group service DM_INLINE_SERVICE_5

service-object gre

service-object tcp destination eq pptp

object-group service DM_INLINE_TCP_1 tcp

port-object eq www

port-object eq https

object-group service DM_INLINE_TCP_10 tcp

port-object eq www

port-object eq https

object-group service DM_INLINE_TCP_11 tcp

port-object eq ftp

port-object eq www

port-object eq https

object-group service DM_INLINE_TCP_2 tcp

port-object eq www

port-object eq https

object-group service DM_INLINE_TCP_3 tcp

port-object eq www

port-object eq https

object-group service DM_INLINE_TCP_4 tcp

port-object eq www

port-object eq https

object-group service DM_INLINE_TCP_5 tcp

port-object eq www

port-object eq https

object-group service DM_INLINE_TCP_6 tcp

port-object eq 5150

port-object eq ftp

port-object eq www

port-object eq https

object-group service DM_INLINE_TCP_7 tcp

port-object eq 3101

port-object eq 995

object-group service DM_INLINE_TCP_8 tcp

port-object eq pop3

port-object eq smtp

object-group service DM_INLINE_TCP_9 tcp

port-object eq 3389

port-object eq 88

object-group protocol TCPUDP

protocol-object udp

protocol-object tcp

object-group service DM_INLINE_TCP_12 tcp

port-object eq ftp

port-object eq www

port-object eq https

port-object eq 5150

object-group service DM_INLINE_TCP_13 tcp

port-object eq 5150

port-object eq ftp

port-object eq www

port-object eq https

object-group service DM_INLINE_TCP_14 tcp

port-object eq 5150

port-object eq ftp

port-object eq www

port-object eq https

object-group service DM_INLINE_TCP_15 tcp

port-object eq www

port-object eq https

object-group service DM_INLINE_SERVICE_6

service-object icmp

service-object tcp destination eq 3389

access-list WAN_access_in remark xxxxxxxxx

access-list WAN_access_in extended permit tcp any object 192.168.10.59 eq www

access-list WAN_access_in extended permit object-group DM_INLINE_PROTOCOL_1 any object 192.168.10.2 inactive

access-list WAN_access_in extended permit object-group DM_INLINE_SERVICE_4 any object 192.168.10.59

access-list WAN_access_in extended permit tcp any object 192.168.10.192 eq www inactive

access-list WAN_access_in extended permit object-group DM_INLINE_SERVICE_5 any object 192.168.10.221

access-list WAN_access_in extended permit ip any object-group DM_INLINE_NETWORK_1 inactive

access-list WAN_access_in extended permit tcp host 10.1.10.0 any object-group DM_INLINE_TCP_11 inactive

access-list WAN_access_in extended permit icmp any any echo

access-list WAN_access_in extended permit icmp any any traceroute

access-list WAN_access_in extended permit icmp any any echo-reply

access-list WAN_access_in extended permit icmp any object 192.168.10.214 echo-reply

access-list VPNUsers_splitTunnelAcl standard permit 192.168.10.0 255.255.255.0

access-list VPNUsers_splitTunnelAcl standard permit 192.168.15.0 255.255.255.0

access-list VPNUsers_splitTunnelAcl standard permit 192.168.11.0 255.255.255.0

access-list VPNUsers_splitTunnelAcl standard permit 10.1.5.0 255.255.255.0

access-list VPNUsers_splitTunnelAcl standard permit 192.168.100.0 255.255.255.0

access-list VPNUsers_splitTunnelAcl standard permit 192.168.101.0 255.255.255.0

access-list VPNUsers_splitTunnelAcl standard permit 192.168.102.0 255.255.255.0

access-list VPNUsers_splitTunnelAcl standard permit 192.168.150.0 255.255.255.0

access-list LAN_access_in extended permit ip any any

pager lines 24

logging enable

logging timestamp

logging list VPN_Logs level informational class vpn

logging monitor notifications

logging buffered notifications

logging trap notifications

logging history VPN_Logs

logging asdm notifications

logging mail alerts

logging from-address xxxxxxxxxxxxxxxxx

logging recipient-address xxxxxxxxxxxxxxx level critical

logging facility 21

logging host LAN 192.168.150.97

logging permit-hostdown

no logging message 106015

no logging message 313001

no logging message 313008

no logging message 106023

no logging message 710003

no logging message 106100

no logging message 302015

no logging message 302014

no logging message 302013

no logging message 302018

no logging message 302017

no logging message 302016

no logging message 302021

no logging message 302020

flow-export destination LAN 192.168.10.224 2055

mtu WAN 1500

mtu LAN 1500

mtu management 1500

mtu DMZ 1500

ip local pool VPNDHCP 192.168.250.1-192.168.250.100 mask 255.255.255.0

no failover

icmp unreachable rate-limit 1 burst-size 1

asdm image disk0:/asdm-634-53.bin

no asdm history enable

arp timeout 14400

nat (LAN,WAN) source static 192.168.10.0 192.168.10.0 destination static NETWORK_OBJ_192.168.250.0_25 NETWORK_OBJ_192.168.250.0_25

nat (WAN,WAN) source static NETWORK_OBJ_192.168.15.0_24 NETWORK_OBJ_192.168.15.0_24 destination static NETWORK_OBJ_192.168.250.0_25 NETWORK_OBJ_192.168.250.0_25

nat (LAN,WAN) source static 192.168.11.0 192.168.11.0 destination static NETWORK_OBJ_192.168.250.0_25 NETWORK_OBJ_192.168.250.0_25

nat (LAN,WAN) source static 10.1.5.0 10.1.5.0 destination static NETWORK_OBJ_192.168.250.0_25 NETWORK_OBJ_192.168.250.0_25

nat (LAN,WAN) source static 192.168.100.0 192.168.100.0 destination static NETWORK_OBJ_192.168.250.0_25 NETWORK_OBJ_192.168.250.0_25

nat (LAN,WAN) source static 192.168.150.0 192.168.150.0 destination static NETWORK_OBJ_192.168.250.0_25 NETWORK_OBJ_192.168.250.0_25

!

object network 192.168.10.59

nat (LAN,WAN) static xxx.xxx.xxx.xxx

!

nat (LAN,WAN) after-auto source dynamic All_Inside_Networks interface

access-group WAN_access_in in interface WAN

access-group LAN_access_in in interface LAN

route WAN 0.0.0.0 0.0.0.0 xxx.xxx.xxx.xxx xxx

route LAN 10.1.5.0 255.255.255.0 192.168.15.1 1

route LAN 10.1.10.0 255.255.254.0 192.168.15.1 1

route LAN 10.1.20.0 255.255.254.0 192.168.15.1 1

route LAN 10.1.30.0 255.255.254.0 192.168.15.1 1

route WAN 10.1.50.0 255.255.255.0 xxxxxxxxxxxxxx

route WAN 10.1.51.0 255.255.255.0 xxxxxxxxxxx

route LAN 10.1.160.0 255.255.240.0 192.168.15.1 1

route LAN 192.168.0.0 255.255.0.0 192.168.15.1 1

route LAN 192.168.150.0 255.255.255.0 192.168.15.1 1

timeout xlate 3:00:00

timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02

timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00

timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00

timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute

timeout tcp-proxy-reassembly 0:01:00

dynamic-access-policy-record DfltAccessPolicy

action terminate

dynamic-access-policy-record "Allow VPN Access"

description "Allow VPN access to AD group VPN Users"

aaa-server DC protocol ldap

aaa-server DC (LAN) host 192.168.10.200

ldap-base-dn dc=xxxxxx,dc=local

ldap-scope subtree

ldap-naming-attribute sAMAccountName

ldap-login-password *****

ldap-login-dn cn=xxxxxxxxx,ou=domain resources,dc=xxxxxxx,dc=local

server-type microsoft

aaa authentication enable console LOCAL

aaa authentication http console LOCAL

aaa authentication ssh console LOCAL

aaa authentication telnet console LOCAL

http server enable

http 192.168.1.0 255.255.255.0 management

http 192.168.15.0 255.255.255.0 LAN

http 192.168.250.0 255.255.255.0 WAN

http 192.168.150.0 255.255.255.0 LAN

snmp-server host LAN 192.168.10.224 community ***** udp-port 161

no snmp-server location

snmp-server contact xxxxxxxxx

snmp-server community *****

snmp-server enable traps snmp authentication linkup linkdown coldstart

snmp-server enable traps syslog

snmp-server enable traps ipsec start stop

snmp-server enable traps entity config-change fru-insert fru-remove

snmp-server enable traps remote-access session-threshold-exceeded

crypto ipsec transform-set ESP-3DES-SHA esp-3des esp-sha-hmac

crypto ipsec transform-set ESP-AES-256-MD5 esp-aes-256 esp-md5-hmac

crypto ipsec transform-set ESP-DES-SHA esp-des esp-sha-hmac

crypto ipsec transform-set ESP-DES-MD5 esp-des esp-md5-hmac

crypto ipsec transform-set ESP-AES-192-MD5 esp-aes-192 esp-md5-hmac

crypto ipsec transform-set ESP-3DES-MD5 esp-3des esp-md5-hmac

crypto ipsec transform-set ESP-AES-256-SHA esp-aes-256 esp-sha-hmac

crypto ipsec transform-set ESP-AES-128-SHA esp-aes esp-sha-hmac

crypto ipsec transform-set ESP-AES-192-SHA esp-aes-192 esp-sha-hmac

crypto ipsec transform-set ESP-AES-128-MD5 esp-aes esp-md5-hmac

crypto ipsec security-association lifetime seconds 28800

crypto ipsec security-association lifetime kilobytes 4608000

crypto dynamic-map SYSTEM_DEFAULT_CRYPTO_MAP 65535 set transform-set ESP-AES-128-SHA ESP-AES-128-MD5 ESP-AES-192-SHA ESP-AES-192-MD5 ESP-AES-256-SHA ESP-AES-256-MD5 ESP-3DES-SHA ESP-3DES-MD5 ESP-DES-SHA ESP-DES-MD5

crypto map WAN_map 65535 ipsec-isakmp dynamic SYSTEM_DEFAULT_CRYPTO_MAP

crypto map WAN_map interface WAN

crypto isakmp enable WAN

crypto isakmp policy 10

authentication pre-share

encryption 3des

hash sha

group 2

lifetime 86400

telnet timeout 5

ssh 192.168.150.0 255.255.255.0 LAN

ssh timeout 5

console timeout 0

dhcpd address 192.168.1.2-192.168.1.254 management

dhcpd enable management

!

threat-detection basic-threat

threat-detection statistics

threat-detection statistics tcp-intercept rate-interval 30 burst-rate 400 average-rate 200

webvpn

enable WAN

smart-tunnel list AllExternalApplications All-Applications * platform windows

group-policy DfltGrpPolicy attributes

webvpn

smart-tunnel enable AllExternalApplications

group-policy VPNUsers internal

group-policy VPNUsers attributes

dns-server value 192.168.10.200 192.168.10.201

vpn-tunnel-protocol IPSec svc webvpn

split-tunnel-policy tunnelspecified

split-tunnel-network-list value VPNUsers_splitTunnelAcl

default-domain value xxxxxxxxx

username admin password jSoYj.edDiNeZnUo encrypted privilege 15

username sycom password VnaY6K57B2JxJva3 encrypted privilege 15

tunnel-group VPNUsers type remote-access

tunnel-group VPNUsers general-attributes

address-pool VPNDHCP

authentication-server-group DC

default-group-policy VPNUsers

password-management

tunnel-group VPNUsers webvpn-attributes

radius-reject-message

tunnel-group VPNUsers ipsec-attributes

pre-shared-key *****

!

class-map global-class

match default-inspection-traffic

class-map inspection_default

match default-inspection-traffic

class-map global-class1

match any

!

policy-map type inspect dns preset_dns_map

parameters

message-length maximum client auto

message-length maximum 512

policy-map global_policy

description Netflow to VS-2

class inspection_default

inspect dns preset_dns_map

inspect ftp

inspect h323 h225

inspect h323 ras

inspect rsh

inspect rtsp

inspect esmtp

inspect sqlnet

inspect skinny

inspect sunrpc

inspect xdmcp

inspect sip

inspect netbios

inspect tftp

inspect ip-options

inspect pptp

class global-class

inspect dns

inspect ftp

inspect http

inspect icmp

inspect icmp error

inspect ip-options

inspect mgcp

inspect netbios

inspect pptp

inspect rsh

inspect rtsp

inspect sip

inspect snmp

inspect tftp

class global-class1

flow-export event-type all destination 192.168.10.224

policy-map type inspect im IM_Inspection

parameters

!

service-policy global_policy global

smtp-server 192.168.10.213

prompt hostname context

call-home

profile CiscoTAC-1

no active

destination address http https://tools.cisco.com/its/service/oddce/services/DDCEService

destination address email callhome@cisco.com

destination transport-method http

subscribe-to-alert-group diagnostic

subscribe-to-alert-group environment

subscribe-to-alert-group inventory periodic monthly

subscribe-to-alert-group configuration periodic monthly

subscribe-to-alert-group telemetry periodic daily

hpm topN enable

Cryptochecksum:51747c5d5bacf63fc394658b9f4ccccf

: end

varrao · ‎09-08-2011

Hi Mitch,

could you quickly give this a shot and let me know:

object network public_ip

host 1.1.1.1 ------------------> public ip of server

object network private_ip

host 192.168.50.xx -----------------> private ip of server

nat (LAN,LAN) source dynamic any interface destination static public_ip private_ip

and also add:

same-security-traffic permit intra-interface

Let me know how it goes.

Thanks,

Varun

Thanks,
Varun Rao

mitchell helton · ‎09-08-2011

Thanks again!

This is in a production environment and currently EVERYONE but the IT subnet is working as they should.

Will this have any impact on anyone else?

varrao · ‎09-08-2011

Hi Mitch,

The config would not hamper any other traffic, I am not really sure how the IT subnet is working fine, since I do not see any nat statement for it on the ASA, is the IT subnet behind the LAN interface only? I would suggest you first test whether the packets hits the ASA LAN interface, when you access it from IT subnet, just to make sure that all the routing is done by the firewall. You can test it by using the captures:

https://supportforums.cisco.com/docs/DOC-1222

Thanks,

Varun

Thanks,
Varun Rao

mitchell helton · ‎09-08-2011

I've had some problems come up... finishing them up then I will test this and post back. Thank you so much for all your time and help.

The IT subnet is behind the LAN interface only... all internal networks are hitting the 192.168.15.2 interface on the ASA when they are being routed externally I believe.

mitchell helton · ‎09-08-2011

I issued the commands that you suggested but it didn't fix the problem. Going to run the capture here in just a few minutes. Thanks

mitchell helton · ‎09-08-2011

Ok... I ran the capture and when I do show capture in-cap and out-cap, I receive 0 packet capture, 0 packet shown as a response. I believe I'm typing everything in correctly, but like I stated in my original post, I'm a novice when it comes to the ASA. Thanks.

varrao · ‎09-08-2011

Could you post the config that you used for packet-capture??

-Varun

Thanks,
Varun Rao

mitchell helton · ‎09-08-2011

access-list cap-list permit tcp host 192.168.150.97 host 192.168.10.59 eq 80 
access-list cap-list permit tcp host 192.168.10.59 eq 80 host 192.168.150.97

capture in-cap interface lan access-list cap-list buffer 1000000 packet 1522  
capture out-cap interface wan access-list cap-list buffer 1000000 packet 1522

varrao · ‎09-08-2011

You can open the access-list a bit and then try again:

access-list cap-list permit tcp host 192.168.150.97 host 192.168.10.59 
access-list cap-list permit tcp host 192.168.10.59 host 192.168.150.97

Try again and let me know if there are any packets onto the firewall.

Thanks,
Varun

Thanks,
Varun Rao

mitchell helton · ‎09-08-2011

I'm still getting the same response. Have a mistyped something somewhere? I even tried pinging the server to see if other traffic would show up since we removed the eq 80 command. Or does this mean it's not an issue with the ASA?

Thanks again sir.

varrao · ‎09-08-2011

Yes, the traffic isnt even reaching the firewall so we might need to troubleshoot why? We might need to check the routing and trace the packets on the switches.

-Varun

Thanks,
Varun Rao

mitchell helton · ‎09-08-2011

What is the next step you recommend?

I am able to access other servers on that subnet--even those that have static nat public IPs as well... our Spiceworks server for example.

If I do a traceroute to the server I'm having problems with, the packet goes I suspect.

It resolves the name of the server, hits the IT subnet gateway on the 3750 (192.168.150.1) then hits the server at 192.168.10.59

I'm really clueless. I am quite certain the switches and routers are functioning properly and was almost sure it was a problem with the ASA. Hoep you can help