08-21-2013 07:15 AM - edited 03-11-2019 07:28 PM
As part of an office move we upgraded our ASA to 9.1(2) and have been having what seem to be NAT problems with some services ever since. These problems manifest themselves with return traffic. For example, network time sync (NTP, port 123) works fine from the ASA, but hosts on the inside network cannot access external NTP servers (ntpq -pe shows all servers stuck in .INIT. status), creating problems with drifting clocks. Services like XBox Live also do not work; the XBox device can contact the internet, but return traffic from the service never gets back to the device.
For NTP specifically, I've tried allowing NTP 123 through the firewall, but it doesn't help. Conceptually, this should not be required since an inside host is initiating the connection and the NAT rules "should" allow the return packets. To further muddy the waters around NTP, a Linux VM CAN get NTP if it's network adapter is in NAT mode (so it's NAT'ing through the host workstation, then through the Cisco) but CAN NOT get NTP if the adapter is running in bridged mode (so the VM is talking directly to the ASA as if it were just another machine on the inside network).
I've stripped down the ASA config to the basics level, but still can't get this resolved. The main symptom of the problem is that if I disable the access-list rules around ICMP, I'll see lots of ICMP warnings in the ASA logs, which seems to indicate that there are traffic problems communiating with the inside hosts. I've narrowed the problem down to the ASA since replacing the device with a simple Netgear consumer-grade "firewall" lets all this traffic flow just fine.
Network is extremely basic:
DHCP ASSIGNED IP from ISP <----------> ASA <-----------------> inside (192.168.50.X)
^
|----------------------- guest vlan (10.0.1.X)
show running-config:
Result of the command: "show running-config"
: Saved
:
ASA Version 9.1(2)
!
hostname border
domain-name mydomain.com
enable password aaa encrypted
passwd bbb encrypted
names
!
interface Ethernet0/0
switchport access vlan 2
!
interface Ethernet0/1
switchport trunk allowed vlan 1,3
switchport trunk native vlan 1
switchport mode trunk
!
interface Ethernet0/2
!
interface Ethernet0/3
!
interface Ethernet0/4
!
interface Ethernet0/5
!
interface Ethernet0/6
!
interface Ethernet0/7
!
interface Vlan1
nameif inside
security-level 100
ip address 192.168.50.1 255.255.255.0
!
interface Vlan2
nameif outside
security-level 0
ip address dhcp setroute
!
interface Vlan3
nameif Guest-VLAN
security-level 10
ip address 10.0.1.1 255.255.255.0
!
boot system disk0:/asa912-k8.bin
boot system disk0:/asa911-k8.bin
boot system disk0:/asa831-k8.bin
ftp mode passive
clock timezone EST -5
clock summer-time EDT recurring
dns domain-lookup outside
dns server-group DefaultDNS
name-server 208.104.2.36
domain-name domain
same-security-traffic permit inter-interface
object network obj_any
subnet 0.0.0.0 255.255.255.0
object network Guest-WLAN
subnet 0.0.0.0 255.255.255.0
description Interent access for guest Wireless
object network xbox-nat-tcp3074
host 192.168.50.54
object network xbox-nat-udp3074
host 192.168.50.54
object network xbox-nat-udp88
host 192.168.50.54
object service xbox-live-88
service udp destination eq 88
object network xbox
host 192.168.50.54
object network obj-inside
subnet 192.168.50.0 255.255.255.0
object network obj-xbox
host 192.168.50.54
object network plex-server
host 192.168.50.5
object network ubuntu-server
host 192.168.50.5
description Ubuntu Linux Server
object network ntp
host 192.168.50.5
object network plex
host 192.168.50.5
object network INTERNET
subnet 0.0.0.0 0.0.0.0
object-group service xbox-live-3074 tcp-udp
port-object eq 3074
object-group protocol TCPUDP
protocol-object udp
protocol-object tcp
object-group service plex-server-32400 tcp
description Plex Media Server
port-object eq 32400
access-list outside_access_in extended permit object-group TCPUDP any object xbox object-group xbox-live-3074 log alerts
access-list outside_access_in extended permit object xbox-live-88 any object xbox log alerts
access-list outside_access_in extended permit tcp any any eq echo
access-list outside_access_in remark Plex Live access
access-list outside_access_in extended permit tcp any object plex-server object-group plex-server-32400
access-list outside_access_in extended permit icmp any any time-exceeded
access-list outside_access_in extended permit icmp any any unreachable
access-list outside_access_in extended permit icmp any any echo-reply
pager lines 24
logging enable
logging asdm informational
mtu inside 1500
mtu outside 1500
mtu Guest-VLAN 1500
no failover
icmp unreachable rate-limit 1 burst-size 1
asdm image disk0:/asdm-713.bin
no asdm history enable
arp timeout 14400
no arp permit-nonconnected
!
object network xbox-nat-tcp3074
nat (inside,outside) static interface service tcp 3074 3074
object network xbox-nat-udp3074
nat (inside,outside) static interface service udp 3074 3074
object network xbox-nat-udp88
nat (inside,outside) static interface service udp 88 88
object network plex
nat (inside,outside) static interface service tcp 32400 32400
object network INTERNET
nat (inside,outside) dynamic interface
!
nat (Guest-VLAN,outside) after-auto source dynamic any interface
access-group outside_access_in in interface outside
timeout xlate 3:00:00
timeout pat-xlate 0:00:30
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute
timeout tcp-proxy-reassembly 0:01:00
timeout floating-conn 0:00:00
dynamic-access-policy-record DfltAccessPolicy
no user-identity enable
user-identity default-domain LOCAL
aaa authentication ssh console LOCAL
aaa authentication telnet console LOCAL
http server enable
http 192.168.50.0 255.255.255.0 inside
no snmp-server location
no snmp-server contact
snmp-server enable traps snmp authentication linkup linkdown coldstart warmstart
crypto ipsec security-association pmtu-aging infinite
crypto ca trustpoint _SmartCallHome_ServerCA
crl configure
crypto ca trustpoint ASDM_TrustPoint0
enrollment self
subject-name CN=border
crl configure
crypto ca trustpool policy
crypto ca certificate chain _SmartCallHome_ServerCA
certificate ca xxxx
quit
crypto ca certificate chain ASDM_TrustPoint0
certificate xxxx
quit
crypto ikev2 remote-access trustpoint ASDM_TrustPoint0
telnet timeout 5
ssh 192.168.50.0 255.255.255.0 inside
ssh timeout 5
ssh key-exchange group dh-group1-sha1
console timeout 0
vpn-addr-assign local reuse-delay 60
dhcp-client client-id interface outside
dhcpd auto_config outside
!
dhcpd address 192.168.50.5-192.168.50.132 inside
!
dhcpd address 10.0.1.50-10.0.1.100 Guest-VLAN
dhcpd dns 208.104.244.45 208.104.2.36 interface Guest-VLAN
dhcpd lease 86400 interface Guest-VLAN
dhcpd enable Guest-VLAN
!
threat-detection basic-threat
threat-detection scanning-threat shun except ip-address 192.168.50.0 255.255.255.0
threat-detection statistics
threat-detection statistics tcp-intercept rate-interval 30 burst-rate 400 average-rate 200
ntp server 152.19.240.5 source outside prefer
ssl trust-point ASDM_TrustPoint0 outside
username xxx password xxx/ encrypted privilege 15
!
class-map inspection_default
match default-inspection-traffic
!
!
policy-map type inspect dns preset_dns_map
parameters
message-length maximum client auto
message-length maximum 512
policy-map global_policy
class inspection_default
inspect icmp
!
service-policy global_policy global
prompt hostname context
service call-home
call-home reporting anonymous
call-home
contact-email-addr me@here.net
profile CiscoTAC-1
destination address http https://tools.cisco.com/its/service/oddce/services/DDCEService
destination address email callhome@cisco.com
destination transport-method http
subscribe-to-alert-group diagnostic
subscribe-to-alert-group environment
subscribe-to-alert-group inventory periodic monthly
subscribe-to-alert-group configuration periodic monthly
subscribe-to-alert-group telemetry periodic daily
hpm topN enable
Cryptochecksum:xxx
: end
08-21-2013 07:51 AM
Hi,
Configuration seems fine.
With regards to the ICMP, you could also add this
class inspection_default
inspect icmp error
I would probably start by trying out some other software level on the ASA
Maybe some 8.4(x) software or 9.0(x) software. See if it some bug perhaps.
One option is ofcourse to capture traffic directly on the ASA or on the hosts behind the ASA. And go through the information with Wireshark.
- Jouni
08-21-2013 09:26 AM
Thanks, Jouni,
I added the inspect icmp error line with no change to my NTP problem. If you have any other ideas, I'm all ears. Next up I'll try a software downgrade.
08-21-2013 07:16 PM
I've FINALLY found the culprit, and it was an HP 1810G switch of all things.
I was working with a Cisco engineer and we did a packet capture, and nothing was hitting the router for NTP or Xbox live. On a lark I checked out the switch settings, and the 1810G has a "Auto DoS" setting on one of the security tabs. This setting apparently kills off UDP packets, and was preventing Xbox LIve and NTP from working.
I've spent MONTHS on this crazy problem...
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide