I have a problem where DNS responses over IPv4 are getting dropped between my firewall and switch and are not getting back to the host. This is resulting in failed DNS lookups. Below is my network layout:
The DNS standard query responses are getting dropped somewhere between the ASA's inside interface and the Catalyst's main uplink interface (Port-channel 1). This is only happening on IPv4. I have the 10.2.28.0/22 network NAT'ed on the ASA. I'm not sure if this is a routing, NAT, packet inspection, or ACL issue for the ASA or an ACL issue for the Catalyst. Details are below:
- From the DNS server you can see that DNS resolution to an IPv4 only NS server (ns1.canonical.com [91.189.94.173] is IPv4 only) does not resolve:
root@server1:~# ip --brief -4 addr show veth-mgmt veth-mgmt@if273 UP 10.2.28.2/22 root@server1:~# dig A archive.ubuntu.com @91.189.94.173 ; <<>> DiG 9.11.3-1ubuntu1.12-Ubuntu <<>> A archive.ubuntu.com @91.189.94.173 ;; global options: +cmd ;; connection timed out; no servers could be reached
- I can however successfully ping the NS server and can even telnet to it over port 53:
root@server1:~# ping -c 4 91.189.94.173 PING 91.189.94.173 (91.189.94.173) 56(84) bytes of data. 64 bytes from 91.189.94.173: icmp_seq=1 ttl=53 time=120 ms 64 bytes from 91.189.94.173: icmp_seq=2 ttl=53 time=120 ms 64 bytes from 91.189.94.173: icmp_seq=3 ttl=53 time=120 ms 64 bytes from 91.189.94.173: icmp_seq=4 ttl=53 time=120 ms --- 91.189.94.173 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3002ms rtt min/avg/max/mdev = 120.767/120.826/120.957/0.355 ms root@server1:~# telnet 91.189.94.173 53 Trying 91.189.94.173... Connected to 91.189.94.173. Escape character is '^]'. ^] telnet> quit Connection closed.
- When I capture the packets on the ASA I can see the DNS queries and the DNS responses come back:
ASA# capture asadnscap interface inside match udp host 10.2.28.2 host 91.189.94.173 ASA# show capture asadnscap 6 packets captured 1: 16:58:50.489552 10.2.28.2.56438 > 91.189.94.173.53: udp 59 2: 16:58:50.610701 91.189.94.173.53 > 10.2.28.2.56438: udp 191 3: 16:58:55.489736 10.2.28.2.56438 > 91.189.94.173.53: udp 59 4: 16:58:55.610808 91.189.94.173.53 > 10.2.28.2.56438: udp 191 5: 16:59:00.490132 10.2.28.2.56438 > 91.189.94.173.53: udp 59 6: 16:59:00.611098 91.189.94.173.53 > 10.2.28.2.56438: udp 191 6 packets shown
- However when looking at the packets on the switch I only see the DNS queries:
SWITCH#monitor capture swdnscap buffer size 10 interface GigabitEthernet 1/1/1 both match ipv4 host 10.2.28.2 host 91.189.94.173 SWITCH#monitor capture swdnscap start SWITCH#monitor capture swdnscap stop SWITCH#show monitor capture swdnscap buffer brief 0.000000 10.2.28.2 -> 91.189.94.173 DNS Standard query A archive.ubuntu.com 5.005996 10.2.28.2 -> 91.189.94.173 DNS Standard query A archive.ubuntu.com 10.005996 10.2.28.2 -> 91.189.94.173 DNS Standard query A archive.ubuntu.com
If I had to guess it has something to do with packet inspection at the ASA level, but I'm not sure how to confirm this or what to do about it. Also, my switch actually uses a port channel and I don't have a recent enough version of IOS-XE to monitor PC's so I just picked one of the ethernet devices in the PC. Not sure if that would effect my packet capture. I ran the test multiple times to make sure I always saw the same results (which I did).
Finally I also tried to see if I could capture dropped packets on the ASA (i.e. capture asadropdns type asp-drop all). I exported the pcap and tried to find any packets from 91.189.94.173 that were dropped, but couldn't find any that specifically contained DNS responses (there were some administrative ICMP drops but their source was not from the DNS server).
Here is my ASA configuration:
interface Port-channel2 lacp max-bundle 8 nameif inside security-level 100 ip address 10.2.1.1 255.255.255.0 ipv6 address 2001:db8::1/64 ipv6 enable ! object network virt-mgmt subnet 10.2.28.0 255.255.252.0 access-list outside_ingress extended permit ip any 203.0.113.0 255.255.255.0 access-list outside_ingress extended permit icmp any 198.51.100.0 255.255.255.248 access-list outside_ingress extended permit tcp any host 203.0.113.84 eq imap4 access-list outside_ingress extended permit tcp any host 203.0.113.85 eq 587 access-list outside_ingress extended permit tcp any host 203.0.113.85 eq smtp access-list outside_ingress extended permit ip any 2001:db8:1::/48 access-list outside_ingress extended permit icmp6 any 2001:db8:2::1/64 access-list outside_ingress extended permit tcp any host 2001:db8:1:1::1 eq imap4 access-list outside_ingress extended permit tcp any host 2001:db8:1:1::2 eq 587 access-list outside_ingress extended permit tcp any host 2001:db8:1:1::3 eq smtp object network virt-mgmt nat (inside,outside) dynamic interface access-group outside_ingress in interface outside policy-map type inspect dns preset_dns_map parameters message-length maximum client auto message-length maximum 512 no tcp-inspection service-policy global_policy global policy-map global_policy class inspection_default inspect dns preset_dns_map inspect ftp inspect h323 h225 inspect h323 ras inspect rsh inspect rtsp inspect esmtp inspect sqlnet inspect skinny inspect sunrpc inspect xdmcp inspect sip inspect netbios inspect tftp inspect ip-options inspect icmp !
And my switch access list:
Extended IP access list harm_acl_ipv4
9 permit ip any 203.0.113.0 0.0.0.255 (205276184 matches)
10 permit tcp any any established
11 permit udp any 10.2.0.0 0.0.255.255 eq domain
15 permit icmp any 10.2.0.0 0.0.255.255 (102 matches)
60 permit tcp any host 203.0.113.84 eq 143
70 permit tcp any host 203.0.113.85 eq 587
80 permit tcp any host 203.0.113.85 eq smtp
90 permit ip 10.2.0.0 0.0.255.255 any (3 matches)
Accepted Solutions
