Hi,
I have been having a problem with Linux clients that use the DNS server on a 1921 router, and was wondering if someone might have some ideas on what the problem below, might be. Worth mentioning, this problem doesn't seem to happen on a similarly configured 2901 router.
In terms of setup, the 1921 has ZBFW, provides DHCP, has local DNS host entries, also provides DNS forwarding, and is an HSRP pair with another 1921. Turning ZBFW off on the interfaces had no effect, and the test 2901 had no firewall configured at all. DNS queries on the router itself are fine, and even tried multiple CLI dig/nslookup calls from any Linux host, regardless of of speed, are fine; it's only DNS lookups via the OS call that fail when grouped together (in a Wireshark/DNS tx ID sense).
The problem is described in the following URLs:
http://philippecloutier.com/blogpost28-dig-1-and-other-DNS-clients-sometimes-taking-5-seconds-to-return-the-results-of-a-local-query
https://udrepper.livejournal.com/20948.html
https://www.math.tamu.edu/~comech/tools/linux-slow-dns-lookup/
The single-request option works, but can be tricky to shoehorn into different distributions. I do know that FreeBSD doesn't require the "option single-request" patch, but it is less than ideal to have to go around editing a bunch of Linux hosts to work around this problem. It should also be mentioned that 8.8.8.8/8.8.4.4/OpenDNS etc, all work perfect with the clients.
On a more technical note, it seems like the Wireshark (see attachment) is showing us that the query asks for the A and AAAA records, but randomly only gets a reply for one of them. Then the host delays the 5 seconds, and then asks for both the A and AAAA again to the same (or next, if multiple servers) DNS server, except this time it gets a response. When the entry is cached on the router, the problem goes away, and is thus only new lookups.
One thing that I noticed was that the queries all used the same src/dst ports, but different transaction IDs. It's not clear that this is related, but possibly someone here knows.
Anyway, that is the gist of the problem. I would love to know of a way to debug the DNS, but the results of a debug on a semi busy DNS are problematic since I fill my scroll back buffer in less than a second. Mostly, I am trying to figure out what happens to the initial request that is apparently being either ignored or dropped. Any suggestions on how to tackle this, are welcome!
