ASA dns inspect & malformed packet

blepeuple · ‎12-01-2011

Hi,

I have a network protected by a pair of ASA 5520, running 8.04.

I do pretty much default dns inspection:

policy-map type inspect dns preset_dns_map

parameters

message-length maximum 1024

policy-map global_policy

class inspection_default

...

inspect dns preset_dns_map

The problem I have is the following: the ASA is silently dropping some malformed packets which is causing some dns queries to fail,because instead of falling back to TCP, they just timeout.

For instance:

with dns inspection turned on:

[blaise@fr-th-ax01 ~]# dig elysee.blog.lemonde.fr +trace

; <<>> DiG 9.3.6-P1-RedHat-9.3.6-4.P1.el5_5.3 <<>> elysee.blog.lemonde.fr +trace

;; global options: printcmd

. 3600 IN NS i.root-servers.net.

. 3600 IN NS h.root-servers.net.

. 3600 IN NS g.root-servers.net.

. 3600 IN NS f.root-servers.net.

. 3600 IN NS e.root-servers.net.

. 3600 IN NS d.root-servers.net.

. 3600 IN NS c.root-servers.net.

. 3600 IN NS b.root-servers.net.

. 3600 IN NS a.root-servers.net.

. 3600 IN NS m.root-servers.net.

. 3600 IN NS l.root-servers.net.

. 3600 IN NS k.root-servers.net.

. 3600 IN NS j.root-servers.net.

;; Received 497 bytes from 172.16.1.20#53(172.16.1.20) in 1 ms

fr. 172800 IN NS d.nic.fr.

fr. 172800 IN NS e.ext.nic.fr.

fr. 172800 IN NS c.nic.fr.

fr. 172800 IN NS g.ext.nic.fr.

fr. 172800 IN NS d.ext.nic.fr.

fr. 172800 IN NS f.ext.nic.fr.

;; Received 408 bytes from 192.36.148.17#53(i.root-servers.net) in 9 ms

lemonde.fr. 172800 IN NS indom30.indomco.fr.

lemonde.fr. 172800 IN NS indom10.indomco.com.

lemonde.fr. 172800 IN NS indom80.indomco.hk.

lemonde.fr. 172800 IN NS indom130.indomco.org.

lemonde.fr. 172800 IN NS indom20.indomco.net.

;; Received 218 bytes from 194.0.9.1#53(d.nic.fr) in 2 ms

;; connection timed out; no servers could be reached

without dns inspect:

[blaise@fr-th-ax01 ~]# dig elysee.blog.lemonde.fr +trace

; <<>> DiG 9.3.6-P1-RedHat-9.3.6-4.P1.el5_5.3 <<>> elysee.blog.lemonde.fr +trace

;; global options: printcmd

. 3600 IN NS l.root-servers.net.

. 3600 IN NS k.root-servers.net.

. 3600 IN NS j.root-servers.net.

. 3600 IN NS i.root-servers.net.

. 3600 IN NS h.root-servers.net.

. 3600 IN NS g.root-servers.net.

. 3600 IN NS f.root-servers.net.

. 3600 IN NS e.root-servers.net.

. 3600 IN NS d.root-servers.net.

. 3600 IN NS c.root-servers.net.

. 3600 IN NS b.root-servers.net.

. 3600 IN NS a.root-servers.net.

. 3600 IN NS m.root-servers.net.

;; Received 501 bytes from 172.16.1.20#53(172.16.1.20) in 1 ms

fr. 172800 IN NS c.nic.fr.

fr. 172800 IN NS d.ext.nic.fr.

fr. 172800 IN NS d.nic.fr.

fr. 172800 IN NS e.ext.nic.fr.

fr. 172800 IN NS f.ext.nic.fr.

fr. 172800 IN NS g.ext.nic.fr.

;; Received 408 bytes from 199.7.83.42#53(l.root-servers.net) in 4 ms

lemonde.fr. 172800 IN NS indom10.indomco.com.

lemonde.fr. 172800 IN NS indom20.indomco.net.

lemonde.fr. 172800 IN NS indom30.indomco.fr.

lemonde.fr. 172800 IN NS indom80.indomco.hk.

lemonde.fr. 172800 IN NS indom130.indomco.org.

;; Received 218 bytes from 192.134.0.129#53(c.nic.fr) in 1 ms

;; Warning: Message parser reports malformed message packet.

;; Truncated, retrying in TCP mode.

elysee.blog.lemonde.fr. 10800 IN CNAME blog.lemonde.fr.2-01-271d-000c.cdx.cedexis.net.

. 518400 IN NS a.root-servers.net.

. 518400 IN NS b.root-servers.net.

. 518400 IN NS c.root-servers.net.

. 518400 IN NS d.root-servers.net.

. 518400 IN NS e.root-servers.net.

. 518400 IN NS f.root-servers.net.

. 518400 IN NS g.root-servers.net.

. 518400 IN NS h.root-servers.net.

. 518400 IN NS i.root-servers.net.

. 518400 IN NS j.root-servers.net.

. 518400 IN NS k.root-servers.net.

. 518400 IN NS l.root-servers.net.

. 518400 IN NS m.root-servers.net.

;; Received 516 bytes from 217.174.200.97#53(indom10.indomco.com) in 5 ms

So is there a solution wher the ASA would send _something_ back as an answer instead of just silently dropping the packet, or am I stuck with either disabling dns inspection (maybe just for this domain) or trying to call lemonde.fr network admin to ask him/her to fix their dns server, which is not going to happen.

Thanks

Blaise

Julio Carvajal · ‎12-01-2011

Hello Blepeuple,

The ASA is going to drop the packets as they are not respecting the RFC, now as you suggested that would be the solution to disable dns inspection for that particular domain.

Regards,

Julio

Julio Carvajal
Senior Network Security and Core Specialist
CCIE #42930, 2xCCNP, JNCIP-SEC

blepeuple · ‎12-01-2011

Thanks.

When I wrote my question, I thought I could disable dns inspection for that domain, but I can't quite figure out how.

The only thing that works is to completely turn off dns inspection (not just have a different policy with protocol enforcement), and since I'm guessing I need the inspection in order to understand the protocol (needed to turn it off for a given domain), I think I'm stuck with an all or nothing situation.

As a side note, the exemple I gave is from one of the biggest french newspaper. I can't believe they inadvertantly block traffic to all users protected by a cisco appliance. I feel I am missing something.

Blaise

Julio Carvajal · ‎12-01-2011

Hello Blaise,

hmm this is complicated ,Can you try the following for me and let me know the result.

Lets say the domain name is test.test right

So lets do the following

-regex example \test\.test

-policy-map type inspect dns preset_dns_map

-match not domain-name regex example

With this we are going to not inspect that particular domain-name, I am not sure if its going to work but lets give it a try

Please rate helpful posts.

Regards,

Julio

Julio Carvajal
Senior Network Security and Core Specialist
CCIE #42930, 2xCCNP, JNCIP-SEC

blepeuple · ‎12-02-2011

Tanks for your suggestion.

Unfortunately, it doesn't work.

I believe the match clause inside the policy map is to specify other kind of actions (drop / log / etc...)

And the match not domain-name regex example will correctly match anything but test.test

however, the test.test packets will still be inspected and dropped.

Blaise

manish arora · ‎12-02-2011

Try this :-

policy-map type inspect dns preset_dns_map

parameters

message-length maximum 1024

no dns-guard

Manish

blepeuple · ‎12-04-2011

Thanks,

but unfortunately again, no luck : it doesn't change a thing.

I already tried "no dns-guard', and "no protocol-enforcement", but the packets are still getting dropped.

Blaise

Julio Carvajal · ‎12-04-2011

Hello Blaise,

Yes, I did not expect the DNS-guard to solve the problem because that feature what does is to close the UDP session as soon as the DNS reply is received, so in our case the problem is that the DNS packet is violating the DNS protocol standar.

I would have used the -no protocol-enforcement but you already try it with no success, so lets do the following and see what happens:

policy-map type inspect dns preset_dns_map
parameters
message-length maximum client auto

I will do some research and let you know if I have found something different that could help us on this.

Regards.

Julio

Julio Carvajal
Senior Network Security and Core Specialist
CCIE #42930, 2xCCNP, JNCIP-SEC

blepeuple · ‎12-05-2011

After a little bit of diggin, it turns out the dns response actually perfectly respects the rfc.

If a packet is bigger than 512 (and the client didn't specify that it supported a larger than 512 payload), the server is supposed to return a truncated packet with the TC flag (truncated) set, so that the client retries with TCP.

So why is the ASA dropping that rfc-conforming packet ?

Should I open a ticket with Cisco ?

manish arora · ‎12-05-2011

Hi Blaise / Julio,

The reason I sent no dns-guard was that I had the same setting on of Firewall and I was able resolve that domain ( using dig +trace) with dns inspection still on.

Anyways, what I missed in that was that you are running Version 8.0 and I am running 8.2(4). Then I tested on one of my 8.0 firewall and It didn't work.

So, most likely its a bug but I can't find it right now in the bug toolkit. if possible please upgrade to version 8.2 or search for the bug and see if they have any work around to this without upgrade.

Manish

Julio Carvajal · ‎12-05-2011

Hello Manish,

Great to know that, thank you very much for the info, so yes that it, seems like Blaise would need to perform an upgrade in order to solve this,

Thanks again Manish

Julio Carvajal
Senior Network Security and Core Specialist
CCIE #42930, 2xCCNP, JNCIP-SEC

blepeuple · ‎12-05-2011

Thank you both for your help.

I'll try an upgrade to 8.2 (which will take a little while, as I have no smartnet yet for this device) and report the results here.

Blaise

eddie.harmoush · ‎12-05-2011

policy-map type inspect dns preset_dns_map

parameters

message-length maximum 1024

Problem might be here. DNS used to use UDP for all queries except when the size of the response was over 512 bytes. Until RFC 2671 came about, which increased this limit to 4096 bytes before DNS must fall back to TCP.

I can't say for sure that this is the problem you are receiving, but you can fix this in one of two ways:

(1) Increase maximum length to 4096:

policy-map type inspect dns preset_dns_map

parameters

message-length maximum 4096

(2) Let the ASA dynamically determine the size value specified by the requester in the OPT pseudo-RR

policy-map type inspect dns preset_dns_map

parameters

message-length maximum client auto

message-length maximum 512

The command message-length maximum client auto is by default enabled in 8.2 and beyond. Which explains why it worked for the user testing on 8.2.

Hope this helps.

manish arora · ‎12-05-2011

Doesn't work with 4096 on 8.0(4) !!!!!!

does work 8.2 with :-

message-length maximum 1024

Manish