topic Hi, in Network Security

ASA 5525X dropping all DNS queries

Hermanus Janse van Vuuren — Tue, 12 Mar 2019 09:13:45 GMT

Hi All,

I have a problem since Friday and no shared support has not been renewed so I cannot open a TAC case.

We have found that the ASA 5525X on ver 9.4.3(8) just started to deny all DNS requests.

We have an ACL that allows the Internal DNS servers to communicated with the internet on port 53 on both udp/tcp.

After I have rebooted the Firewall everything was working 100% for between 20 - 40 min then just suddenly all UDP DNS queries gets dropped from the internal DNS servers to the external DNS servers. You reboot the FW and everything works again for 20 - 40 min.

I have tried ver 9.4.4 and ver 9.7.1 and even downgraded to ver 8.6.1 all giving exactly the same problem, after the reboot all DNS queries is allowed and then suddenly everything is denied.

It is as if the FW skips all the ACL's created and denies the request on the global access ACL.

What is even strangers is I have an Active/Standby setup, switch the 2 FW's around and the same happens, only after a reboot the DNS queries hits the correct ACL but only for 20 - 40 minutes. I have switched off the 1 FW now and is running on only 1 firewall.

Any suggestions would be welcome on where or what to look at.

No environmental changes was made only a few Office365 FQDN's that was added before all of this started.

Thank you in advance.

You should collect logs from

ajay chauhan — Mon, 17 Apr 2017 03:49:26 GMT

You should collect logs from ASA to debug what exactly is going on . Is that just DNS does not work or other services or also getting stopped ?

Number of connection ,syslog ,cpu ...all these things will help to debug ?where is you DNS server located even packettracer will help .

Ajay

+1. What does the log say as

Philip D'Ath — Mon, 17 Apr 2017 03:56:08 GMT

+1. What does the log say as the reason for the traffic being blocked?

Hi,

Hermanus Janse van Vuuren — Mon, 17 Apr 2017 06:02:01 GMT

Hi,

I have done a show tech support report on the asa but cannot see myself any thing odd on the firewall.

Yes all DNS queries gets dropped so we do not have any internet access out making use of the proxy server as it comes back saying DNS cannot be resolved, how ever if you bypass the proxy and open your IP on the FW for direct access and make use of an external DNS like 8.8.4.4 you are able to browse the internet.

My problem here is the internal DNS that sits on the internal network is being blocked accessing the external DNS for queries like for example Internal DNS is querying 8.8.8.8 but gets denied.

Deny udp src Inside_interface:x.x.x.x/53432 dst ISP_Interface 8.8.8.8/53 by access-group "global_access"

This is one example I get similar denies for the same interface with different acces-groups.

My CPU is running on 76% and my Memory usage is 3.9GB on the device.

This seems to be issue with

ajay chauhan — Mon, 17 Apr 2017 06:14:07 GMT

This seems to be issue with ACL's . Do you have any specific ACL applied on interface where your DNS reside and UDP 53 is allowed ?

Global access list applies logically to the entire firewall in inbound direction to all interface.

You may paste device configuration here.

Ajay

Hi Ajay,

Hermanus Janse van Vuuren — Mon, 17 Apr 2017 07:31:11 GMT

Hi Ajay,

Yes I have an ACL explicit for my DNS servers to access the internet on UDP/TCP port53. As I said this works well after a reboot for about 20 - 40 minutes then it skips all ACL's and go straight down to the global_access ACL.

I have now allowed my DNS servers on the global_Access ACL to query DNS on port 53 TCP/UDP and problem has been resolved.

Not the ideal solution but at-least we can browse now again and send email.

I suspect some changes was made the day before this started adding a new VPN group, DMZ for them and some ACL's but I cannot see how this would stop the Firewall from reading the spesific interface ACL's and just go straight down to the Global one.

Thank you Sir for actually

Hermanus Janse van Vuuren — Mon, 17 Apr 2017 07:32:01 GMT

Thank you Sir for actually pointing me in the right direction with your question here, I never looked at the end of the deny output.

I just found something

Philip D'Ath — Mon, 17 Apr 2017 08:03:10 GMT

I just found something interesting. There is an ASA DNS memory leak bug in the version of code you are running.

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvd71473

It is not showing any fixed releases, but I would upgrade to asa963-1-smp-k8.bin to try and help.

I have upgraded to the

Hermanus Janse van Vuuren — Mon, 17 Apr 2017 12:11:58 GMT

I have upgraded to the version you have mentioned below 9.6.3 and it seems to have resolved my issue. Just a point of note it would seem as if ver 9.7.1 also has the memory leak then as this did not resolve the issue when I ran it on that version.

Thank you very much for the kind advice.