topic Re: Firewall 5506-x blocking all DNS queries in Network Security

Firewall 5506-x blocking all DNS queries

Zargham Haider — Tue, 12 Mar 2019 14:01:53 GMT

Hi all

I am observing my firewall 5506-x since a month that after 2 or 3 days, firewall suddenly stop resolving DNS queries and all internet traffic stop. while in this situation IP communication remain OK which means i can ping/browse across the firewall but if i will reboot firewall then it will start working normally.

any idea what is happening with firewall ?

regards

Re: Firewall 5506-x blocking all DNS queries

Martin Kling — Mon, 15 Oct 2018 07:38:44 GMT

What code do you use? ASA, ASA + Firepower or FTD? What version?

Re: Firewall 5506-x blocking all DNS queries

Zargham Haider — Mon, 15 Oct 2018 08:59:25 GMT

Hi Martin ...

Thanks for reply..

this is ASA 5506-x FirePOWER

# sh ver

Cisco Adaptive Security Appliance Software Version 9.6(2)23
Device Manager Version 7.6(1)

Compiled on Thu 28-Sep-17 07:50 PDT by builders
System image file is "disk0:/asa962-23-lfbff-k8.SPA"
Config file at boot was "startup-config"

ProjectFW up 3 days 23 hours

Hardware: ASA5506, 4096 MB RAM, CPU Atom C2000 series 1250 MHz, 1 CPU (4 cores)
Internal ATA Compact Flash, 7168MB
BIOS Flash N25P64 @ 0xfed01000, 16384KB

Encryption hardware device : Cisco ASA Crypto on-board accelerator (revision 0x1)
Number of accelerators: 1

1: Ext: GigabitEthernet1/1 : address is 2c5a.0f79.d225, irq 255
2: Ext: GigabitEthernet1/2 : address is 2c5a.0f79.d226, irq 255
3: Ext: GigabitEthernet1/3 : address is 2c5a.0f79.d227, irq 255
4: Ext: GigabitEthernet1/4 : address is 2c5a.0f79.d228, irq 255
5: Ext: GigabitEthernet1/5 : address is 2c5a.0f79.d229, irq 255
6: Ext: GigabitEthernet1/6 : address is 2c5a.0f79.d22a, irq 255
7: Ext: GigabitEthernet1/7 : address is 2c5a.0f79.d22b, irq 255
8: Ext: GigabitEthernet1/8 : address is 2c5a.0f79.d22c, irq 255

Re: Firewall 5506-x blocking all DNS queries

Martin Kling — Mon, 15 Oct 2018 09:04:45 GMT

Have configured Firepower or are you just using ASA? Do you get any logs when the issue occurs?

Re: Firewall 5506-x blocking all DNS queries

Zargham Haider — Mon, 15 Oct 2018 09:14:56 GMT

Hi Martin........

I am just using Firewall services not FirePOWER....unfortunately i didn't copy logs. (Just configured syslog today.. ) right now firewall is working fine. One thing i want to share is....I think due to some strange activity ASA skips all ACL rules for DNS query and all dns queries fall in global deny list. because that specific time i checked packet trace which was denied by global acl list. but i am not able to find the reason why it happening like this. need your expert judgement here.

Re: Firewall 5506-x blocking all DNS queries

Martin Kling — Mon, 15 Oct 2018 09:41:25 GMT

Can you mask you config and paste it here?

Re: Firewall 5506-x blocking all DNS queries

Zargham Haider — Mon, 15 Oct 2018 11:26:37 GMT

Ok Martin...here is config file info

: Saved

: Serial Number: XXXXXXXXXXX

: Hardware: ASA5506, 4096 MB RAM, CPU Atom C2000 series 1250 MHz, 1 CPU (4 cores)

ASA Version 9.6(2)23

hostname ProjectFW

domain-name abc.com

enable password 7sI3Z.cdfer2iY encrypted

passwd 7sI3Z.xcvfgt2iY encrypted

names

interface GigabitEthernet1/1

nameif outside

security-level 0

ip address x.x.x.x 255.255.255.0

interface GigabitEthernet1/2

nameif inside

security-level 100

ip address 192.168.81.7 255.255.255.0

dhcprelay server 192.168.81.25

interface GigabitEthernet1/3

shutdown

no nameif

no security-level

no ip address

interface GigabitEthernet1/4

nameif Winside

security-level 100

ip address 192.168.83.7 255.255.255.0

interface GigabitEthernet1/5

shutdown

no nameif

no security-level

no ip address

interface GigabitEthernet1/6

shutdown

nameif xxx_Outside

security-level 0

ip address x.x.x.x 255.255.255.252

interface GigabitEthernet1/7

shutdown

no nameif

no security-level

no ip address

interface GigabitEthernet1/8

shutdown

no nameif

no security-level

no ip address

interface Management1/1

management-only

no nameif

no security-level

no ip address

boot system disk0:/asa962-23-lfbff-k8.SPA

ftp mode passive

dns domain-lookup inside

dns server-group DefaultDNS

name-server 192.168.81.25 inside

domain-name abc.com

same-security-traffic permit inter-interface

same-security-traffic permit intra-interface

object network obj_any

subnet 0.0.0.0 0.0.0.0

object network Printer_192.168.81.31_Plotter

host 192.168.81.31

description Plotter

object network Printer_192.168.81.36_xxx

host 192.168.81.36

description xxxx

object network Printer_192.168.81.47_xxx

host 192.168.81.47

description xxx Printer

object network Printer_192.168.81.41_xxx

host 192.168.81.41

description xxxx

object network Printer_192.168.81.45_xxxx

host 192.168.81.45

description xxxx

object network Printer_192.168.81.48_xxxx

host 192.168.81.48

description xxxx

object network Printer_192.168.81.42_xxxx

host 192.168.81.42

description xxxx

object network Printer_192.168.81.43_xxxx

host 192.168.81.43

description xxxx

object-group service Syslog udp

port-object eq syslog

access-list inside_access_in extended permit udp object-group DM_INLINE_NETWORK_2 any object-group DM_INLINE_UDP_1 log disable

access-list inside_access_in extended permit object-group DM_INLINE_SERVICE_1 192.168.81.0 255.255.255.0 object-group ApprovedDNSServer

access-list inside_access_in extended permit tcp 192.168.81.0 255.255.255.0 any object-group DM_INLINE_TCP_16 log disable

access-list inside_access_in extended permit tcp 192.168.81.0 255.255.255.0 object sftp.norc.org eq ssh

access-list inside_access_in extended permit object-group TCP-UDP 192.168.81.0 255.255.255.0 any object-group DM_INLINE_TCPUDP_1

access-list inside_access_in extended permit ip object Server_xxx any

access-list inside_access_in extended permit ip object-group NoRestrictionSource_aaa any log disable

access-list inside_access_in extended permit ip object-group NoRestrictionSource_IT_Team any log disable

access-list inside_access_in extended permit ip object-group NoRestrictionSource_aaaa any log disable

access-list inside_access_in extended permit ip object-group Server_Live any

access-list inside_access_in extended deny ip object-group Restricted_IPs any log disable

access-list inside_access_in extended permit ip object-group NoRestrictionSources any

access-list inside_access_in extended permit tcp 192.168.81.0 255.255.255.0 any object-group DM_INLINE_TCP_14

access-list inside_access_in extended permit tcp 192.168.81.0 255.255.255.0 any object-group DM_INLINE_TCP_8

access-list inside_access_in extended permit tcp 192.168.81.0 255.255.255.0 any object-group ApplePushNotificationService

access-list inside_access_in extended permit tcp any object-group Printers object-group Printer_TCP

access-list inside_access_in extended permit tcp 192.168.81.0 255.255.255.0 any object-group DM_INLINE_TCP_15

access-list inside_access_in extended permit ip 192.168.81.0 255.255.255.0 192.168.83.0 255.255.255.0 inactive

access-list inside_access_in extended permit icmp 192.168.81.0 255.255.255.0 any

access-list inside_access_in extended permit udp 192.168.81.0 255.255.255.0 any eq ntp

access-list inside_access_in extended permit tcp object Server_MigrationManager_81.199 any object-group DM_INLINE_TCP_18 inactive

access-list inside_access_in extended permit tcp 192.168.81.0 255.255.255.0 any object-group DM_INLINE_TCP_9

access-list inside_access_in extended permit tcp 192.168.81.0 255.255.255.0 any object-group DM_INLINE_TCP_5

access-list inside_access_in extended permit udp any any object-group WhatsApp_UDP

access-list inside_access_in extended permit object-group TCP-UDP any any object-group XMPP

access-list inside_access_in extended permit tcp 192.168.81.0 255.255.255.0 any object-group DM_INLINE_TCP_7 inactive

access-list inside_access_in extended permit ip 192.168.81.0 255.255.255.0 object-group GoodServers

access-list inside_access_in extended deny ip any object-group Blocked_Addresses

access-list inside_access_in extended deny object-group TCP-UDP any any object-group Torrent

access-list inside_access_in extended deny ip object-group Blocked_Sources any

access-list inside_access_in extended deny object-group TCP-UDP 192.168.81.0 255.255.255.0 any object-group HotspotShield

access-list inside_access_in extended deny ip any any

access-list Winside_access_in extended permit udp 192.168.83.0 255.255.255.0 any object-group DM_INLINE_UDP_2

access-list Winside_access_in extended permit object-group DM_INLINE_SERVICE_6 192.168.83.0 255.255.255.0 object Server_xxx

access-list Winside_access_in extended permit udp 192.168.83.0 255.255.255.0 object-group Printers object-group Printer_HP_Ports

access-list Winside_access_in extended permit tcp 192.168.83.0 255.255.255.0 192.168.81.0 255.255.255.0 object-group DM_INLINE_TCP_13

access-list Winside_access_in extended permit tcp object-group xxx object-group DM_INLINE_NETWORK_4 object-group xxx

access-list Winside_access_in extended permit tcp 192.168.83.0 255.255.255.0 object-group ApprovedDNSServer eq domain

access-list Winside_access_in extended permit ip object-group NoRestrictionSources any

access-list Winside_access_in extended permit tcp 192.168.83.0 255.255.255.0 object-group Printers object-group Printer_TCP

access-list Winside_access_in extended permit icmp any any

access-list Winside_access_in extended permit object-group DM_INLINE_SERVICE_5 any object Server_xxx inactive

access-list Winside_access_in extended permit object-group DM_INLINE_SERVICE_4 192.168.83.0 255.255.255.0 any inactive

access-list outside_access_in extended permit object-group ICMP any any

access-list outside_access_in extended permit tcp any interface outside object-group DM_INLINE_TCP_1

access-list outside_access_in extended permit tcp any object aaa_x.x.x.x_ object-group DM_INLINE_TCP_2

access-list outside_access_in extended permit tcp any object aab_x.x.x.x object-group DM_INLINE_TCP_3

access-list outside_access_in extended permit tcp any object aac_x.x.x.x object-group DM_INLINE_TCP_4

access-list inboundSurvey extended permit icmp any any object-group DM_INLINE_ICMP_1

access-list inboundSurvey extended permit tcp any object xxx_Interface_Outside object-group DM_INLINE_TCP_12

access-list OUTSIDE-IN extended permit icmp any any

access-list outside_access_Out extended permit tcp any4 object xxx object-group DM_INLINE_TCP_10

access-list outside_access_Out extended permit tcp any4 object Server_MigrationManager_81.199 object-group DM_INLINE_TCP_0

access-list outside_access_Out extended permit ip object-group xxx object xxx_81.29

access-list outside_access_Out extended permit object-group DM_INLINE_SERVICE_7 any4 object xxx

access-list outside_access_Out extended permit object-group DM_INLINE_SERVICE_0 any4 object xxx

access-list outside_access_Out extended permit icmp object-group DM_INLINE_NETWORK_3 any inactive

access-list outside_access_Out extended permit icmp any any

access-list outside_access_Out extended deny ip any any

access-list x_Outsite_access_in extended permit icmp any any

access-list xxx-THROTTLE extended permit ip object Server_xxx any

access-list xxxx-THROTTLE extended permit ip any object Server_xxx inactive

access-list xxx extended permit tcp 192.168.83.0 255.255.255.0 any object-group DM_INLINE_TCP_17

pager lines 24

logging enable

logging timestamp

logging trap critical

logging asdm informational

logging host inside 192.168.81.x

no logging message 106015

no logging message 313001

no logging message 313008

no logging message 106023

no logging message 710003

no logging message 106100

no logging message 302015

no logging message 302014

no logging message 302013

no logging message 302018

no logging message 302017

no logging message 302016

no logging message 302021

no logging message 302020

flow-export destination inside 192.168.81.17 9996

flow-export delay flow-create 15

mtu outside 1500

mtu inside 1500

mtu Winside 1500

mtu NTL_Outside 1500

icmp unreachable rate-limit 1 burst-size 1

no asdm history enable

arp timeout 14400

no arp permit-nonconnected

arp rate-limit 16384

nat (inside,outside) source dynamic any interface dns

nat (Winside,inside) source dynamic any interface dns

object network aaa

nat (inside,outside) static interface service tcp 3389 3389

object network aab

nat (inside,outside) static xxx

object network aac

nat (inside,outside) static xxx

object network aad

nat (inside,outside) static xxx

object network aae

nat (inside,outside) static xxx

access-group outside_access_Out in interface outside

access-group inside_access_in in interface inside

access-group Winside_access_in in interface Winside

access-group x Outsite_access_in in interface xxx Outside

route-map xxx permit 10

match ip address xxx

set ip next-hop x.x.x.x

route-map xxx permit 20

route outside 0.0.0.0 0.0.0.0 x.x.x.x 1

timeout xlate 3:00:00

timeout pat-xlate 0:00:30

timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 sctp 0:02:00 icmp 0:00:02

timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00

timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00

timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute

timeout tcp-proxy-reassembly 0:01:00

timeout floating-conn 0:00:00

timeout conn-holddown 0:00:15

user-identity default-domain LOCAL

aaa authentication ssh console LOCAL

http server enable

http 192.168.1.0 255.255.255.0 inside

http 192.168.81.0 255.255.255.0 inside

snmp-server host inside 192.168.81.x community *****

no snmp-server location

no snmp-server contact

snmp-server community *****

service sw-reset-button

crypto ipsec security-association pmtu-aging infinite

crypto ca trustpool policy

telnet 192.168.81.0 255.255.255.0 inside

telnet timeout 30

ssh stricthostkeycheck

ssh timeout 5

ssh key-exchange group dh-group1-sha1

console timeout 0

dhcpd domain abc.com

dhcpd address 192.168.83.30-192.168.83.245 Winside

dhcpd dns 192.168.81.25 interface Winside

dhcpd option 3 ip 192.168.83.253 interface Winside

dhcpd option 6 ip 192.168.81.25 interface Winside

dhcprelay server 192.168.81.25 inside

dhcprelay enable Winside

dhcprelay timeout 160

threat-detection basic-threat

threat-detection scanning-threat shun except object-group NoRestrictionSource_IT_Team

threat-detection scanning-threat shun except object-group NoRestrictionSource_xx

threat-detection scanning-threat shun except object-group NoRestrictionSources

threat-detection scanning-threat shun except object-group NoShunnGroup

threat-detection scanning-threat shun duration 1800

threat-detection statistics host number-of-rate 3

threat-detection statistics port number-of-rate 3

threat-detection statistics protocol number-of-rate 3

threat-detection statistics access-list

threat-detection statistics tcp-intercept rate-interval 30 burst-rate 400 average-rate 200

dynamic-access-policy-record DfltAccessPolicy

username xxx password 2dMuEBodaRTg/ojQ encrypted privilege 15

username xxx password rFMCRvdj4RRRNLzF encrypted privilege 15

username xxx password cmyrcWm5arRxckSs encrypted privilege 15

class-map global-class-NetFlow

match any

class-map CM-xxx-THROTTLE

match access-list xxx-THROTTLE

class-map inspection_default

match default-inspection-traffic

policy-map type inspect dns preset_dns_map

parameters

message-length maximum client auto

message-length maximum 512

no tcp-inspection

policy-map PM-xxx-THROTTLE

class CM-xxx-THROTTLE

police input 4000000 4000

police output 4000000 4000

policy-map global_policy

class inspection_default

inspect dns preset_dns_map

inspect ftp

inspect h323 h225

inspect h323 ras

inspect rsh

inspect rtsp

inspect esmtp

inspect sqlnet

inspect skinny

inspect sunrpc

inspect xdmcp

inspect sip

inspect netbios

inspect tftp

inspect ip-options

class global-class-NetFlow

flow-export event-type all destination 192.168.81.17

class class-default

user-statistics accounting

policy-map global-policy

class inspection_default

inspect icmp

inspect icmp error

service-policy global_policy global

service-policy PM-xxx-THROTTLE interface inside

prompt hostname context

no call-home reporting anonymous

hpm topN enable

Cryptochecksum:0d74e8c37dxxxxxxxe6d6166105

: end

no asdm history enable

Re: Firewall 5506-x blocking all DNS queries

Martin Kling — Tue, 16 Oct 2018 06:54:53 GMT

Hi,

Normally wouldn't configuration issues cause working functions to stop working as they wouldn't work from the beginning. But I see that you have a lot of functions enabled like netflow, route maps, threat inspection etc enabled. Normally you are more exposed to hitting bugs with the more functions you enable. You are also running a code that no longer is downloadable.

My suggestion would be to upgrade the firewall to 9.6.4 or 9.8.2 (both last interim release) that have fixes for various bugs like memory leaks (which could explain your issues) alon with monitoring the logs on your syslog server

Best regards,

Martin

Re: Firewall 5506-x blocking all DNS queries

Zargham Haider — Tue, 16 Oct 2018 11:18:18 GMT

Hi Martin,

Thanks for your reply....

Actually you are right.... this was Memory leak bug. i found the solution. this is Cisco Bug: CSCvd71473. it relates to DNS memory leak "slow memory leak when using many DNS queries". This issue is seen whenever a DNS query gets resolved on ASA. We could see a small amount of memory leak (around 64 Bytes with each DNS query getting resolved) on ASA.

for detailed information please read this Bug info:

https://quickview.cloudapps.cisco.com/quickview/bug/CSCvd71473

Now i must look for latest codes as you said.