ISE sending null DNS queries since upgrade to 3.3 patch 4

[Danny Dulin]

Within hours of upgrading ISE to 3.3 patch 4, we began seeing ISE sending DNS A record queries with an empty name. The packet capture reads "Null".

Domain Name System (query)

Transaction ID: 0x5e24

Flags: 0x0100 Standard query

Questions: 1

Answer RRs: 0

Authority RRs: 0

Additional RRs: 0

Queries

null: type A, class IN

Name: null

Name Length: 4

Label Count: 1

Type: A (Host Address) (1)

Class: IN (0x0001)

Also, I see multiple request by the PAN for its own FQDN. Also, I see request by the PAN for the secondary's FQDN.

[Arne Bier]

Strange - I have been running 3.3p4 for a long time and never seen this. I just ran a capture now and I can't find any null queries. Or queries from a node requesting the IP address for its own FQDN.

How often do you see these null queries?

ISE never used to cache DNS replies, which resulted in ISE making a horrendously high number of requests, putting strain on DNS servers. The solution is to configure DNS caching on the CLI of every ISE node.

conf t
  service cache enable hosts ttl 3600
end

The command takes immediate effect - in my example I have set TTL of 3600 seconds if the DNS record has no TTL set - ISE will honour the TTL - but in the absence of a value, it will use your configured value instead.

Since ISE 3.4, the DNS caching command was mandated during install (not sure if upgrading to ISE 3.4 also forces this command onto the CLI)

Having said all that, I just noticed that the DNS entries for my ISE nodes have a TTL of 3600 seconds in the DNS Answer frame, but ISE doesn't seem to care about that - and it will perform a DNS query every 30 seconds instead.  Unless I missed something, that seems broken to me.  I might have to open a TAC case to get some answers on this. It surprised me also that since the DNS caching feature was introduced, there was no "show" command to see how (if) it's working.

[Danny Dulin]

These Null DNS requests happen quite often. At least I receive IPS alerts often since SNORT thinks the null DNS request is malicious.

I have the default setting service cache ttl 180.

How did you determine this:

I just noticed that the DNS entries for my ISE nodes have a TTL of 3600 seconds in the DNS Answer frame, but ISE doesn't seem to care about that - and it will perform a DNS query every 30 seconds instead.

[Arne Bier]

I have not done a complete analysis of all node types (PAN, MNT, PSN, pxGrid) but I noticed that on my PAN, the same DNS query was being made every 30 seconds, despite the TTL in the DNS server's response containing 3600 seconds. That tells me that there are most likely different software libraries running in ISE that don't play in harmony with the rest of the system - they just do whatever they like and disregard the cache.

On a PSN (used only for TACACS) I ran a 2 hour tcpdump with the filter "udp port 53" and looked at the results in wireshark. In this case the TTL was being honoured and a new DNS query for other ISE nodes was seen every 3600 seconds.

[Danny Dulin]

Arne,

In all your investigation, did you ever find a null DNS request?

[Arne Bier]

Not yet. But I also don't look into this much. If I was in this situation I would engage the TAC. Not much we can do as end users.

[Danny Dulin]

Thanks @Arne Bier I have engaged TAC and they've been no help.

[Arne Bier]

You have to keep on them and escalate - you'll eventually end up in Cisco developer land

[Marcelo Morais]

Hi @Danny Dulin ,

 please take a look at: CSCwk63923 DNS cache timeout is not honored.

Although you update to ISE 3.3 P4, my recommendation is to apply the Workaround.

Hope this helps !!!