cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
15860
Views
5
Helpful
7
Replies

CISCO DNS server problem - it stops serving the clients

Hi,

 

I have the following problem: my Cisco 851 Router stops resolving DNS requests from the hosts.

This is happening at random intervals after periods when all works fine.

Below are the captures of one of these moments when I did not got any DNS resolution for one site: accounts.google.ro

All hosts in the network use the Cisco Router as DNS server.

 

 

My Host (XP machine) can't reach a site:

Server not found

Firefox can't find the server at accounts.google.ro.

    Check the address for typing errors such as ww.example.com instead of www.example.com
    If you are unable to load any pages, check your computer's network connection.
    If your computer or network is protected by a firewall or proxy, make sure that Firefox is permitted to access the Web.

 

Test the site accounts.google.ro.

 

C:\>ping accounts.google.ro
Ping request could not find host accounts.google.ro. Please check the name and try again.

 

I've started the Domain name debugging on my Cisco and I also started WhireShark

 

ygh#sh debugging
Domain Name System debugging is on


018212: Mar 28 14:29:16.526 EET: DNS: Incoming UDP query (id#44315)
018213: Mar 28 14:29:16.526 EET: DNS: Type 1 DNS query (id#44315) for host 'accounts.google.ro' from 192.168.1.185(58891)
018214: Mar 28 14:29:16.526 EET: DNS: Servicing request using view default
018215: Mar 28 14:29:16.526 EET: DNS: Replying to query (id#44315) with NS
018216: Mar 28 14:29:16.526 EET: DNS: Reply to client 192.168.1.185/58891 query A with NS
018217: Mar 28 14:29:16.526 EET: DNS: Finished processing query (id#44315) in 0.000 secs
018218: Mar 28 14:29:16.526 EET: DNS: Sending response to 192.168.1.185/58891, len 36

Above lines does not look ok to me: the Cisco box should forward the request to the external DNS server 208.67.222.222!

I need recursive DNS resolution from my Cisco.


WhireShark packets: (see attached picture)

818    1334.358526000    192.168.1.185    192.168.1.1    DNS    78    Standard query 0xad1b  A accounts.google.ro
819    1334.362361000    192.168.1.1    192.168.1.185    DNS    78    Standard query response 0xad1b

 

From the host I test if the name resolution works with the External DNS server used by Cisco

C:\>nslookup accounts.google.ro 208.67.222.222
Server:  resolver1.opendns.com
Address:  208.67.222.222

Non-authoritative answer:
Name:    accounts.google.ro.no-ip.biz
Address:  67.215.65.132

 

Same request but using Cisco DNS service:

C:\>nslookup accounts.google.ro 192.168.1.1
1.1.168.192.in-addr.arpa
        primary name server = localhost
        responsible mail addr = nobody.invalid
        serial  = 1
        refresh = 600 (10 mins)
        retry   = 1200 (20 mins)
        expire  = 604800 (7 days)
        default TTL = 10800 (3 hours)
*** Can't find server name for address 192.168.1.1: No information
Server:  UnKnown
Address:  192.168.1.1

*** No address (A) records available for accounts.google.ro

 

Cisco debugging information for above host request looks like this:


018314: Mar 28 14:32:20.584 EET: DNS: Incoming UDP query (id#1)
018315: Mar 28 14:32:20.584 EET: DNS: Type 12 DNS query (id#1) for host '1.1.168.192.in-addr.arpa' from 192.168.1.185(1522)
018316: Mar 28 14:32:20.584 EET: DNS: Servicing request using view default
018317: Mar 28 14:32:20.584 EET: DNS: Re-sending DNS query (type 12, id#34776) to 208.67.222.222
018318: Mar 28 14:32:20.628 EET: DNS: Incoming UDP query (id#34776)
018319: Mar 28 14:32:20.632 EET: DNS: Type 12 response (id#34776) for host <1.1.168.192.in-addr.arpa> from 208.67.222.222(53)
018320: Mar 28 14:32:20.632 EET: DNS: Forwarded back non-A response
018321: Mar 28 14:32:20.632 EET: DNS: Finished processing query (id#1) in 0.048 secs
018322: Mar 28 14:32:20.632 EET: DNS: Forwarding back reply to 192.168.1.185/1522
018330: Mar 28 14:32:20.648 EET: DNS: Incoming UDP query (id#3)
018331: Mar 28 14:32:20.648 EET: DNS: Type 1 DNS query (id#3) for host 'accounts.google.ro' from 192.168.1.185(1524)
018332: Mar 28 14:32:20.648 EET: DNS: Servicing request using view default
018333: Mar 28 14:32:20.648 EET: DNS: Replying to query (id#3) with NS
018334: Mar 28 14:32:20.648 EET: DNS: Reply to client 192.168.1.185/1524 query A with NS
018335: Mar 28 14:32:20.648 EET: DNS: Finished processing query (id#3) in 0.000 secs
018336: Mar 28 14:32:20.648 EET: DNS: Sending response to 192.168.1.185/1524, len 36

WhireShark packets:(see attached picture)
840    1518.481220000    192.168.1.185    192.168.1.1    DNS    78    Standard query 0x0003  A accounts.google.ro
841    1518.484925000    192.168.1.1    192.168.1.185    DNS    78    Standard query response 0x0003

==========================================

 

Cisco DNS cache looks like this:


ygh#sh hosts
Default domain is no-ip.biz
Name/address lookup uses domain service
Name servers are 208.67.222.222, 208.67.220.220, 193.231.252.1, 213.154.124.1 - Note the DNS servers used by Cisco box

Codes: UN - unknown, EX - expired, OK - OK, ?? - revalidate
       temp - temporary, perm - permanent
       NA - Not Applicable None - Not defined

Host                      Port  Flags      Age Type   Address(es)
safebrowsing.cache.l.goog None  (temp, OK)  0   IP    62.231.75.241
                                                      62.231.75.237
                                                      62.231.75.242
                                                      62.231.75.247
                                                      62.231.75.216
                                                      62.231.75.221
                                                      62.231.75.226
                                                      62.231.75.222
                                                      62.231.75.212
                                                      62.231.75.227
                                                      62.231.75.236
                                                      62.231.75.217
                                                      62.231.75.232
                                                      62.231.75.251
                                                      62.231.75.246
                                                      62.231.75.231
  safebrowsing-cache.google
Host                      Port  Flags      Age Type   Address(es)
clients.l.google.com      None  (temp, OK)  0   IP    62.231.75.227
                                                      62.231.75.231
                                                      62.231.75.221
                                                      62.231.75.242
                                                      62.231.75.222
                                                      62.231.75.232
                                                      62.231.75.251
                                                      62.231.75.246
                                                      62.231.75.226
                                                      62.231.75.247
                                                      62.231.75.236
                                                      62.231.75.237
                                                      62.231.75.216
                                                      62.231.75.241
                                                      62.231.75.217
                                                      62.231.75.212
  safebrowsing.clients.goog
sb-ssl.l.google.com       None  (temp, EX)  0   IP    82.76.79.114
                                                      82.76.79.108
                                                      82.76.79.93
                                                      82.76.79.88
                                                      82.76.79.89
Host                      Port  Flags      Age Type   Address(es)
                                                      82.76.79.94
                                                      82.76.79.84
                                                      82.76.79.109
                                                      82.76.79.123
                                                      82.76.79.119
                                                      82.76.79.98
                                                      82.76.79.113
                                                      82.76.79.103
                                                      82.76.79.104
                                                      82.76.79.99
                                                      82.76.79.118
  sb-ssl.google.com
google.com                None  (temp, OK)  0   IP    62.231.75.247
                                                      62.231.75.221
                                                      62.231.75.241
                                                      62.231.75.237
                                                      62.231.75.216
                                                      62.231.75.212
                                                      62.231.75.227
                                                      62.231.75.217
                                                      62.231.75.231
                                                      62.231.75.246
Host                      Port  Flags      Age Type   Address(es)
                                                      62.231.75.242
                                                      62.231.75.236
                                                      62.231.75.251
                                                      62.231.75.222
                                                      62.231.75.232
                                                      62.231.75.226
                                                SOA      ns1.google.com dns-admin.google.com
                                                   1551121 7200 1800 1209600 300
no-ip.biz                 NA    (temp, OK)  0
google.ro                 NA    (temp, OK)  0
bud02s01-in-f12.1e100.net None  (temp, OK)  0   IP    173.194.39.76
bud02s02-in-f10.1e100.net None  (temp, OK)  0   IP    173.194.39.106
nf4.no-ip.com             None  (temp, OK)  0   IP    180.92.187.122
nf3.no-ip.com             None  (temp, OK)  0   IP    69.65.40.108
nf2.no-ip.com             None  (temp, OK)  0   IP    69.72.255.8
nf1.no-ip.com             None  (temp, OK)  0   IP    50.31.129.129
fra02s20-in-f11.1e100.net None  (temp, OK)  0   IP    173.194.113.43
easylist-downloads.adbloc None  (temp, OK)  0   IP    213.239.212.163
                                                      78.46.51.36
                                                      78.46.70.139
                                                      85.10.195.245
Host                      Port  Flags      Age Type   Address(es)
                                                      88.198.10.10
                                                      88.198.15.197
                                                      88.198.16.240
                                                      88.198.34.145
                                                      88.198.35.145
                                                      88.198.48.196
                                                      88.198.50.132
                                                      88.198.59.19
                                                      178.63.96.74
                                                      178.63.103.200
                                                      188.40.105.83
ygh#

 

I clear the DNS cache - in 30-40% of the cases this clears my problem


ygh#clear  host *
ygh#sh hosts
Default domain is no-ip.biz
Name/address lookup uses domain service
Name servers are 208.67.222.222, 208.67.220.220, 193.231.252.1, 213.154.124.1

Codes: UN - unknown, EX - expired, OK - OK, ?? - revalidate
       temp - temporary, perm - permanent
       NA - Not Applicable None - Not defined

Host                      Port  Flags      Age Type   Address(es)
ygh#

 

After this all is back to normal:

 

C:\>ping accounts.google.ro

Pinging accounts-cctld.l.google.com [173.194.70.94] with 32 bytes of data:

Reply from 173.194.70.94: bytes=32 time=30ms TTL=48
Reply from 173.194.70.94: bytes=32 time=29ms TTL=48
Reply from 173.194.70.94: bytes=32 time=29ms TTL=48
Reply from 173.194.70.94: bytes=32 time=29ms TTL=48

Ping statistics for 173.194.70.94:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 29ms, Maximum = 30ms, Average = 29ms

 

Cisco Debugging information:

    ygh#
018339: Mar 28 14:35:59.493 EET: DNS: Incoming UDP query (id#50658)
018340: Mar 28 14:35:59.493 EET: DNS: Type 1 DNS query (id#50658) for host 'accounts.google.ro' from 192.168.1.185(60647)
018341: Mar 28 14:35:59.493 EET: DNS: Servicing request using view default
018342: Mar 28 14:35:59.493 EET: DNS: Re-sending DNS query (type 1, id#23851) to 208.67.222.222
018343: Mar 28 14:35:59.545 EET: DNS: Incoming UDP query (id#23851)
018344: Mar 28 14:35:59.545 EET: DNS: Type 1 response (id#23851) for host <accounts.google.ro> from 208.67.222.222(53)
018345: Mar 28 14:35:59.545 EET: DOM: dom2cache: hostname is accounts.google.ro, RR type=5, class=1, ttl=85917, n=29
018346: Mar 28 14:35:59.545 EET: DOM: dom2cache: hostname is accounts.google.ro, RR type=1, class=1, ttl=300, n=4
018347: Mar 28 14:35:59.545 EET: DNS: Forwarding back A response - no director required
018348: Mar 28 14:35:59.545 EET: DNS: Finished processing query (id#50658) in 0.052 secs
018349: Mar 28 14:35:59.545 EET: DNS: Forwarding back reply to 192.168.1.185/60647

WhireShark packets: (see attached picture)
842    1737.330561000    192.168.1.185    192.168.1.1    DNS    78    Standard query 0xc5e2  A accounts.google.ro
843    1737.386290000    192.168.1.1    192.168.1.185    DNS    135    Standard query response 0xc5e2  CNAME accounts-cctld.l.google.com A 173.194.70.94

 

This time all worked fine but usually the clear host * does not help.

 

Here is the Cisco configuration and model:


ip domain name no-ip.biz
ip name-server 208.67.222.222
ip name-server 208.67.220.220
ip dns server

access-list 100 permit udp any host 208.67.222.222 eq domain //first external DNS server
access-list 100 permit udp any host 192.168.1.1 eq domain //the Cisco box
access-list 100 permit udp any host 208.67.220.220 eq domain // second external DNS server
access-list 100 permit tcp any host 192.168.1.1 eq domain //Cisco box on TCP.
access-list 100 deny   udp any any eq domain
access-list 100 deny   tcp any any eq domain

interface Vlan1
 description $ETH-SW-LAUNCH$$INTF-INFO-HWIC 4ESW$$FW_INSIDE$
 ip address 192.168.1.1 255.255.255.0
 ip access-group 100 in
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 ip nat inside
 ip virtual-reassembly
 ip tcp adjust-mss 1452


ygh#sh ver
Cisco IOS Software, C850 Software (C850-ADVSECURITYK9-M), Version 12.4(15)T17, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2012 by Cisco Systems, Inc.
Compiled Tue 24-Jan-12 14:40 by prod_rel_team

ROM: System Bootstrap, Version 12.3(8r)YI4, RELEASE SOFTWARE

ygh uptime is 1 week, 5 days, 21 hours, 29 minutes
System returned to ROM by reload at 17:09:09 EET Sat Mar 15 2014
System restarted at 17:09:56 EET Sat Mar 15 2014
System image file is "flash:c850-advsecurityk9-mz.124-15.T17.bin"

 

Why Cisco DNS stops forwarding the request to the external DNS server and respond to the client with NS record?

The host is not able to reach that NS: see access list 100. The host can reach the same external DNS server like Cisco - but is not allowed to reach other DNS servers! Cisco should resolve the query from the host in iterative mode!

 

As I said: this behavior is random. Is happening 1-2 times per week.
   

 

7 Replies 7

Hello

Try changing you dhcp server dns settings for clients to be serviced by DNS on the public addressing rather than using you router as a forwarder and also disable the dns server service on your router

res

Paul


Please rate and mark as an accepted solution if you have found any of the information provided useful.
This then could assist others on these forums to find a valuable answer and broadens the community’s global network.

Kind Regards
Paul

Hi Paul,

I already do (did) this by manually setting DNS servers on my host.

But it is outside of my goal. And, with all do respect, I was expecting more from the official Cisco Support forum: a solution to fix the usage of the DNS service and not to bypass it.

 

My goal is to use the Cisco box 851 in the end.

If it not suitable for the job I rather change it (back) with the Linksys router I used in the past 5 years (WRT54GL). For the record: the Linksys worked flawlessly for 5 years as Router & Cache DNS server (including DNS spoofing).

My final goal is to activate the DNS spoofing and redirect all DNS requests to OpenDNS servers in order to secure my network from Parental Control & some security perspective (blocked all .ru and .cn sites, malware known sites, P2P sites, etc.

 

I just want to know if this is a problem that can be fixed (but still using the Cisco box) or I need to forget about using the Cisco as DNS Cache (and later as DNS spoofing).

If my box is has some built in problems with DNS service I will throw away Cisco machine and reuse the old Linksys router - but first I want to give a chance to the box.

 

On the topic:

After reading some DNS materials on Cisco site I discovered a tool for DNS stress tests.

I've got a DNS digger (TXDNS) and hit the Cisco box with around 12000 requests in 4-5 minutes!

TXDNS.exe -rt -s 192.168.1.1 -t google.com
 Resolved names: 12110
 Failed queries: 40
  Total queries: 12150

ygh#show ip dns statistics
DNS requests received = 109326 ( 108461 + 865 )
DNS requests dropped  = 0 ( 0 + 0 ) - Is this "fake"? I've captured packets from the Cisco box with no response in the packets! OK, request was received and answer returned, but the answer was "EMPTY".
DNS responses replied = 6741 ( 5915 + 826 )

Forwarder queue statistics:
Current size = 0
Maximum size = 21
Drops        = 0

Director queue statistics:
Current size = 0
Maximum size = 0
Drops        = 0

 

The DNS cache resolved all +12000 requests and the Cache become quite impressive.

No problems during the tests or after that.

As I said, clearing the cache solve my problem in 30%-40% of the cases. So I don't suspect a DNS cache problem. Maybe something else is to blame...

My above stress test might not be relevant for DNS cache performance. If this is the case, please indicate another stress test and I will perform it.

 

PS: due to the rate-limit imposed in logging, my access-list displays lower number of hits for DNS traffic rules but I saw a big increase in the number of access-list hit also. But this number is not comparable with the number of DNS requests solved by the DNS service or by the TXDNS report at the end of the test.

 

Problem solved by recycling the Cisco box and building my own router&firewall starting from a Small Board Computer and adding on top of it one of the FW listed here: http://en.wikipedia.org/wiki/Comparison_of_firewalls 

 

 

Hello

My understanding is cisco Ios running as a DNS sever doesn't perform recursion on the query's, it just forwards them, and this ro me points to the public dns servers and as to the reason the recursion isn't working from them I cannot say.

 

So that being said that, This was the reason I asked you to bypass the router, to check if this soho router had any reference to your issue and to is see if you still experienced the problem you reported, However you didn't post the result of this change if it work or not?

 

 

res
Paul


Please rate and mark as an accepted solution if you have found any of the information provided useful.
This then could assist others on these forums to find a valuable answer and broadens the community’s global network.

Kind Regards
Paul

Hello So for future reference did you bypass the cisco box and put dns on a separate device? Res Paul

Please rate and mark as an accepted solution if you have found any of the information provided useful.
This then could assist others on these forums to find a valuable answer and broadens the community’s global network.

Kind Regards
Paul

Hi,

Yes, I put the Cisco box on trash bin and build a new router. No more Cisco in my business!

 

Sadly I paid 600€ for the Ciso box and only 130 € for the new SBC (hardware).

The software (router + firewall + GeoIP filtering + content filtering + content rating + IDS + IPS) was free - one of the free vendors listed on Wikipedia.

And now I get a full 100Mbit/s download&upload speed. The Cisco was limiting me to 30 Mbit/sec (CBAC was putting down the box resources!)

To give 600 Euro and your basic services to crash is unacceptable.

To pay 130 Euro and get 3 times the speed of the Cisco and more functionality (including the GeoIP Filtering - the one that Cisco said is hard to obtain) is ...priceless!  :)

 

 

 

 

chad patterson
Level 1
Level 1

I have the a similar problem. It is not exactly the same, but the Cisco 1801 router also stops serving DNS responses to clients. Clearing the hosts table seems to be the only solution. I have tried to resolve the issue. to no avail, by limiting the forwarding with the command:

#ip dns server queue limit forwarder 'VARIOUS INTEGERS'

I believe that my problem stems from running too much on the router. The memory  gets full and seems to cause this issue for me. I have turned off layer 7 filtering (using nbar), and I am watching now for improvement. 

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: