dns through asa returns inspect-dns-invalid-pak

[mcmurphytoo]

ASA5510, ASA 8.0(4), ASDM 6.1(5), this is a productino ASA with plenty of lookups working through its 3 interfaces - outside, inside, dmz.  The problem is a new use.  I've segmented a switch on the inside network with a VLAN, and have a workstation routing through the switch to the default VLAN where all other hosts on the inside network reside so far.  The ASA inside interface is the default gateway for the inside network.  My test worksttion can PING inside hosts, so the static route is OK.

     ASA 10.1.1.2/16     DNS Server 10.1.5.1/16

                |                                  |

------------------------------------------------------------------

                    |

               Switch 10.1.8.20/16

               VLAN 10.7.1.1/16

                         |

--------------------------------------------

                                   |

                              Test Wkst. 10.7.1.10/16

  But lookups fail, Wireshark says the test workstation sends, the dns server receives and responds, but the test workstation never receives.  I used the Packet Tracer tool, it gets to the last step syayin OK then finally "inspect-dns-invalid-pak".  I can't find any more there to tell just what is invlid about it.  So I'm trying to figure out global inspection.  While I'm trying to learn I appreciate any help.  Here's an extract from the config:

class-map inspection_default

match default-inspection-traffic

!

!

policy-map global_policy

class inspection_default

  inspect ftp

  inspect h323 h225

  inspect h323 ras

  inspect rsh

  inspect rtsp

  inspect skinny 

  inspect sunrpc

  inspect xdmcp

  inspect sip 

  inspect netbios

  inspect tftp

  inspect dns

policy-map type inspect dns migrated_dns_map_1

parameters

  message-length maximum 512

!

service-policy global_policy global

[mcmurphytoo]

Curiouser and curiouser.  I nno longer see the dns packet inspsection error I started with, but I still do not get NS lookups working.  I re-enabled DNS inspection, since disabling it did not seem to change the problem:

class-map inspection_default

match default-inspection-traffic

!

!

policy-map global_policy

class inspection_default

  inspect ftp

  inspect h323 h225

  inspect h323 ras

  inspect rsh

  inspect rtsp

  inspect skinny 

  inspect sunrpc

  inspect xdmcp

  inspect sip 

  inspect netbios

  inspect tftp

  inspect dns

policy-map type inspect dns migrated_dns_map_1

parameters

  message-length maximum 512

class-map inspection_default

match default-inspection-traffic

!

!

policy-map global_policy

class inspection_default

  inspect ftp

  inspect h323 h225

  inspect h323 ras

  inspect rsh

  inspect rtsp

  inspect skinny 

  inspect sunrpc

  inspect xdmcp

  inspect sip 

  inspect netbios

  inspect tftp

  inspect dns

policy-map type inspect dns migrated_dns_map_1

parameters

  message-length maximum 512

Here's the sequence when I try lookups from my test PC at 10.7.1.10:

Sniffer on test PC shows dns requests going to 10.1.5.1 and 10.1.5.54 (I have 2 dns servers)

Sniffer on dns server 10.1.5.54 show it receives DNS query, sends DNS response

Real-Time Log Viewer on ASA shows a recurring sequence of:

  Built inbound UDP connection for inside 'dns-server' to inside 'test PC'

  Teardown UDP connection for inside 'dns-server' to inside 'test PC' duration 0:00:00 bytes 0

I ran the Capture,  it showed:

Is this sequence a problem?  10.7.1.10 sends packet destination 10.1.5.54.  In the switchVLAN the packet routes to the 10.1 network, where the destination 10.1.5.54 is local.  So the packet 10.7.1.10 sends does not touch the ASA 10.1.1.2, but goes directly to 10.1.5.54.  10.1.5.54 replies with a packet with destination 10.7.1.10 on another network, so it has to route the packet through the ASA 10.1.1.2.  The ASA has a static route 10.7.0.0 255.255.0.0 10.1.8.20, the ASA builds the connection 10.1.5.54 to 10.7.1.10, but tears it down right away for some reason and drops the packet.

[Julio Carvajal]

Hello,

Asymetric routing seems to be the problem, ASA might be dropping the packets as he is not waiting for a DNS reply.

Here is what I though.

Test-pc------R1-----------Switch-------ASA

                                     !

                                     !

                                 Dns server

DNS server default gateway is ASA.

1-So test-pc udp packet will go to its default gateway R1, he will see the destination IP address and will know is directly connected so will send it to the DNS server without going to the ASA....

2-DNS server will reply to its default gateway witch is the ASA, he will receive the DNS reply and will say, Wait a minute I do not have on my existing connections a entry for this reply, so it will drop it.

How can we change this:

I did a lab recreation and I think this is the real solution ( work-around (Not real solution)make dns default gateway R1)

We need to let R1 that in order to get to the DNS server he needs to send the packet to the ASA interface,

For that I need you to give me an ip address you are not using on that segment ( inside network connected to dns) Lets say you gave me 10.1.5.33

1-Static (inside,inside) 10.1.5.33 10.1.5.54

The ASA will let R1 I have 10.1.5.33, You also will need to change in the PC test the DNS ip addres to 10.1.5.33

You also need a Global for the packet comming from the inside because as you know all the packets getting to a ASA with nat control enable need a global to match

2- global (inside) 1 inside

Here is the packet tracer output on my lab recreation (Test pc is 192.168.20.2, Server DNS 192.168.15.2 natted to 192.168.2.44):

# packet-tracer input inside udp 192.168.20.2 1025 192.168.15.

Phase: 1

Type: FLOW-LOOKUP

Subtype:

Result: ALLOW

Config:

Additional Information:

Found no matching flow, creating a new flow

Phase: 2

Type: UN-NAT

Subtype: static

Result: ALLOW

Config:

static (inside,inside) 192.168.15.44 192.168.15.2 netmask 255.255.255.255

  match ip inside host 192.168.15.2 inside any

    static translation to 192.168.15.44

    translate_hits = 0, untranslate_hits = 4

Additional Information:

NAT divert to egress interface inside

Untranslate 192.168.15.44/0 to 192.168.15.2/0 using netmask 255.255.255.255

Phase: 3

Type: ACCESS-LIST

Subtype:

Result: ALLOW

Config:

Implicit Rule

Additional Information:

Phase: 4

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Config:

Additional Information:

Phase: 5

Type: NAT

Subtype:

Result: ALLOW

Config:

nat (inside) 1 0.0.0.0 0.0.0.0

  match ip inside any inside any

    dynamic translation to pool 1 (192.168.15.1 [Interface PAT])

    translate_hits = 13, untranslate_hits = 0

Additional Information:

Dynamic translate 192.168.20.2/1025 to 192.168.15.1/54941 using netmask 255.255.255.255

Phase: 6

Type: NAT

Subtype: host-limits

Result: ALLOW

Config:

nat (inside) 1 0.0.0.0 0.0.0.0

  match ip inside any inside any

    dynamic translation to pool 1 (192.168.15.1 [Interface PAT])

    translate_hits = 13, untranslate_hits = 0

Additional Information:

Phase: 7

Type: NAT

Subtype: rpf-check

Result: ALLOW

Config:

static (inside,inside) 192.168.15.44 192.168.15.2 netmask 255.255.255.255

  match ip inside host 192.168.15.2 inside any

    static translation to 192.168.15.44

    translate_hits = 0, untranslate_hits = 2

Additional Information:

Phase: 8

Type: NAT

Subtype: host-limits

Result: ALLOW

Config:

static (inside,inside) 192.168.15.44 192.168.15.2 netmask 255.255.255.255

  match ip inside host 192.168.15.2 inside any

    static translation to 192.168.15.44

    translate_hits = 0, untranslate_hits = 2

Additional Information:

Phase: 9

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Config:

Additional Information:

Phase: 10

Type: FLOW-CREATION

Subtype:

Result: ALLOW

Config:

Additional Information:

New flow created with id 20, packet dispatched to next module

Phase: 11

Type: ROUTE-LOOKUP

Subtype: output and adjacency

Result: ALLOW

Config:

Additional Information:

found next-hop 192.168.15.2 using egress ifc inside

adjacency Active

next-hop mac address ca00.0e10.001c hits 2

Result:

input-interface: inside

input-status: up

input-line-status: up

output-interface: inside

output-status: up

output-line-status: up

Action: allow

pixfirewall(config)#  sh run glo

pixfirewall(config)#  sh run global

global (inside) 1 interface

global (outisde) 1 interface

pixfirewall(config)# packet-tracer input inside udp 192.168.20.2 1025 192.168.$

Phase: 1

Type: FLOW-LOOKUP

Subtype:

Result: ALLOW

Config:

Additional Information:

Found no matching flow, creating a new flow

Phase: 2

Type: UN-NAT

Subtype: static

Result: ALLOW

Config:

static (inside,inside) 192.168.15.44 192.168.15.2 netmask 255.255.255.255

  match ip inside host 192.168.15.2 inside any

    static translation to 192.168.15.44

    translate_hits = 0, untranslate_hits = 3

Additional Information:

NAT divert to egress interface inside

Untranslate 192.168.15.44/0 to 192.168.15.2/0 using netmask 255.255.255.255

Phase: 3

Type: ACCESS-LIST

Subtype:

Result: ALLOW

Config:

Implicit Rule

Additional Information:

Phase: 4

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Config:

Additional Information:

Phase: 5

Type: NAT

Subtype:

Result: ALLOW

Config:

nat (inside) 1 0.0.0.0 0.0.0.0

  match ip inside any inside any

    dynamic translation to pool 1 (192.168.15.1 [Interface PAT])

    translate_hits = 14, untranslate_hits = 0

Additional Information:

Dynamic translate 192.168.20.2/1025 to 192.168.15.1/1194 using netmask 255.255.255.255

Phase: 6

Type: NAT

Subtype: host-limits

Result: ALLOW

Config:

nat (inside) 1 0.0.0.0 0.0.0.0

  match ip inside any inside any

    dynamic translation to pool 1 (192.168.15.1 [Interface PAT])

    translate_hits = 14, untranslate_hits = 0

Additional Information:

Phase: 7

Type: NAT

Subtype: rpf-check

Result: ALLOW

Config:

static (inside,inside) 192.168.15.44 192.168.15.2 netmask 255.255.255.255

  match ip inside host 192.168.15.2 inside any

    static translation to 192.168.15.44

    translate_hits = 0, untranslate_hits = 3

Additional Information:

Phase: 8

Type: NAT

Subtype: host-limits

Result: ALLOW

Config:

static (inside,inside) 192.168.15.44 192.168.15.2 netmask 255.255.255.255

  match ip inside host 192.168.15.2 inside any

    static translation to 192.168.15.44

    translate_hits = 0, untranslate_hits = 3

Additional Information:

Phase: 9

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Config:

Additional Information:

Phase: 10

Type: FLOW-CREATION

Subtype:

Result: ALLOW

Config:

Additional Information:

New flow created with id 21, packet dispatched to next module

Phase: 11

Type: ROUTE-LOOKUP

Subtype: output and adjacency

Result: ALLOW

Config:

Additional Information:

found next-hop 192.168.15.2 using egress ifc inside

adjacency Active

next-hop mac address ca00.0e10.001c hits 3

Result:

input-interface: inside

input-status: up

input-line-status: up

output-interface: inside

output-status: up

output-line-status: up

Action: allow

Hope this helps!!

Regards,

Julio

[mcmurphytoo]

Sorry for the delay, the usual distractions from working things that break.  I'm trying to be sure I understand this before I put it in on a working ASA.  My config already has a "Global (outside) 1 interface", and all packets arriving on the inside are translated unless they meet one of the NAT exceptions.  Is the "Global (inside) 1 inside" you recommend also noeeded?  And does it not conflict naming the same nat_id?

[Julio Carvajal]

Hello,

It will no affect,because the ASA only use the global (inside) to contact inside ip addresses not outside ip addresses (public ones)

Regards,

Julio

[mcmurphytoo]

I see where I add a static route to the ASA:

1-Static (inside,inside) 10.1.5.33 10.1.5.54

This next I don't understand.  Am I to assign 10.1.5.33 to an interface?

The ASA will let R1 I have 10.1.5.33.

I get this - change the Test PC the DNS ip address to 10.1.5.33 from 10.1.5.54

And you explained the global, so it's OK

2- global (inside) 1 inside

[Julio Carvajal]

Hello,

All we are doing is trying to avoid the Asymetric routing on this particular connection to see if that solves our problem..

1-Static (inside,inside) 10.1.5.33 10.1.5.54

This next I don't understand.  Am I to assign 10.1.5.33 to an interface?

The ASA will let R1 I have 10.1.5.33.

What we are doing here is to natting the internal server 10.1.5.33 to x.x.x.x.54 so when a user behind the router wants to connect to the server will need to go to the ASA first, instead of sending the packet direct to the server.

Do you understand?

Regards,

Julio

[mcmurphytoo]

Yes.  I've spent my time with the ASDM, not enought with the command line, I'd confused Static NAT and static route.  The NAT entry is in OK, but the global command gives me:

[Julio Carvajal]

Hello,

I am sorry, it should be global (inside) 1 interface

[mcmurphytoo]

I used 10.1.5.90 as the mapped address

My Test PC has 10.1.5.90 as DNS Server.

I try to do a nslookup, it gets "Can't find server name for address 10.1.5.90"

I'm running Wireshark monitoring the switch port that is the network inside interface, 10.1.8.20

I see 20.1.8.20 sending an ARP request Who Has 10.1.5.90, and getting no response

That causes the Test PC to fail to find its DNS server 10.1.5.90

I know the ASA config includes "sysopt noproxyarp inside"

It looks like if proxyarp were not disabled then the ASA would be responding to the ARP with its MAC

Then the 10.1.8.20 would be sending the nslookup to the ASA, which would via NAT send it to the real DNS server 10.1.5.54

The Cisco command reference says that proxyarp by defaualt is enabled, so I'm trying to remember at what point I disabled it.

I think it was at the same time as the "same-security-traffic permit intra-interface", but couldn't swear to it.

[Julio Carvajal]

That is correct, you will need to enable it again.!!

[mcmurphytoo]

Long interruption, sorry.  I did enable proxyarp again.  Now I'm back to my 'inspect-dns-invalid-pak' errors.  I think at this point my plan is to be sure I have an inside router being my default gateway.  I'll let it do the inside work, and let my ASA focus on the DMZ, the Internet, and inside host communications with them.  I'm sure the ASA can it all, but if I can simplify its configuration that will be easier for me to keep up with, and let it apply its resources where best utilized.  

[Julio Carvajal]

Hello,

That looks Okay, let me know if there is something else I can do for you.

Julio

[dedmundson]

After updating our Cisco ASA 5550 from 8.0 code to 9.0(3)8 code we had the same problem. And from this post fixed our problem. Here are the changes that I made to out ASA.

I also had to add two rules for out public facing DNS servers.

access-list DMZ1_access_in extended deny udp object HQDNS2 host 63.170.xxx.xxx eq domain

access-list DMZ1_access_in extended deny udp object DRDNS1 host 63.170.xxx.xxx eq domain

I hope this helps anyone else that runs into this problem.