Cisco ISR900 - DNS query issue from client machine

[orochi_yagami]

Hi, 

I've setup an ISR900 router in DC which come with two internet links. Each internet link come with it's own /29 subnet. Both the WAN interfaces are connected to these two internet link with primary outgoing point to G4 with a static default route and secondary to G5 with static default an higher AD. 

The client machine are groups of VM's behind this WAN router. The WAN router can queries internet IP and hostname with no issue. DNS servers are public DNS provided by local ISP. 

The client machine are configured with the public DNS. The issue is, whenever the client machine initiated the DNS queries (e.g.: web browsing), there isn't any query replied received. i ran the packet capture on the client machine and confirmed this. The client machine able to ping to the public internet DNS and from the traceroute the traffic are routed correctly. In other words, the the ACL and the NAT are worked fine. 

DNS tracert

ACL to allow virtual machine to internet. The [443] hit count are increase when i replaced the URL with IP address. I can also SSH any host in internet which is IP based. 

Configuration:

track 1 ip sla 1 reachability

!
ip dns server
ip nat inside source route-map NAT_INET1 interface GigabitEthernet4 overload
ip nat inside source route-map NAT_INET2 interface GigabitEthernet5 overload
ip route 0.0.0.0 0.0.0.0 XXX.XXX.XXX.XXX track 1
ip route 0.0.0.0 0.0.0.0 YYY.YYY.YYY.YYY 10
!

ip access-list extended TO-INET
permit ip 10.110.130.0 0.0.1.255 any
!
ip sla 1
icmp-echo XXX.XXX.XXX.XXX source-ip XXX.XXX.XXX.XX2
ip sla schedule 1 life forever start-time now
!
route-map NAT_INET2 permit 10
match ip address TO-INET
match interface GigabitEthernet5
!
route-map NAT_INET1 permit 10
match ip address TO-INET
match interface GigabitEthernet4
!
!

Btw, i'm running the zone-based FW for the access control. 

TIA 

[orochi_yagami]

Hi Georg,

Yes, G2 is configured as trunk port as i wanted to have multiple "INSIDE" vlans to be provisioned in the future based on different services. Since these vlans will be "INSIDE", i dont see any issue to keep the "zone-member security inside" as VM may require to access internet as well. 

The VM "virtual machine" are simply the Windows 10 machine that run different services and application.

Furthermore, not exactly sure how these recomemdation can relate to the DNS query? may you explain a little?

TIA

[Georg Pauwen]

Hello,

the Windows 10 machines are Hypervisors ?

Either way, time for some radical measures, since it takes way too long to solve an apparently small and simple problem.

I have stripped down your configuration to the bare essential (ZBF with Internet access for just Vlan 131). If this configuration does not work, the problem is with the workstations:

version 15.9
service tcp-keepalives-in
service tcp-keepalives-out
service timestamps debug datetime msec localtime show-timezone
service timestamps log datetime msec localtime show-timezone
service password-encryption
service compress-config
service sequence-numbers
!
hostname XXXXX01
!
boot-start-marker
boot system flash:c900-universalk9-mz.SPA.159-3.M2.bin
boot-end-marker
!
logging buffered 64000
!
aaa new-model
!
aaa session-id common
!
no ip source-route
!
ip cef
no ipv6 cef
!
multilink bundle-name authenticated
!
spanning-tree portfast bpduguard
vtp domain abc
vtp mode transparent
!
redundancy
!
vlan 130
name INSIDE-VPN
!
vlan 131
name INSIDE-VM-MGMT
!
vlan 132
name DMZ-VM
!
vlan 134
name MANAGEMENT
!
class-map type inspect match-any INSIDE-TO-OUTSIDE-CLASS
match protocol dns
match protocol http
match protocol https
match protocol ntp
!
policy-map type inspect INSIDE-TO-OUTSIDE-POLICY
class type inspect INSIDE-TO-OUTSIDE-CLASS
inspect
class class-default
drop log
!
zone security INSIDE
zone security OUTSIDE
zone-pair security IN-TO-OUT source INSIDE destination OUTSIDE
service-policy type inspect INSIDE-TO-OUTSIDE-POLICY
!
interface GigabitEthernet0
description |-- HP DL360P iLO --|
switchport access vlan 134
switchport mode access
no ip address
!
interface GigabitEthernet1
description |-- Hypervisor ESXi VMNIC3 --|
switchport access vlan 134
switchport mode access
no ip address
!
interface GigabitEthernet2
description |-- INSIDE-ZONE VMNIC1 --|
switchport mode trunk
no ip address
!
interface GigabitEthernet3
description |-- DMZ-ZONE VMNIC0 --|
switchport access vlan 132
switchport mode access
no ip address
!
interface GigabitEthernet4
ip address XXX.XXX.XXX.XXX 255.255.255.248
ip nat outside
ip virtual-reassembly in
zone-member security OUTSIDE
duplex auto
speed auto
!
interface GigabitEthernet5
ip address YYY.YYY.YYY.YYY 255.255.255.248
ip nat outside
ip virtual-reassembly in
duplex auto
speed auto
!
interface Vlan1
no ip address
shutdown
!
interface Vlan131
description *** INSIDE-VM-MGMT ***
ip address 10.110.131.1 255.255.255.0
no ip redirects
no ip unreachables
ip nat inside
ip virtual-reassembly in
zone-member security INSIDE
no ip route-cache
!
interface Vlan132
description *** DMZ-ZONE ***
ip address 10.110.132.1 255.255.255.0
no ip unreachables
ip virtual-reassembly in
no ip route-cache
!
interface Vlan134
description *** XXXXX01-MGMT ***
ip address 10.110.134.1 255.255.255.0
no ip redirects
no ip route-cache
!
ip forward-protocol nd
no ip http server
no ip http secure-server
!
ip nat inside source list 1 interface GigabitEthernet4 overload
ip route 0.0.0.0 0.0.0.0 GigabitEthernet4
!
access-list 1 permit 10.110.131.0 0.0.0.255
!
control-plane
!
end

[orochi_yagami]

Hi Georg,

Thanks for the suggestion. 

Let me do a network packet capture at a few hop to see what exactly is going on for the DNS traffic. 

TIA

[paul driver]

Hello

---

@orochi_yagami wrote:

Hi Paul,

I tried the suggestion above to replace the ACL matching with protocol, unfortunately there is no difference. 

---

Did you also remove the zone-pair security OUT-TO-IN ?
This policy  isn’t required do to the fact the initiated traffic is from the zone-pair security IN-TO-OUT policy as such there isn’t requirement to inspect that specific return traffic

[orochi_yagami]

Hi Paul,

Yes, i did removed the zone-pair "OUT-TO-IN". If you refer to my config attached above you may see that.

As you said, I created this zone-pair for the troubleshooting purpose as I wasn't sure if the return DNS query traffic from ISP/Internet being dropped due to the zone policy, but it's confirmed that not related to the return traffic.

TIA.