Cisco 2901 - Getting log message with Source and Destination port of 0

[GW M]

Hi have a Cisco 2901 and last night I started to get log message over and over again with the source being the router's Internet interface with a source port of 0. This continues over and over again with different destinations every 5 secs. How do I determine where this is coming from? Also, why is this using port 0?

025099: Dec 14 08:17:13.795 PCTime: %SEC-6-IPACCESSLOGP: list 100 denied udp 74.74.64.220(0) -> 91.226.236.222(0), 1 packet
025100: Dec 14 08:17:15.151 PCTime: %SEC-6-IPACCESSLOGP: list 100 denied udp 74.74.64.220(0) -> 42.177.204.175(0), 1 packet
025101: Dec 14 08:17:16.275 PCTime: %SEC-6-IPACCESSLOGP: list 100 denied udp 74.74.64.220(0) -> 78.142.19.231(0), 1 packet
025102: Dec 14 08:17:17.779 PCTime: %SEC-6-IPACCESSLOGP: list 100 denied udp 74.74.64.220(0) -> 78.142.19.216(0), 1 packet
025103: Dec 14 08:17:18.783 PCTime: %SEC-6-IPACCESSLOGP: list 100 denied udp 74.74.64.220(0) -> 178.162.199.244(0), 1 packet
025104: Dec 14 08:17:20.007 PCTime: %SEC-6-IPACCESSLOGP: list 100 denied udp 74.74.64.220(0) -> 36.10.18.196(0), 1 packet
025105: Dec 14 08:17:21.747 PCTime: %SEC-6-IPACCESSLOGP: list 100 denied udp 74.74.64.220(0) -> 120.88.58.173(0), 1 packet
025106: Dec 14 08:17:24.403 PCTime: %SEC-6-IPACCESSLOGP: list 100 denied udp 74.74.64.220(0) -> 93.198.187.153(0), 1 packet
025107: Dec 14 08:17:25.431 PCTime: %SEC-6-IPACCESSLOGP: list 100 denied udp 74.74.64.220(0) -> 107.188.64.210(0), 1 packet
025108: Dec 14 08:17:27.315 PCTime: %SEC-6-IPACCESSLOGP: list 100 denied udp 74.74.64.220(0) -> 100.214.127.59(0), 1 packet
025109: Dec 14 08:17:28.783 PCTime: %SEC-6-IPACCESSLOGP: list 100 denied udp 74.74.64.220(0) -> 44.197.139.134(0), 1 packet
025110: Dec 14 08:17:30.023 PCTime: %SEC-6-IPACCESSLOGP: list 100 denied udp 74.74.64.220(0) -> 82.187.212.23(0), 1 packet

Here is a section of the configuraton

ip nat inside source route-map NAT_MAP_1 interface GigabitEthernet0/0 overload

access-list 100 deny ip 192.168.1.0 0.0.0.255 host 192.168.10.1
access-list 100 deny ip 192.168.1.0 0.0.0.255 host 192.168.10.2
access-list 100 deny ip 192.168.2.0 0.0.0.255 host 192.168.10.1
access-list 100 deny ip 192.168.2.0 0.0.0.255 host 192.168.10.2
access-list 100 deny ip 192.168.3.0 0.0.0.255 host 192.168.10.1
access-list 100 deny ip 192.168.3.0 0.0.0.255 host 192.168.10.2
access-list 100 deny ip 192.168.4.0 0.0.0.255 host 192.168.10.1
access-list 100 deny ip 192.168.4.0 0.0.0.255 host 192.168.10.2
access-list 100 permit ip 192.168.1.0 0.0.0.255 any
access-list 100 permit ip 192.168.2.0 0.0.0.255 any
access-list 100 permit ip 192.168.3.0 0.0.0.255 any
access-list 100 permit ip 192.168.4.0 0.0.0.255 any
access-list 100 deny ip any any log

route-map NAT_MAP_1 permit 1
match ip address 100

[Richard Burts]

I have seen this symptom before and it is easy to explain. When you include the log parameter in the access list the logging process will have fields for source and destination address and source and destination port. If the line in the access list is examining protocol ports (for example deny tcp any any eq 23 - to deny telnet) then the log message produced will have correct port numbers since IOS was examining ports. But if the access list is denying based just on IP address (as this access list is doing) then IOS does not know what the port numbers are and will put zero into the report for port numbers.

HTH

Rick

[GW M]

Is there anyway to stop it? It just started to happen. Also, why would the source be the router itself going to internet?

Thanks

GM

[Richard Burts]

GM

You have not provided enough information for us to be able to give helpful answers about why it just started or about what is the basic cause of this behavior. You could certainly stop seeing the log messages by removing the log parameter from the deny ip any any in your access list.

But stopping the underlying behavior will require a better understanding of what this traffic is. As a first step in the process of understanding I would suggest that you insert this line and make sure that it is inserted ahead of the deny ip any any log

access-list 100 deny udp any any log

This will give us some insight into what this traffic is.

It would also be good to know if there have been config changes or changes in the topology recently.

HTH

Rick

[GW M]

The Cisco 2901 uses this ACL to NAT local networks (192.168.0.1-4/24) attached to the 2901. In addition, it denies 192.168.0.1-4/24  to 192.168.10.0.1-2 are for allowing IPsec clients to the router. Other than that, everything should be NATed by the Cisco 2901 using 74.74.64.220 to the Internet for back-end networks.

There have been no changes to the configuration topology. This started last night at 22:00.

I applied the ACL "access-list 100 deny udp any any log" that you recommended and the source and destinations ports are still showing 0

GM

[Richard Burts]

GM

Please post the output of show access-list 100

HTH

Rick

[GW M]

Extended IP access list 100
10 deny ip 192.168.1.0 0.0.0.255 host 192.168.10.1
20 deny ip 192.168.1.0 0.0.0.255 host 192.168.10.2 (1291 matches)
30 deny ip 192.168.2.0 0.0.0.255 host 192.168.10.1
40 deny ip 192.168.2.0 0.0.0.255 host 192.168.10.2
50 deny ip 192.168.3.0 0.0.0.255 host 192.168.10.1
60 deny ip 192.168.3.0 0.0.0.255 host 192.168.10.2 (3545 matches)
70 deny ip 192.168.4.0 0.0.0.255 host 192.168.10.1
80 deny ip 192.168.4.0 0.0.0.255 host 192.168.10.2
90 permit ip 192.168.1.0 0.0.0.255 any (212 matches)
100 permit ip 192.168.2.0 0.0.0.255 any (5222 matches)
110 permit ip 192.168.3.0 0.0.0.255 any (3236 matches)
120 permit ip 192.168.4.0 0.0.0.255 any (4 matches)
121 deny udp any any log (26195 matches)
130 deny ip any any log (114796 matches)

[Richard Burts]

GM

Thank you for posting this output. Let me revise my suggestion. Replace this

access-list 100 deny udp any any log

with this

access-list 100 deny udp any any range 1 65535log

This will make IOS examine the port numbers which should then be able to put the port numbers into the log message. Please make sure that it goes about the deny ip any any log

HTH

Rick

[GW M]

It looks like the router is trying to go to a DNS server that is defined in the config, which is being denied by the access-list 100.

ip name-server 24.92.226.11
ip name-server 24.92.226.12

PCTime: %SEC-6-IPACCESSLOGP: list 100 denied udp 74.74.64.220 (61828) -> 24.92.226.11(53), 1 packet
PCTime: %SEC-6-IPACCESSLOGP: list 100 denied udp 74.74.64.220 (63961) -> 24.92.226.12(53), 1 packet
Dec 14 11:41:16.491 PCTime: %SEC-6-IPACCESSLOGP: list 100 denied udp 74.74.64.220 (49406) -> 24.92.226.12(53), 1 packet

Can't I source that from the router's internal interface and then it pass the ACL?

GM

[Richard Burts]

GM

While it is possible to specify source interfaces for certain protocol on an IOS router I am not aware of a way to specify the source address for DNS packets. I am a bit puzzled why this would be needed. I would have thought that the DNS request by default would be sourced from the router outside interface address and would have thought that this would not need to be translated. Other than these log messages (which mean that the packets will not be translated but does not mean that these packets are being denied going through the interface) do there seem to be issues with DNS on the router? If DNS is being impacted then perhaps we need to think about putting a permit into access list 100 for the router DNS traffic.

I wonder if there are some other parts of the router config that we may need to understand. For example the fact that you are using a route map to control Network Address Translation suggests that this router config is a bit more complex than most.

The other thing that puzzles me is that your original post shows a variety of destination addresses that do not match up with your DNS server configuration.  So I wonder if things have changed a bit since those log messages were generated?

HTH

Rick

[GW M]

Agree. Why would the router query DNS over and over again? Is there anything else that I could do to debug the issue?

GM

[Richard Burts]

GM

Among the many things that you have not told us about this router is whether the router is configured to act as a DNS server for any devices within your network. If it is configured to act as a DNS server then the fact that it query upstream DNS over and over is to be expected. If it is not configured as DNS server then we may have to look a bit harder to find why it is sending so much DNS query.

HTH

Rick

[GW M]

The router is configured as the DNS server for internal networks. Those devices on the internal network are resolving hostnames just fine. I don't see any problems besides this UDP port 53 deny all the time. It seems like DNS wouldn't work if it was denying but it is working.

I changed the DNS IP's in the router and it did change in the log and the deny continues like before.

no ip dns lookup

ip dhcp pool office-pc-pool
import all
network 192.168.2.0 255.255.255.0
default-router 192.168.2.1
dns-server 192.168.2.1
!
ip dhcp pool wifi-pool
import all
network 192.168.3.0 255.255.255.0
default-router 192.168.3.1
dns-server 192.168.3.1

ip name-server 209.18.47.61
ip name-server 209.18.47.62

GM

[Richard Burts]

GM

Thanks for the clarification. If the router is configured as the DNS server for the inside networks then that certainly explains why there is so much DNS traffic from the router outbound.

It is interesting to know that DNS continues to work ok for the devices inside. That reinforces the comment in my previous post that a deny in access list 100 does not mean that traffic is denied as it goes out the interface but only is denied from being translated.

So what we have (so far) appears to be a cosmetic problem of entries in the log about denied traffic in access list 100. There are a couple of options of what to do about the log entries for the DNS traffic.

- you could re-evaluate whether the log parameter on the deny ip any any is really helpful. If you take the log parameter off of that line then all the behaviors of the router forwarding traffic remain the same but you no longer see log entries about it.

- you could put a statement into access list 100 that deals with the DNS traffic sourced from the router. You could deny it (which I might prefer) or you could permit it. The main thing is to do something with that traffic before it gets to deny ip any any log.

- you could decide that now that you know what it is that you can just live with the log entries. I personally do not like this option and prefer to have log entries only for things that I would potentially take action about.

And I am still a bit concerned about the log entries in your original post that have IP addresses that do not match up with your DNS server config.

HTH

Rick

[GW M]

My original config did have ip name-server 24.92.226.11 and ip name-server 24.92.226.12. I changed them to ip name-server 209.18.47.61
and ip name-server 209.18.47.62 to see what would happen. It produced the same result.

As far as the logs prior to adding the access-list 100 deny udp any any range 1 65535 log

Dec 14 08:17:13.795 PCTime: %SEC-6-IPACCESSLOGP: list 100 denied udp 74.74.64.220(0) -> 91.226.236.222(0), 1 packet
025100: Dec 14 08:17:15.151 PCTime: %SEC-6-IPACCESSLOGP: list 100 denied udp 74.74.64.220(0) -> 42.177.204.175(0), 1 packet

and after adding access-list 100 deny udp any any range 1 65535 log, it's a bit strange

PCTime: %SEC-6-IPACCESSLOGP: list 100 denied udp 74.74.64.220 (61828) -> 24.92.226.11(53), 1 packet
PCTime: %SEC-6-IPACCESSLOGP: list 100 denied udp 74.74.64.220 (63961) -> 24.92.226.12(53), 1 packet
Dec 14 11:41:16.491 PCTime: %SEC-6-IPACCESSLOGP: list 100 denied udp 74.74.64.220 (49406) -> 24.92.226.12(53), 1 packet

I removed the log command on the access-list ip deny any any, which eliminated the logs messages but not sure why the problem started last night and I didn't have the problem before.

If the DNS queries where coming from the clients on the internal network they would be NATed and it would never hit the deny ip any any ACL. It almost seems like the router is initiating the DNS queries, which then hits the deny ACL. To bad I couldn't source the router DNS query from an internal interface.

GM