pat xlate bug?

jsalminen · ‎07-16-2003

This morning I had an issue where none of my end users or servers where unable to pass from the inside to the outside of my PIX 520 firewall. I have a NAT & PAT configured on the firewall. When I show xlate I only saw two servers that had translations and they had hundreds maybe thousands of PAT translations. In the past I would only see a few (>10) translations to these two particular servers. These two servers are internal DNS servers that only resolve an internal domain and internet domains. They don't respond to internet hosts.

To resolve the issue I cleared xlate and immediately all systems were able to pass thru.

My system is a Pix 520 with 6.1.4. This has never happened before and we haven't changed a single line of config in months.

TIA

gfullage · ‎07-16-2003

Difficult to say without seeing the "sho xlate" output, but it sounds like the servers were sending out a boatload of packets through the PIX to various hosts, each packet created a translation and eventually all available translations were used up, stopping any legitimate users from going out.

Again at this point it's impossible to say what caused it without seeing some of the output, if you happened to save it then please post a portion of the output so we can check it out.

tomborst · ‎07-25-2003

I have the same problem, the internal DNS servers are sending out massive UDP packets to a select number of external hosts. Why they are doing this, I have no idea. The following is an sample of the show conn.

CiscoPix515E-01(config)# SHOW CONN LOCAL 172.16.238.150

4687 in use, 11868 most used

UDP out 209.73.164.7:15442 in 172.16.238.150:15442 idle 0:01:20 flags D

UDP out 64.80.255.250:14553 in 172.16.238.150:14553 idle 0:01:54 flags D

UDP out 152.163.159.232:13353 in 172.16.238.150:13353 idle 0:01:23 flags D

UDP out 66.163.169.170:12644 in 172.16.238.150:12644 idle 0:00:23 flags D

UDP out 64.80.255.251:3053 in 172.16.238.150:3053 idle 0:01:39 flags D

UDP out 152.163.159.232:14784 in 172.16.238.150:14784 idle 0:00:15 flags D

UDP out 198.41.0.4:16073 in 172.16.238.150:16073 idle 0:00:53 flags D

UDP out 202.12.27.33:14742 in 172.16.238.150:14742 idle 0:00:12 flags D

UDP out 198.41.0.10:13746 in 172.16.238.150:13746 idle 0:00:57 flags D

UDP out 198.6.1.162:14226 in 172.16.238.150:14226 idle 0:00:53 flags D

UDP out 192.58.128.30:13796 in 172.16.238.150:13796 idle 0:01:01 flags D

The 172.16.238.150 is the internal DNS server. The UDP ports are always random (I was expecting UDP-53). The max number of connections showed up as over 11,000, typical number of connections is probably less than 200.

Any idea what would cause a pair of DNS servers to send out 11000 UDP packets to external addresses (mostly to other DNS servers)???

jsalminen · ‎07-25-2003

What is your PIX IOS? Mine is 6.1.4. I searched the bug toolkit, but I was unable to associate any of the known bugs to this specific problem. For a moment I thought it might have been related to the Cert advisory "CERT Advisory CA-2003-17 Exploit available for for the Cisco IOS Interface Blocked Vulnerabilities", but this was Cisco IOS only. Besides it seemed the only two entries (DNS servers) were able to get through. My PIX license is unlimited inside users and the maximum translation (PAT) slots are in the 60,000 range and these two servers couldn't possible have consumed all 60k slots.

I have left this to a unreported or very rare bug in the PIX IOS.

Post any new information if you find some. Thanks!

tomborst · ‎07-26-2003

The PIX is running version 6.2.2, it's a 515E. I'm not even sure it's a bug on the PIX, it almost appears to be some sort of virus that triggers these massive amounts of UDP sessions to be generated by the DNS servers. Both of these servers have the most recent Mcafee DAT version. This is happening almost every day now. The only cure is the clear xlate, but that's only temporary.