Pix: Going crazy with proxy arp (at least I think I shoud blame on it)

[m.laporta]

Hi Experts.

Last saturday I attempted to replace an old Pix 520 OS 5.1(5) with 2 brand-new Pix 515E OS 6.3(3) in standard failover configuration.

The Customer has many public servers on the inside and a few servers on the DMZ as well. Additionally, the Customer does not have an internal DNS server. For this reason, there are many statemtents like these:

static (inside,outside) <publ> <priv>

alias (inside) <priv> <publ> 255.255.255.255

To access servers on the DMZ, the alias command is used both for DNAT (inside->dmz) and for DNS Doctoring (dmz->dmz).

While migrating from the old Pix 520 to the new Pix's 515E, I simply changed conduits to access-lists. However, the Customer started to observe lack of communications between servers on the inside interface.

I did never come across this before... uhmmm ...

I assumed that L3 was properly working and started to blame upon L1 not functioning properly, but that revealed clearly wrong as when we switched the old firewall back on everything was ok again... Oh dear!

Hurt in my pride, I went back home and started to look at the Cisco documentation... and I discovered that I failed to enter the "sysopt noproxy inside" command. However, I had never entered the command in the Pix 520 and had never used it before, even if using the alias command.

I'm going to enter that command on my next try (tomorrow), however, can anybody tell me:

1. is the command going to help?

2. if answer is "yes", how can the Pix influence the arp process in a LAN, given that (to my knowledge) each host on the LAN has a default gateway configured (which is not the Pix btw)?

3. if answer is "no", do you have any suggestion about how to solve my issues?

Thank you in advance for your help!

Michele

[jmia]

Michele,

Here's an expalnation form the docs:

sysopt noproxyarp

ARP (Address Resolution Protocol) is a layer two protocol that resolves an IP address to a physical address, also called a Media Access Controller (MAC) address. A host sends an ARP request asking "Who is this IP?" The device owning the IP should reply with "Hey, I am the one, here's my MAC address."

Proxy ARP refers to a gateway device, in this case, the firewall, "impersonating" an IP address and returning its own MAC address to answer an ARP request for another device.

The firewall builds a table from responses to ARP requests to map physical addresses to IP addresses. A periodic ARP function is enabled in the default configuration. The presence of entries in the ARP cache indicates that the firewall has network connectivity. The show arp command lists the entries in the ARP table. Usually, administrators do not need to manually manipulate ARP entries on the firewall. This is done only when troubleshooting or solving network connectivity problems.

The arp command is used to add a permanent entry for host on a network. If one host is exchanged for another host with the same IP address then the "clear arp" command can be used to clear the ARP cache on the PIX. Alternatively, you can wait for the duration specified with the arp timeout command to expire and the ARP table rebuilds itself automatically with the new host information.

The sysopt noproxyarp command is used to disable Proxy ARPs on an interface from the command-line interface. By default, the PIX Firewall responds to ARP requests directed at the PIX Firewall's interface IP addresses as well as to ARP requests for any static or global address defined on the PIX Firewall interface (which are proxy ARP requests).

The sysopt noproxyarp if_name command lets you disable proxy ARP request responses on a PIX Firewall interface. However, this command does not disable (non-proxy) ARP requests on the PIX Firewall interface itself. Consequently, if you use the sysopt noproxyarp if_name command, the PIX Firewall no longer responds to ARP requests for the addresses in the static, global, and nat 0 commands for that interface but does respond to ARP requests for its interface IP addresses.

To disable Proxy ARPs on the inside interface:

sysopt noproxyarp inside

To enable Proxy ARPs on the inside interface:

no sysopt noproxyarp inside

Hope the helps,

Jay

[m.laporta]

Hi Jay.

I too had read the info you reported. Anyway, thank you for providing it.

However, it is not clear to me what symptoms the "no sysopt noproxyarp inside" addresses and I'm not able compare them with the symptoms I observe. So it's difficult to me to evaluate whether disabling proxy arp is going to be the solution.

In my understanding, regardless of the proxy arp setting or any aliases configured, the PIX should not interfere with the ARP process between two hosts on the inside network, correct? Additionally, if the hosts have a default gateway (which is not the Pix in my case), the proxy arp feature should be never invoked, correct?

If I am correct, the proxy arp setting should not change the behaviour and the problem will still be there...

Any more hints?

Thank you

Michele

[jmia]

Michele,

If want "DNS Doctoring" to work correctly you need to apply the "no sysopt noproxyarp inside" command, here is a document that might be of interest to you:

http://www.cisco.com/warp/public/110/alias.html

Let me know if this helps or require furthre explanation.

Jay

[sachin]

First of all I would like to correct your statmenet i.e. "ARP Process betwenn two host on the inside network". ARP protocol is broadcast in nature.So nothing is between two host only. Those packets will go to each and every machine in that Subnet.

I have seen this behaviour what I found is -- you will get Right MAC address for that server but being PIX is replying for those servers, those MAC addresses are overwritten by PIX MAC address.

I cleared ARP cache on one of the host and tried to ping to server . I always got one packet response, rest failed.So if you check your system arp table while pinging to server,for fraction of seconds you will see correct ARP entry then it will be over written by PIX MAC address.

By default, the PIX Firewall responds to ARP requests directed at the PIX Firewall's interface IP addresses as well as to ARP requests for any static or global address defined on the PIX Firewall interface (which are proxy ARP requests).

So probably you must have static enteries for your servers so that PIX is replying for those queries.

Hope this clears,

Sachin

[garethhinton]

I would take a look at outbound NAT rather than the alias command. If I remember correctly, the alias command will be phased out at some point in the future, you replace it with similar static commands to what you would use from outside to inside.

It would be better for you to read up on it from the command reference/configuration guide. No explanation I could fit on here would quite explain it as well.

One major reason I say this - If next week your customer says "I'd like the GUI enabled (PDM)" it would seem simple enough with just two commands.

However, PDM does not support the alias command, so you would be back on site to change to outbound NAT.

The config guide I think explains it fairly well, but if you're stuck, come back here.

Answering your question "Additionally, if the hosts have a default gateway (which is not the Pix in my case), the proxy arp feature should be never invoked, correct?"

The Pix can mess things up if proxy arp is on where not needed (regularly on the inside interface). If it receives an arp request (broadcast) for a local address, it will answer for it and it's a lottery whether the pix, or the device itself answer first. The PC doesn't care about gateway for local subnet devices.

[m.laporta]

Sachin and Gareth,

thank you very much for your answers. What I really needed was field experience (exactly what you reported), as I can find manuals and config tips myself.

More or less, I had the suspect that the Pix was messing up ARP tables, but didn't have any proof and the Customer didn't give me enough time for troubleshooting.

When I went to the Customer's site on my 2nd attempt, I tried to disable proxy arp but nothing seemed to change. But as soon as I replaced the "static"+"alias" CLI with the "static dns" CLI everything started to work.

In conclusion, the "static dns" solved my issue.

Thank you again for sharing your experience with me.

Cheers

michele