ASA Interface Hang Fixed by Clearing ARP?

[Jay Wilson]

Hi folks...

Haven't posted in a while so I hope this is the right place. I bounced this off of the TAC guys and they said this should work....alas it does not. :(

We have two ASA failover pairs. One pair handles inbound/outbound web traffic and we are trying to terminate Site to Site VPN traffic on the other pair. Internal traffic is routed statically from a layer 3 switch on the inside interfaces. They are configured as shown.

When we enable the outside interface on the VPN pair, the web traffic pair eventually loses connectivity thru its outside interface. We can correct the problem by shutting down the outside interface of the VPN pair and doing a clear arp on the web traffic pair. Connectivity is never lost to either pair on the inside interfaces.

TAC said there should not be anything that would prevent this from working.

Obviously this is a problem with ARP because the clear arp command corrects it but I cannot figure out the cause. Is it possible this is some kind of Proxy ARP issue?

Thanks in advance!!

[Jay Wilson]

So after some research we decided to set this up in a test-ish environment. With proxy-arp disabled on all interfaces in the ASA VPN pair (using the sysopt noproxyarp <interface>) everything is stable. Proxy-arp is still enabled on the ASA web traffic pair. But unless I can find a good reason to keep it, that will change soon as well. ;)

We will do some more testing and post results.

How about some cisco gurus giving or pointing to a quick run thru on what the ASA does with proxy-arp and arp?

It is my understanding that with proxy-arp enabled (it is by default) a totally legit NAT statement can cause the ASA to arp for inside addresses on the outside interface. I guess if your inside address structure has routable addresses, arp-ing for them on the outside interface could be desirable. But wouldn't it be better with routing in place and proxy-arp turned off?

Is proxy-arp required for the ASA to arp for additional addresses in the same subnet as the outside interface when using them for NAT? - (i.e. 1.1.1.1 on the outside interface with NAT statements using 1.1.1.2-1.1.1.8 for translation to/from a private subnet on the inside) Or is that just plain old vanilla arp to the ASA?

Sure would be good to have a clear understanding of proxy-arp, arp and the ASA.

[Jay Wilson]

We are able to duplicate this issue in our test environment.

When proxy-arp is enabled on the active interfaces of both ASA pairs in the diagram the outside interface of the web traffic pair will eventually crash. The crashed interface can be temporarily fixed by issuing the clear arp command on the web traffic pair. However it will keep crashing.

When proxy-arp is disabled (all interfaces) on the VPN pair both ASA pairs are stable. 

I have asked TAC for clarification. Is this expected behavior or a bug in the code??

The VPN pair is running 8.4(7)30 and the web traffic pair is running 9.3(3)7.

[Jay Wilson]

From Cisco TAC...

"When no sysopt noproxyarp outside is enabled on both the ASA pairs (ASA Web traffic, ASA VPN) , then they do a proxy arp for same inside subnet 10.10.10.0/24(directly connected on inside), thus mapping there outside interface mac address to the destination IP on the inside subnet. Reason for ARP request being sent to both the outside interfaces is that they belong to same subnet as well (ARP request is broadcast message).   Conflict in IP to MAC mapping (one IP address mapped to 2 different MAC addresses), leads to drop in the connectivity . When we disable the proxy arp functionality on ASA VPN outside interface , that resolves the conflict and hence outside interface connectivity stays stable. This is an expected behaviour in the given scenario"

That nails it down. Thanks!