puzzle: when the ISP refuses to use arp requests...

[busitech]

I have been having trouble doing something that should be incredibly simple, and now that it has proven difficult, it's a puzzle waiting to be solved.

The ISP has given us a few static addresses to use.  We have a Cisco 2800 series router / firewall device.  I'd like to assign the first address to the router, and use the rest with NAT / PAT.

With a routed subnet, this would be easy.  However, the catch is that the static addresses are within one huge /23 subnet.

Any of the addresses work fine when assigned to the router as the primary address.  The other addresses do not work when configured as a secondary address on the same interface as the primary, or with a NAT mapping, unless the second address has been recently used as the primary address of the router, or a physical host in the DMZ.  If the address is removed as the primary address, and added to the NAT / PAT configuration we want, an "activated" address will work for many hours and then die, usually within 24 hours.

I decided to log arp packets on the outside interface for a few days.  I discovered that the default gateway on the ISP side will never send an arp request for an address that has died off, even though I'm sitting on the other side of the 'net pinging the address.  It seems obvious to me that the arp cache has expired on their end, and neither the router nor the ISP is announcing or requesting via arp.

After having had (way too many unproductive) conversions with the ISP, they continually refuse to make any changes on their end.  I suggested a static arp entry for our mac address, or to start sending arp requests.  I also can't get enough visibility into their configuration.

Does anyone know how to bridge arp announcements (gratuitous arp) from inside to outside, or how to get the 2800 IOS to generate gratuitous arp for the addresses that are used in NAT?

[Jason M.]

If using 1:1 NAT could he try having the servers ping the WAN gateway IP and get a similar result? Although not an arp, it would be a packet destined for the gateway router with the source address being the WAN IP for each of the servers.

[busitech]

I have tried doing a ping with extended commands, specifying the secondary ip address as the source address, and the pings time out unfortunately.

[Jason M.]

Did you check to see if your router has unique MAC addresses for each sub-interface or allows the mac address to be manually set? If so you may be able to create a sub-interface for each address and use that to NAT.

[busitech]

I went into a sub-interface and did not find a mac-address command available.

If I enable transparent bridging and routing, I can create bridges with separate mac addresses, but to attach a sub-interface to a bridge group, it asks:

% Configuring IP routing on a LAN subinterface is only allowed if that 
subinterface is already configured as part of an IEEE 802.10, IEEE 802.1Q, 
or ISL vLAN.

I don't have remote access to the switch to set up a trunking port, so I haven't tried to set up multiple VLANs.  I also anticipate there might be a problem if I try to assign addresses on the same subnet to two different sub-interfaces.

[busitech]

I am going to mark Milan's answer as Correct, as this does work to generate GARP, which was my original question.  Thanks Milan!

[Jason M.]

Hello busitech,

I have seen this problem happen on a GEPON network where the OLT code prevented acceptance of ARP replies from MAC addresses that were already in the ARP table limiting them to 1:1 associations. I have also seen this problem on a CMTS. In both cases the problem was acknowledged by the equipment vendor and the interim solution was to set up static ARPs.

In the CMTS scenario the extra ARPs would remain if the CMTS continued to receive traffic from the IP/MAC's in question (which would happen as the primary address like you mentioned). I don't think we got that far on the GEPON situation to say if that would have affected anything. It was a long time ago and I don't have my notes with me.

Are you doing static 1:1 NAT for all the addresses? Are these primarily output connection or are you running servers?

Not sure how viable it would be with that device but I know some routers allow MAC manipulation on sub-interfaces and can support NAT on them as well.

[busitech]

Thanks for the information!  This ISP is definitely of the GEPON / OLT type, and in this case, our MAC address is already in the ARP table, and would be trying to be entered multiple times.

If I cycle through all of the addresses one by one, assigning them to the primary address of the router interface, I can get them all working simultaneously.  Then, over time, all but one die out and stop responding.

We have servers behind the addresses, and prefer to use 1:1 NAT.

[Jason M.]

Looking at my notes, the following two options were suggested for troubleshooting for the Aroura units. My notes also said there were some problems after the static ARPs were added but just not as much.

Haven't had a problem with it since the code was upgraded back in 2013.

Option 1: The OLT firmware of 0.4.84 has a known bug that may be causing the problem. He recommends that we update the firmware to version 0.5.23.

Options 2: Under CPE Admissions Control can we can change the CPE admission controls from IP Address Pool based to MAC Address Based.

Is the sub-interface option feasible?