Multiple incomplete arp entries when only 1 cable plugged in

[craig.howson]

Hi all

I have a 2960XR switch which is showing multiple incomplete arp entries. As a test I unplugged all ethernet cables and plugged a single cable back in. Regardless of which cable and what kind of device (AP, Voip, Laptop) you get multiple incomplete entries with just a single cable/device.

Internet X.X.8.1 - c444.a0fa.8c42 ARPA Vlan400
Internet X.X.8.3 0 Incomplete ARPA
Internet X.X.8.4 0 Incomplete ARPA
Internet X.X.8.51 24 001a.e80b.cfef ARPA Vlan400

Internet X.X.8.78 0 Incomplete ARPA

Many searches point towards faulty cables but to me this cannot be the case as multiple entries appear when any of the cables are plugged in by themselves.

I also did a "debug arp" and you can see the incomplete entries being created, see below for one example.....

May 9 14:48:03.883: IP ARP: creating incomplete entry for IP address: X.X.8.4 interface Vlan400
May 9 14:48:03.883: IP ARP: sent req src X.X.8.1 c444.a0fa.8c42,
dst X.X.8.4 0000.0000.0000 Vlan400

Any suggestions?

Thanks in advance

[Seb Rupik]

Hi there,

From the output you have provided we can see that X.X.8.1 c444.a0fa.8c42 (the 2960) is sending an ARP request for other devices on the subnet, eg X.X.8.4

This will be caused by a connected device trying to reach a destination address which the 2960XR does not have a L3 adjacency for. It will then send out an ARP request to resolve this issue, and whilst waiting for a reply will create an Incomplete ARP cache entry.

cheers,

Seb.

[craig.howson]

That kind of makes sense, but im still unsure what I can do about it.

I have seen this in the log also....

May 10 12:15:23.052: %ADJ-5-RESOLVE_REQ_FAIL: Adj resolve request failed for X.X.8.4 on Vlan400

We only use static routing and I only associate adjacency's with dynamic. Do I have this right?  

[Seb Rupik]

The message is triggered when an adjacency cannot not be resolved via ARP.

To stop the CEF process initiating ARP resolution you could use this command:

no ip cef optimize neighbor resolution

https://www.cisco.com/c/en/us/td/docs/ios/ipswitch/command/reference/isw_book/isw_i1.html#pgfId-1084549

...but doing so has the potential to cause you forwarding issues in the future.

With regard to the term adjacency in this context, consult Cisco CEF documentation to fully understand it:

https://www.cisco.com/c/en/us/support/docs/ip/express-forwarding-cef/17812-cef-incomp.html#topic1

cheers,

Seb.

[craig.howson]

Thanks for the information you sent.

Id rather not turn off CEF ARP resolution, reading through the documentation it seems useful (I understand CEF adjacencies now :) ).

However, I have just turned it off as a test (no ip cef optimize neighbor resolution) but this does not make any difference to the incomplete ARP entries.

I ran some show adjacency ......... commands and nothing in or out showing. Also run debug cef all and still showing nothing.

[Seb Rupik]

If an adjacency cannot be determined by CEF the packet will be punted to the CPU for ARP resolution. The no ip cef optimize neighbor resolution command prevents CEF process from doing this, therefore you should stop seeing the %ADJ-5-RESOLVE_REQ_FAIL log entries.

It is not a command I have seen or used in production before so there must be some nuance with how it differs from CPU ARP resolution.

So you will still see the Incomplete ARP entries as these are being generated by the CPU ARP resolution.

With one device connected you should see at least one entry via sh adjacency .

cheers,

Seb.

[craig.howson]

Thanks for your assistance Seb :)

I do see adjacency's using the sh adjacency command. I see incomplete arp entries for the following ip addresses....

x.x.8.3

x.x.8.4

x.x.8.78

x.x.5.68

And I can see entries in the adjacency table for x.x.5.78 only.

This does not seem to be causing any performance issues but I just need to understand why it is happening and if i can, stop it.

Thanks

[Seb Rupik]

The only way you will "fix" this issue is to stop x.x.5.78 trying to send packets to:

x.x.8.3

x.x.8.4

x.x.8.78

x.x.5.68

...or you could connect the devices which are assigned those IP addresses which will resolve the Incomplete ARP cache entries.

cheers,

Seb.

[Richard Burts]

As Seb has pointed out the incomplete entries are created when  the switch sends an arp request for an IP address. If the switch receives a response then a normal arp entry is created. If there is no response to the arp request then the incomplete entry remains in the table for a brief time and is then removed. It does not matter how many cables are connected. If the switch sends an arp request and does not receive a response then there will be incomplete entries in the table. If you regard this as something that should be fixed then I suggest that your first step should be to determine what devices are associated with the incomplete entries 8.3, 8.4, 8.78, 5.68, etc. Are these devices that should be active on your network? If so are they connected to the network? Or is there some reason why those devices are not responding to arp requests. The other question to ask is why is the switch sending arp requests for these addresses? My guess is that the switch is attempting to forward packets to those addresses. If that is the case then the next question is who is sending traffic to these addresses and should they be sending packets to those addresses.

My perspective is that most of the time incomplete arp entries are normal operations and not anything that needs to be fixed. If you are seeing lots of incomplete entries then it might be something to look into. And the real question there is what is sending the packets that are causing the incomplete entries. There is perhaps the possibility that someone is attempting to probe your network and that might lead to questions about how well Is your network secured.

HTH

Rick