ARP cache timeout on Cisco routers

[Sam Preston]

Hello,

I was reading a book on Cisco routers in which the author says : "The router resets the ARP age counter to zero whenever it sees valid traffic from the corresponding device. This ensures that the addresses of active devices are never flushed out of the cache, no matter how long they have been known."

I am really surprised about that because I have always thought that the ARP age counter was an absolute counter and not relative to the last time a packet was seen coming from the corresponding IP. After reading this, I made some tests which tend to confirm that the ARP age counter is absolute and does not care whether we have active traffic from the corresponding IP or not.

QUESTION 1 : can somebody confirm this please ?

I am unable to find clear assertions in Cisco documentation.

QUESTION 2 : when does the router send a new ARP request ?

For example, when the ARP timeout is 4 hours or 240 minutes (Cisco default value), the router sends an ARP request when reaching 239 minutes (1 minute before the expiration time). Is this value a fixed one (we send an ARP request 1 minute before aging) or is it a relative value (x % of the timeout value) ?

Thanks for your help.

[MATTHIAS SCHAERER]

It is as you read in the book: tha ARP counter is reset to 0 when traffic is seen from/to the device. The new ARP request would be sent after 4 hours (240) minutes or after a 'clear arp' command.

[Richard Burts]

I believe that Sam may be referring to a section in the IOS Cookbook which does say that the router resets the ARP timer when it sees traffic from a host. I do not agree with Matthias that this is correct. My experience with Cisco routers is that the ARP timer counts down even if it is receiving traffic from a host.

I know that Cisco routers will send an ARP request for an entry in the ARP table before that entry actually expires. I do not have anything right now that tells us how that interval is determined (fixed value or percentage). And I do not know of any command that would allow us to change the interval.

HTH

Rick

[dpavun]

Hi Richard,

Why the router should send unicast ARP request before it is going to expire? If it sends, definitely it will get the reply unless the destination device is down. So, irrespective of traffic from that destination, the ARP entry for that destination will remain in the ARP cache. If there is any specific reason, please help me to understand this and Is there any cisco document reference now which says about this ARP request time interval before it actually expires?

Thanks,

Dhamodiran

[Richard Burts]

Dhamodiran

I am not clear what you are asking. Are you asking why Cisco would send an ARP request before the expiration of the existing ARP entry, or are you asking why Cisco sends it as unicast?

From the context of the question it seems that perhaps you are asking why Cisco would send the request before the ARP entry expires. The most important reason can be understood if you think about what would happen if Cisco did not send the request before the entry expires. When the existing ARP entry expires the Cisco would wait for the next packet that it needs to forward to that host. When that packet arrives Cisco discovers that it does not have an ARP entry so it sends an ARP request. And it drops the packet that it would have forwarded. So it would have a negative impact on network availability if Cisco lets the existing entry expire and waits for the next packet before it sends the ARP request.

Sending the ARP request allows Cisco to verify that the host is still active in the network and to maintain consistent of forwarding of traffic to the host. And sending the request as unicast allows Cisco to check on the host without impacting all other hosts in that subnet.

HTH

Rick 

[paul driver]

Hello Rick

This is an interesting post - after reading it i did some testing and , it shows an unicast arp refresh is initiated around 50% of the specified arp timeout value.

R1#ping 2.2.2.2

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 2.2.2.2, timeout is 2 seconds:

*Mar  1 00:36:10.539: IP ARP: creating incomplete entry for IP address: 2.2.2.2 interface FastEthernet0/0
*Mar  1 00:36:10.539: IP ARP: sent req src 10.1.12.1 c201.5c7c.0000,
                 dst 2.2.2.2 0000.0000.0000 FastEthernet0/0
*Mar  1 00:36:10.563: IP ARP: rcvd rep src 2.2.2.2 c202.628c.0000, dst 10.1.12.1 FastEthernet0/0.!!!!
Success rate is 80 percent (4/5), round-trip min/avg/max = 24/270/1004 ms

*Mar  1 00:36:12.567: IP ARP: rcvd req src 10.1.12.2 c202.628c.0000, dst 10.1.12.1 FastEthernet0/0
*Mar  1 00:36:12.571: IP ARP: creating entry for IP address: 10.1.12.2, hw: c202.628c.0000
*Mar  1 00:36:12.571: IP ARP: sent rep src 10.1.12.1 c201.5c7c.0000,
                 dst 10.1.12.2 c202.628c.0000 FastEthernet0/0

R1(config-if)#arp timeout 240

R1(config-if)#int fa0/0
R1(config-if)#arp timeout 240

*

R1#clear arp
R1#
*Mar  1 00:39:12.759: ARP: flushing ARP entries for all interfaces
*Mar  1 00:39:12.767: IP ARP: sent rep src 10.1.12.1 c201.5c7c.0000,
                 dst 10.1.12.1 ffff.ffff.ffff FastEthernet0/0
*Mar  1 00:39:12.771: IP ARP: sent rep src 10.1.13.1 c201.5c7c.0001,
                 dst 10.1.13.1 ffff.ffff.ffff FastEthernet0/1
*Mar  1 00:39:12.775: IP ARP: sent req src 10.1.12.1 c201.5c7c.0000,
                 dst 2.2.2.2 c202.628c.0000 FastEthernet0/0
*Mar  1 00:39:12.779: IP ARP: sent req src 10.1.12.1 c201.5c7c.0000,
                 dst 10.1.12.2 c202.628c.0000 FastEthernet0/0
R1#
*Mar  1 00:39:12.799: IP ARP: rcvd rep src 2.2.2.2 c202.628c.0000, dst 10.1.12.1 FastEthernet0/0
*Mar  1 00:39:12.815: IP ARP: rcvd rep src 10.1.12.2 c202.628c.0000, dst 10.1.12.1 FastEthernet0/0

tick tock..... tick tock........tick tock..... tick tock

*Mar  1 00:41:19.723: IP ARP: sent req src 10.1.12.1 c201.5c7c.0000,
                 dst 2.2.2.2 c202.628c.0000 FastEthernet0/0
*Mar  1 00:41:19.735: IP ARP: rcvd rep src 2.2.2.2 c202.628c.0000, dst 10.1.12.1 FastEthernet0/0

*Mar  1 00:41:33.035: IP ARP: sent req src 10.1.12.1 c201.5c7c.0000,
                 dst 10.1.12.2 c202.628c.0000 FastEthernet0/0
*Mar  1 00:41:33.063: IP ARP: rcvd rep src 10.1.12.2 c202.628c.0000, dst 10.1.12.1 FastEthernet0/0

R1#un all
All possible debugging has been turned off

res

Paul

[Richard Burts]

Paul

Yes it is an interesting post and I am glad to see it come up again. My experience is that sending the arp request is not based on 50 % of the timeout. I believe that the results of your test are skewed by your choice of a quite short arp timeout. With a timeout of 4 minutes the request is sent at close to 2 minutes. But if you had set the timeout to an hour I am confident that the request would have come closer to 58 minutes than to 30 minutes.

A while back I had an experience that may shed some light on this question. I was working on a problem in a customer network where the timing of the arp request seemed to be part of the issue. And we were not able to get a consistent measurement of the arp request was generated.  I opened a case with Cisco TAC about it. What the TAC engineer explained to me was that the request to refresh the arp entry is generated near the end of the arp lifetime. But that Cisco adds a variable amount of time in determining when to generate the arp request. The logic in that is that they want to avoid synchronization of the arp entries. If we have flushed all the entries in the table and re-learned all the entries at approximately the same time then when they are about to expire we do not want to flush them all at the same time and relearn them again in a big batch. So the timeout is somewhat variable. But it is within some number of minutes of the timeout and not based on a percentage of the timeout.

HTH

Rick

[paul driver]

Hello Rick

interface FastEthernet0/0
 ip address 10.1.12.1 255.255.255.0
 duplex auto
 speed auto
 arp timeout 3600
end

interface FastEthernet0/1
 ip address 10.1.13.1 255.255.255.0
 duplex auto
 speed auto
 arp timeout 3600

R1#clear arp
R1#
Jul 28 07:53:10.063: ARP: flushing ARP entries for all interfaces
Jul 28 07:53:10.071: IP ARP: sent rep src 10.1.12.1 c201.5c7c.0000,
                 dst 10.1.12.1 ffff.ffff.ffff FastEthernet0/0
Jul 28 07:53:10.071: IP ARP: sent rep src 10.1.13.1 c201.5c7c.0001,
                 dst 10.1.13.1 ffff.ffff.ffff FastEthernet0/1
Jul 28 07:53:10.071: IP ARP: sent req src 10.1.13.1 c201.5c7c.0001,
                 dst 10.1.13.3 c203.55d8.0001 FastEthernet0/1
Jul 28 07:53:10.071: IP ARP: sent req src 10.1.13.1 c201.5c7c.0001,
                 dst 3.3.3.3 c203.55d8.0001 FastEthernet0/1
R1#
Jul 28 07:53:10.071: IP ARP: sent req src 10.1.12.1 c201.5c7c.0000,
                 dst 10.1.12.2 c202.628c.0000 FastEthernet0/0
Jul 28 07:53:10.071: IP ARP: sent req src 10.1.12.1 c201.5c7c.0000,
                 dst 2.2.2.2 c202.628c.0000 FastEthernet0/0
Jul 28 07:53:10.083: IP ARP: rcvd rep src 10.1.13.3 c203.55d8.0001, dst 10.1.13.1 FastEthernet0/1
Jul 28 07:53:10.087: IP ARP: rcvd rep src 10.1.12.2 c202.628c.0000, dst 10.1.12.1 FastEthernet0/0
Jul 28 07:53:10.099: IP ARP: rcvd rep src 3.3.3.3 c203.55d8.0001, dst 10.1.13.1 FastEthernet0/1
Jul 28 07:53:10.099: IP ARP: rcvd rep src 2.2.2.2 c202.628c.0000, dst 10.1.12.1 FastEthernet0/0






R1#
Jul 28 09:02:51.019: IP ARP: sent req src 10.1.13.1 c201.5c7c.0001,
                 dst 3.3.3.3 c203.55d8.0001 FastEthernet0/1
Jul 28 09:02:51.039: IP ARP: rcvd rep src 3.3.3.3 c203.55d8.0001, dst 10.1.13.1 FastEthernet0/1
R1#
Jul 28 09:03:29.931: IP ARP: sent req src 10.1.13.1 c201.5c7c.0001,
                 dst 10.1.13.3 c203.55d8.0001 FastEthernet0/1
Jul 28 09:03:29.955: IP ARP: rcvd rep src 10.1.13.3 c203.55d8.0001, dst 10.1.13.1 FastEthernet0/1
R1#
Jul 28 09:05:18.475: IP ARP: sent req src 10.1.12.1 c201.5c7c.0000,
                 dst 2.2.2.2 c202.628c.0000 FastEthernet0/0
Jul 28 09:05:18.491: IP ARP: rcvd rep src 2.2.2.2 c202.628c.0000, dst 10.1.12.1 FastEthernet0/0


res
Paul

[Richard Burts]

Paul

Thanks for modifying your test and posting the results which are quite interesting. The three addresses were cleared and new entries learned all at 07:53:10. But the requests to refresh those entries were spread from 09:02:51 to 09:05:18. And while the arp timeout was set to one hour the requests to refresh the entries were all longer than an hour. So the variability that Cisco adds can be longer than the time, while I had assumed that the variability would be shorter than the timer. And it seems that the bottom line is that the arp timeout is not a precise timer but an approximation of when the entries will be refreshed or be purged from the table.

HTH

Rick

[paul driver]

Hello Rick

Not a problem , However  both tests show something I didn't expect, I expected either one or the other but not two different results.

If I wanted to decrease the arp timeout say to negate non deterministic unicast flooding of the vlan on a switch, Cisco suggests either, increase cam table aging timeout of the switch or lower the arp timeout of the rtr that is either equal too or lower value of the switch.

My first test shows arp refresh is around half lifetime on a low specified arp timeout value , which is not even close to refresh timeout value of the second test with a much longer specified arp timeout, which shows refresh is either equal to or just above the specified value.

So to summarise to perform the cisco suggestion and inline with the above testing, the value would be as follows:


cam table aging default to 300secs
arp timeout = 480 or 600 secs ( as you see its above and not equal or lower then the cam aging value



res
paul

[Richard Burts]

Paul

I am not convinced that your two tests show two different results. I believe that at least in one way of looking at the behavior that both tests demonstrate a consistent outcome. It is a bit easier to see it in your second test. The time to send the arp request was approximately plus or minus 2 1/2 minutes of the arp timeout. If you look at your first test this same outcome would apply in that the arp request was generated within 2 1/2 minutes of your arp timeout.

HTH

Rick

[#12]

Here is a good link that explains some more about ARP timeouts.  It mentions that random jitter is added to the ARP cache timeout in order to avoid synchronous expiration of the ARP entries, which might trigger an ARP storm. Jitter should be a random number between 0 seconds and 30 minutes, with a maximum jitter of 30 minutes.  

http://www.cisco.com/c/en/us/support/docs/ip/address-resolution-protocol-arp/117398-qanda-arp-timeout-00.html

Thanks,

Harry

[Richard Burts]

Harry

This is an interesting article. Thank you for pointing it out. (and +5 for you). It is one of the few places I am aware of that mention the variable jitter that is added to the arp timeout.

HTH

Rick

[Nisterio]

Hello Harry

I am reading this discussion, and it's yummy to digest.  Here is a question I have: what happens if the ARP cache entry in the TCAM expires before the ARP timeout?

Vlan Aging Time : 600

   ip arp timeout:  1500

Let s assume that  I am running NX-OS 7.3(2) and I have VPC enable between 2 Nexus switch and HSRP configured as well/

[Sam Preston]

I agree with Richard when he says "I do not agree with Matthias that this is correct" 

I confirm that this sentence comes from the "Cisco cookbook" and my own experience also proves that this is untrue. So I don't understand why the author stated that so clearly in the aforementionned book...

I don't understand either why Cisco does no clearly indicates in its operating documentation how the ARP aging timer works on its routers. If an ARP request is issued a short time before the timer ages, why can't they write this down in the documentation ? This is an important point and a few more words just to explain that wouldn't cost a lost...