cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1540
Views
25
Helpful
7
Replies

Sudden Increase in ARP Traffic - ARP Inspection Limit Breaches

marcg1910
Level 1
Level 1

Hi all first time on this forum, looking for advice as I'm not sure how to proceed with this situation.

I have recently come across a series of ARP inspection limit breaches on ports connected to Access Points, which is of course shutting down the port temporarily which in turn brings the APs down.

All our switches at each site are WS-C2960X-24PD-L models. For about 3 years we have had this limit set to 200 packets per second, and have had no issues at all.

Now we have had quite a big change in how the clients at all sites are managed - from Windows Domain based management by on-prem servers, to cloud management with Azure and Intune. During the initial stages of this change-over there were no problems at all, all clients were running fine on Intune management and no problems on the network.

When I first noticed this happening at one of our sites, I started investigating what had recently changed to cause this surge in ARP traffic on the network. I started using packet capturing to try and isolate any particular cause - all ARP traffic looked normal and I was not seeing the volume of traffic that these ports were reporting. Mind you I was capturing the packets from the site firewall due to limitations with troubleshooting this remotely - I was not able to use something as effective as port mirroring on the switches.

The breaches I was seeing at this site were reading as high as 600 packets in 50ms. At this point I had never seen breaches for this volume of ARP traffic before.

The next couple of days I was seeing this at 3 other sites as well. To mitigate the issue from affecting the APs at each site I simply removed the limit from the port config until a solution can be found.

Dicussed this with a senior engineer on my team and found he had recently implemented 'Windows Update Delivery Optimization' in Intune, very close to the same time this first site started having the ARP breaches.

It made sense to me that this would generate a lot more LAN traffic as it uses P2P to deliver updates in order to spare bandwidth on the WAN connection, so to test this we switched it off at this site. For the next 2 days I hadn't seen any more breaches leaving the limit at 300 - which it was easily breaching multiple times daily beforehand at this site.

Manager told me Delivery Optimization has to stay on - it's not going anywhere - and has asked me to simply turn the limit off all together and leave it off. To my understanding this limit is there to prevent DoS attacks to the switch, and I have conveyed that to manager. Personally I do not think this would be a good idea to weaken network security for the sake of making a feature work, so here I am trying to find a solution where both can co-exist.

I am hoping that maybe I can find some helpful suggestions here that will allow me to come up with a solution for this situation, maybe someone out there has come across something similar. What do you guys think would be a safe number to raise the limit to? I wanted to avoid raising it too high other-wise what is the point of it.

Here is the config applied to all AP ports on switches (across all sites):

 

description UNIFI_AP
switchport access vlan 2
switchport mode access
switchport nonegotiate
load-interval 30
ip arp inspection trust
ip arp inspection limit rate 200
ip verify source
ip dhcp snooping limit rate 100
power inline port 2x-mode 

 

7 Replies 7

johnd2310
Level 8
Level 8

Hi,

 

Are your access points autonomous or are they managed by a controller. If your wireless is controller based, is the traffic tunneled to the controller or is it switched out the local switch port?

 

Thanks

John

**Please rate posts you find helpful**

APs are Unifi and are managed by a controller. APs communicate with controller through an MPLS route.

I do not think this setup has anything to do with the issue however as it's been like this way before the issue surfaced.

this MS feature need each PC send update to other PC and hence send ARP to build Mac address. 
if we can make Mac aging time longer to reduce the ARP request from PC and keep ARP limit as before. 

OK this is an interesting idea, was not aware of this to be honest.

I have looked into this and I think you may be talking about setting MAC aging with 'mac address-table aging-time' command?

Currently it has a default global aging time of 300 seconds, any recommendations on how much to increase by?

Not SW must change the mac aging time the PC/Server must change.

for the recommend, 

for example the update must send every 6 min but the aging is 5 min.

so the aging must be > update time. 

this make when PC/Server want to send update it Mac table is full and if it need send some ARP, but 

when want to send update but mac table is empty it will send ARP and here the limit is pass.

Hello,

 

I don't know what and what not you are allowed to change, but on the 'Delivery Optimization' settings page, you can select:

 

Allow downloads from other PCs

PCs on my local network

PCs on my local network, and PCs on the Internet (the default)

 

If you change this to 'PCs on my local network', that might reduce ARP traffic as well.

Hello

Options:
1) increase arp timeout
2) increase DAI limit
3) turn DAI off
4) turn off WUDO and inplement a WSUS (windows server update service) which is a central server that downloads windows updates on behalf of all clients and then group policy the clients towards the update server


Please rate and mark as an accepted solution if you have found any of the information provided useful.
This then could assist others on these forums to find a valuable answer and broadens the community’s global network.

Kind Regards
Paul
Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: