topic Re: Meraki SDK rate limiting in Network Platform API

Meraki SDK rate limiting

cscbrannent — Wed, 08 Nov 2023 19:28:25 GMT

Hello - I've written a python script using the Meraki SDK to retrieve subnets from our org, about 600 networks, about 3 vlans/subnets per network. I'd like to compile a list and sort/organize all the subnets.

The script works fine if I limit it to the first 10. If I "unleash it", it crashes after about 30 networks. Error messages are coming from inside the SDK - it appears to be rate limiting.

Google fu suggested using "ratelimit" package - ratelimit with a decorator around functions making calls.

@limits(calls=10, period=1)

However, this hasn't helped. Googling and watching cisco/meraki videos is talking about making API calls, and that the SDK magically takes care of ratel imiting. I'm looking for the actual reference, in the SDK documentation, to how to configure rate limiting, how to check on it, troubleshoot, etc.

My function call, that appears to be over running the ratelimit.

@limits(calls=10, period=1)
def get_vlans(client, network_id): 
    """Get the subnets of the network(branch).  The branch will have multiple subnets
    Return format: [{"name":vlanA, "subnet":"1.2.3.4/24"}, {"name":"vlanB", "subnet":"1.2.3.4/24"} ]"""
    vlans_controller = client.vlans
    results = vlans_controller.get_network_vlans(network_id)

    subnets = []
    
    for item in results:
        a_subnet = {} # 
        a_subnet['name'] = item['name']
        a_subnet['subnet'] = item['subnet']
        subnets.append(a_subnet) # Adding an invididual vlan dictionary to the list of subnets.
    
    mx_vlans = subnets
    return mx_vlans

Thanks

Re: Meraki SDK rate limiting

cscbrannent — Wed, 08 Nov 2023 19:30:02 GMT

Crash messages:

  File "c:\Users\brannen.taylor\Code\meraki\meraki_env\Lib\site-packages\meraki_sdk\controllers\vlans_controller.py", line 60, in get_network_vlans
    self.validate_response(_context)
  File "c:\Users\brannen.taylor\Code\meraki\meraki_env\Lib\site-packages\meraki_sdk\controllers\base_controller.py", line 94, in validate_response
    raise APIException('HTTP response not OK.', context)
meraki_sdk.exceptions.api_exception.APIException: HTTP response not OK.

Re: Meraki SDK rate limiting

Raphael_L — Wed, 08 Nov 2023 19:47:47 GMT

Pretty sure you are hitting the API rate limit of 10 CPS.

Can you print the HTTP code ? It should be 429.

The logic is : if you get a 429 , wait ... if you get a 429 again , wait a bit longer...

Are you the only one using API on your Org ?

Re: Meraki SDK rate limiting

Philip D'Ath — Wed, 08 Nov 2023 20:09:21 GMT

Specify maximum_concurrent_requests and set it to something like 5. Here is an example (this is using async io, but you get the idea).

	async with meraki.aio.AsyncDashboardAPI(
		output_log=False,
		print_console=False,		
#		maximum_retries=100,
		maximum_concurrent_requests=5
	) as dashboard:

Re: Meraki SDK rate limiting

cscbrannent — Thu, 09 Nov 2023 13:24:26 GMT

I think I'm the only one calling the API, as the api key wasn't turned on in the dashboard until I turned it on. I've been using the SDK, instead of calling the API directly. I think I'll refactor to use requests and call the API directly.

Re: Meraki SDK rate limiting

John-on-API — Tue, 14 Nov 2023 01:56:31 GMT

using the Meraki SDK

Hi @cscbrannent would you be able to link to the SDK that you are using? Based on your code, it looks like you might be using an obsolete SDK. It doesn't look like you're using the current Python library, which does handle rate limiting for you.

Also, check out this example local subnet dumper script.