Rate Limit exceeded -DNA center API

[beginnerciscouser]

Hello,
I am beginner user of Cisco catalyst center. I want to get the client details using mac-address as the parameter.At the moment, i am able to receive the client details of a list of mac-address (which inlcude 10,000 mac address) but this process takes about 5-6 hours to receive the data.

So now i want to speed up the process, for that i would like to receive atleast 5 clients/ minute .Such the process will take less time.

This is the error message :

Error: 429, {
"error" : "Rate Limit exceeded; BapiName: Get Client Detail, RateLimit config details: RateLimitContext{rate=100, windowUnit='minute', windowDuration=1, maxConcurrentExecutionsPermitted=100}",
"bapiExtendedStatusCode" : "REJECTED_ABOVE_THROTTLE_LIMIT",
"bapiExtendedStatusDescription" : "For BAPI: Get Client Detail, maximum allowed BAPI instances per 1 minute is 100. The limit has been reached for the time-window between Fri Feb 09 09:34:19 UTC 2024 and now"
}
Could someone help me to solve this issue.

Thanks in advance

[bigevilbeard]

Not sure what version of DNAC you are running, but certain APIs have rate-limits (throttling). Your issue is identified in your DNAC error. Like your error notes, "429 Too Many Requests" error indicates you're exceeding the permitted BAPI calls per minute (100 in this case). If you reduce your calls to 100 per minute or adjust your approach based on the actual rate limit and your needs.

I guess one way would to instead of retrieving all details for 10,000 clients at once, use paginated API calls or filter your requests to specific attributes if this is possible (you might need to look at the SDK for this, as last i was aware, unfortunately, as per the API documentation for DNA, there is no pagination mechanism provided, requiring manual incrementation of the offset parameter). You might want to look at these links by referring to: "offset = 500" if this will meet your use case, and this will reduce data transfer and potential throttling or timing constraint in your code to delay your loops, but the latter would increase the time it took to gather that data back and you already mentioned this is in the hours now (wow btw!)

https://community.cisco.com/t5/automation-and-analytics/method-to-return-more-than-500-devices-in-api-get-quot-network/td-p/3437814

https://community.cisco.com/t5/cisco-digital-network/dna-center-api-500-record-limit/td-p/4125116

@Gabriel Zapodeanu is the master here on this subject

[beginnerciscouser]

Thank you for your reply @bigevilbeard 

I am using the version 2.3.5.5

I have used "offset" when retrieving network-device details from DNA center.

But at the moment i am trying to get the client details with mac address as the parameter.I am not sure if setting an "offset" would solve my issue . 
I am getting the client details using this API With mac-address from an excel file. 

"https://XXXXXXXXXXXXXXXXX//dna/intent/api/v1/client-detail?macAddress=xx:xx:xx:xx:xx:xx"

Some of the similar issues:

https://community.cisco.com/t5/controllers/what-is-the-limitation-for-frequency-to-using-dnac-api/td-p/3857189

https://community.cisco.com/t5/cisco-digital-network-architecture-dna/dna-api-limit-ap-report-is-there-a-better-way/td-p/4182308

[rslaski]

@bigevilbeard @Gabriel Zapodeanu I am facing exactly the same issue but with DNAC Terraform provider, so I have no control over how it sends API calls internally. Sorry to say, but the DNAC API quality is totally crappy in all aspects (bugs, performance, documentation, lifecycle).

dnacenter_sda_fabric_site.hector: Still creating... [2m0s elapsed]
╷
│ Error: Failure when executing AddSiteInSdaFabric
│
│ with dnacenter_sda_fabric_site.hector,
│ on sda_fabric.tf line 3, in resource "dnacenter_sda_fabric_site" "hector":
│ 3: resource "dnacenter_sda_fabric_site" "hector" {
│
│ error with operation AddSiteInSdaFabric
│ {
│ "error" : "Rate Limit exceeded; BapiName: Add Site in SDA Fabric, RateLimit config details: RateLimitContext{rate=50, windowUnit='minute', windowDuration=1, maxConcurrentExecutionsPermitted=1}",
│ "bapiExtendedStatusCode" : "REJECTED_ABOVE_MAX_CONCURRENCY_LIMIT",
│ "bapiExtendedStatusDescription" : "For BAPI: Add Site in SDA Fabric, DNACP Runtime is at maximum allowed concurrent BAPI executions: 1"
│ }

[Gabriel Zapodeanu]

Terraform has a default parallelism of 10, which may cause issues for some workflows. You may try using a value of 1, and avoid hitting API concurrency limits: https://developer.hashicorp.com/terraform/cli/commands/apply#parallelism-n

[rslaski]

@Gabriel Zapodeanu I am aware of this and I am using 1 as parallelism parameter. Some resources can be created (ex. site), others (ex. SDA) fail with 429. There's something seriously broken in 2.3.7.4 or the provider.

[Gabriel Zapodeanu]

@rslaski The Terraform provider does not support Catalyst Center 2.3.7.4. The latest supported version is 2.3.5.x. We are planning for provider support for a future 2.3.7.x GA version.

[bigevilbeard]

If you are using the TF provider, i would open an issue on the GitHub repo for the maintainers of this.

Hope this helps.

[Gabriel Zapodeanu · Accepted solution]

There is another option to collect the client details using the Client Details report being triggered via APIs: https://github.com/cisco-en-programmability/dnacenter_reports_operations

[beginnerciscouser]

Thank for the reply @Gabriel Zapodeanu 
It was helpful.

I was using client-details API to retrieve the HostName, Linkspeed, Mac_address, IPV4,  connecteddevice, VLANID and port of the client devices.

Using your script i am able to get the details except the port to which the client is connected.
Is there any way i can get the details of the port

[Gabriel Zapodeanu]

@beginnerciscouser You welcome.
In my report, (Catalyst Center version 2.3.7.4) I have these params included:
"connectedDeviceName": "PDX-M",
"port": "GigabitEthernet1/0/33"
providing the access switch and switchport details

[beginnerciscouser]

Hi,
I am using your dnacenter_create_report_download.py
The output of the function "get_report_file" is dictionary. These are the params i am receiving in one of the dictionary

 {"hostName": "--", "username": "xxxxxxx", "macAddress": "xxxxxxxx, "ipv4": "xxxxxxxx", "ipv6": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", "deviceType": "Wireless", "connectionStatus": "Connected", "averageHealthScore_min": "xx", "averageHealthScore_max": "10", "averageHealthScore_median": "7", "usage_sum": "0.0", "connectedDeviceName": "xxxxx", "frequency": "xx", "rssi_median": "xx", "snr_median": "xx", "site": "xxxx", "lastUpdated": "14 Feb, 06:50:00 AM", "apGroup": "xxx", "ssid": "xxxx", "vlan": "xxxx", "vnid": "--", "onboardingEventTime": "14 Feb, 06:40:14 AM", "assocDoneTimestamp": "14 Feb, 06:40:09 AM", "authDoneTimestamp": "14 Feb, 06:40:10 AM", "aaaServerIp": "xxxxx", "dhcpDoneTimestamp": "14 Feb, 06:40:14 AM", "maxDhcpDuration_max": "xxx", "dhcpServerIp": "xxxx", "linkSpeed": "--", "txRate_min": "0.0", "txRate_max": "0.0", "txRate_avg": "0.0", "rxRate_min": "0.0", "rxRate_max": "xxxx", "rxRate_avg": "xxxxx", "txBytes_sum": "0.0", "rxBytes_sum": "0.0", "dataRate_median": "xxx", "dot11Protocol": "xxxxx"},
From this dict, i am able to retrieve the HostName, Linkspeed, Mac_address, IPV4, connecteddevice, VLANID.  There is no information about the port

I am using the version 2.3.5.5 maybe upgrading to the version 2.3.7.4 which you are using might help me get the port details

Hoping to hear from you soon.
Thanks in advance

[beginnerciscouser]

I have updated my Cisco Catalyst Center to version 2.3.7.4.
Unfortunately, still no information about the port details. Could you please help me with this issue?

@Gabriel Zapodeanu 

[Gabriel Zapodeanu]

Could you please check using the client detail or client enrichment details APIs if the port details are available? Is this info not available for any clients from the report? Are the details available from the UI?

[beginnerciscouser]

Yes, I have cross checked it using the client-details API

"https://xx.xx.xx.xx/dna/intent/api/v1/client-detail?macAddress=F8:E4:3B:BB:91:B8

and the port details are available.Also in the UI it is available
But with the report the port details are not available.

Below you can find the sample output of the report

{"hostName": "--", "username": "--", "macAddress": "FE:D7:41:E8:AD:09", "ipv4": "xx.xx.xx.xx", "ipv6": "fe80::fcd7:41ff:fee8:ad09", "deviceType": "Wired", "connectionStatus": "Connected", "averageHealthScore_min": "10", "averageHealthScore_max": "10", "averageHealthScore_median": "10", "usage_sum": "201287.41", "connectedDeviceName": "net166.net.xx.xx.xx", "frequency": "--", "rssi_median": "--", "snr_median": "--", "site": "Global/01\u00fclich/Geb\u00e4udebereich 16/16.60uvxy", "lastUpdated": "21 Feb, 11:48:08 AM", "apGroup": "--", "ssid": "--", "vlan": "18", "vnid": "--", "onboardingEventTime": "--", "assocDoneTimestamp": "--", "authDoneTimestamp": "--", "aaaServerIp": "--", "dhcpDoneTimestamp": "--", "maxDhcpDuration_max": "--", "dhcpServerIp": "--", "linkSpeed": "1000.0", "txRate_min": "34944.6", "txRate_max": "8.5218272048E8", "txRate_avg": "1.297082128E7", "rxRate_min": "81603.99", "rxRate_max": "2.8971478557E8", "rxRate_avg": "5665746.0", "txBytes_sum": "140099.59", "rxBytes_sum": "61187.82", "dataRate_median": "--", "dot11Protocol": "--"},{"hostName": "--", "username": "--", "macAddress": "xx.xx.xx.xx", "ipv4": "134.94.154.91", "ipv6": "fe80::fe34:97ff:fee4:54e1", "deviceType": "Wired", "connectionStatus": "Connected", "averageHealthScore_min": "10", "averageHealthScore_max": "10", "averageHealthScore_median": "10", "usage_sum": "12043.4", "connectedDeviceName": "net177.net.xxxx", "frequency": "--", "rssi_median": "--", "snr_median": "--", "site": "Global/01J\u00fclich/Geb\u00e4udebereich 15/15.21u", "lastUpdated": "21 Feb, 11:48:08 AM", "apGroup": "--", "ssid": "--", "vlan": "11", "vnid": "--", "onboardingEventTime": "--", "assocDoneTimestamp": "--", "authDoneTimestamp": "--", "aaaServerIp": "--", "dhcpDoneTimestamp": "--", "maxDhcpDuration_max": "--", "dhcpServerIp": "--", "linkSpeed": "1000.0", "txRate_min": "1758.17", "txRate_max": "4116977.81", "txRate_avg": "66717.65", "rxRate_min": "639802.74", "rxRate_max": "3685795.32", "rxRate_avg": "1048423.94", "txBytes_sum": "720.4", "rxBytes_sum": "11322.99", "dataRate_median": "--", "dot11Protocol": "--"}

Kindly please let me know how can this be resolved.
Thanks in advance