topic Re: Response other than 200 - scanning API in Network Platform API

Response other than 200 - scanning API

lee2 — Sun, 12 Nov 2017 16:34:35 GMT

Hey everyone. I was following the CMX AWS lambda guide - https://github.com/dexterlabora/cmxreceiver-lambda-inline

It looks like everything is all good on the AWS end. If I copy my post URL into the browser, it returns the correct meraki validation key. However, the URL will not validate. I receive a "Response other than 200". If I check the AWS logs, it looks like the request is never making it to the API gateway. I checked the event logs for my MX64 and OpenDNS to see if the URL was somehow being blocked.

Any recommendations or ideas would really be beneficial.

Re: Response other than 200 - scanning API

colo — Wed, 15 Nov 2017 21:07:17 GMT

Hey Lee,

I suggest emailing support@meraki.com for some extra help. They'll probably start by asking you to test your URL is valid by pasting it into a web browser.

Here is the documentation on troubleshooting:

https://documentation.meraki.com/MR/Monitoring_and_Reporting/Location_Analytics#Scanning_API_Data_is_not_received

Regards,

Colin Lowenberg

Re: Response other than 200 - scanning API

lee2 — Thu, 16 Nov 2017 15:57:30 GMT

Thanks Colin. I am working with Meraki support right now. They seem to think it's a SSL issue. If I execute the URL in the browser, I get the proper response with the correct validator. I'll keep everyone updated just in case anyone else runs into the same problem.

Re: Response other than 200 - scanning API

DexterLabora — Fri, 17 Nov 2017 09:10:26 GMT

Let me know if you find any issues with the AWS lambda code. I wrote the NodeJS receiver code section a year ago and I'm happy to update something if it proves to be the problem child.

https://github.com/dexterlabora/cmxreceiver-lambda-inline/blob/master/src/cmx-lambda-inline.js

Cheers,

Cory

Re: Response other than 200 - scanning API

lee2 — Fri, 17 Nov 2017 13:46:54 GMT

Cory! You're awesome. I was actually considering emailing you, but thought "Nah, he probably won't respond".

First, I love your tutorial. I'm not a developer by trade, but I understand coding. It was very easy to follow. I've been working with Meraki support but it looks like they are kind of blowing me off and telling me to talk to AWS support. All my API gateway logs in AWS are showing that when I execute the URL from the browser, it returns code 200 and the correct validation key. When I execute the URL from the Meraki Dashboard, there is nothing happening on the AWS side. No log created. No API call to the gateway. I'm guessing that means the problem is from the Meraki cloud to the AWS API gateway. It's hard to tell since nothing is being logged on the AWS side.

Re: Response other than 200 - scanning API

DexterLabora — Fri, 17 Nov 2017 15:25:15 GMT

Thanks for the kind words!

Since I'm not positive on your entire setup, you might find this Postman collection I just wrote helpful for troubleshooting.

It will basically allow you to simulate a Meraki GET and POST request to your AWS server or any other for that matter. Just update the Endpoint address (i.e. localhost:1880) to your instance. I use this all the time while developing on an airplane 😉

https://documenter.getpostman.com/view/897512/71FUpux

First test with the GET request, to see if AWS return your validator key

Then, test with one of the POST requests to send an observation set.

Look into your AWS Cloud Watch logs to see if you get any helpful info.

The AWS permissions are somewhat complicated to get right.

Also, double check your Lambda Trigger settings.

(Here is an example test setup)

Good luck and let us know what you find out.

Cory

Re: Response other than 200 - scanning API

lee2 — Sun, 19 Nov 2017 14:24:48 GMT

Thanks Cory. I got it working. The data is being posted into DynamoDB. The ES with Kibana is a pain and I'm still working through that, but I'm happy I'm collecting the data! Thanks again

Re: Response other than 200 - scanning API

Cisco Super Expert — Mon, 11 Dec 2017 11:42:04 GMT

I had same problem, and I found one workaround to use "HTTP" request to AWS API gateway.

AWS API gateway is normally working only with HTTPS, so I place AWS CloudFront to use HTTP method on front side to the Meraki Cloud.

Meraki Cloud > (http) > CloudFront > (https) > API Gateway/Lambda/DynamoDB

Here is configuration example of CloudFront.

Origin SettingCache Setting

Re: Response other than 200 - scanning API

dyoshiha — Fri, 15 Dec 2017 11:13:03 GMT

Thank you for useful information.

I have the same problem. I have tried to apply your workaround, but the problem has not been solved. Should the 'Post URL' on Meraki dashboard start with 'http', or 'https' ? Do we need to change configuration on API gateway?

Any comment would be helpful.

Re: Response other than 200 - scanning API

Cisco Super Expert — Sat, 16 Dec 2017 02:25:25 GMT

In my configuration, Meraki dashboard 'Post URL' start with "http", not using "https", and no change at API Gateway.

Here is URL/Domain examples in this case.

API Gateway URL: https://xxxxx.execute-api.ap-northeast-1.amazonaws.com/prod/cmxreceiver-dynamodb

CloudFront Origin: xxxxx.execute-api.ap-northeast-1.amazonaws.com/prod

CloudFront Domain: yyyyy.cloudfront.net

Meraki dashboard Post URL: http://yyyyy.cloudfront.net/cmxreceiver-dynamodb

Re: Response other than 200 - scanning API

dyoshiha — Sat, 16 Dec 2017 03:41:29 GMT

That's what I wanted to know. It worked!!

Thank you very much!

Re: Response other than 200 - scanning API

Lawrence2 — Wed, 20 Dec 2017 00:44:55 GMT

Hi LeeVG,

Can you share how did you get it to work?

I have the same issue: Getting "Response other than 200" when validating my URL on AWS API Gateway from Meraki.

But if I call my URL via postman or browser, I'm getting the right validator back and response 200.
(I also used the tested from postman collection from Cory and it also worked)

From AWS log, it appears nothing has reached AWS at all. It seems like an connection issue from Meraki cloud.

I have also tried setting up a Cloud front to API Gateway via a http call from Meraki as suggested from other users on this post, but still getting the same error.

Any help is appreciated.

Regards,
Lawrence

Re: Response other than 200 - scanning API

Lawrence2 — Wed, 20 Dec 2017 02:03:13 GMT

Update: I got this to work now, by changing the API endpoint type from "Edge Optimized" to "Regional".

Have kept https in Meraki post URL to call API gateway directly.

Re: Response other than 200 - scanning API

lee2 — Mon, 08 Jan 2018 18:51:39 GMT

Oh man. I did the jerk IT thing. Ask for help and then never respond when I figured out the resolution. Sorry about that. I'm glad you got it to work. I actually shut it back down for a few reasons.

1. I'm not an AWS expert, but man... Security controls are a nightmare. I got it to work by basically turning off as many security measures as I could and granting everything permission. If you know AWS, then it is pretty awesome, but personally, I'm no expert. I can't remember the exact solution, but I think I read a blog post somewhere explaining the regional thing and that did the trick.

2. I wasn't exactly sure what to do with the data that the built in location analytics didn't already do. I guess longer retention rate? I can see this being useful if you were also pulling analytics into the same database from other things and pairing them with the Meraki data. Any other sample use cases would be welcome. This was a great tutorial.

3. Elastisearch is expensive.

Sorry about the late response!

Re: Response other than 200 - scanning API

colo — Mon, 08 Jan 2018 20:05:31 GMT

2. You can re-create the Meraki dashboard's analytics - but generally the use case is to go further into the data and combine it with other data. You get the real MAC address with the API and you can locate a specific user and link them to Syslog alerts/events, captive portal data with social media / email / SMS / loyalty programs. The best use without extra data sets would be simply ranking all of your sites and identifying trends like a stock ticker showing the increase in traffic over the last day, week, month. Meraki dashboard only let's you compare 2 networks or 2 network-tags or 2 AP-tags. If you want to get your top 10 gains and top 10 losses, like a stock tracker, that would need something like elastic search.

Re: Response other than 200 - scanning API

DexterLabora — Mon, 08 Jan 2018 20:43:02 GMT

1. When I was building this integration, I got tripped up on the permissions also, so you are not alone!

2. The purpose is to capture the raw data (i.e MAC addresses) to use as desired. The Meraki built-it reporting anonymizes the data and has a limited amount of reports. By collecting this data, you can trigger workflows for known clients, aggregate the data with other sources and analyze multiple networks as needed.

(Glad you liked the tutorial 😉 We hope to build many more tutorials like this for our developers and integrators.

3. As an alternative to Elasticsearch and AWS, I have also written a basic Google Firebase Database version.

It will provide a Firebase Function to run the CMX/Scanning API receiver. That data is then stored in the Firebase Database, by MAC address. The service is FREE. To get a solution similar to Elasticsearch, you could integrate Google Big Query, but there is a cost associated with that.

https://github.com/dexterlabora/MerakiLocationFirebase

Re: Response other than 200 - scanning API

Lawrence2 — Mon, 08 Jan 2018 21:36:12 GMT

No worries~ Thanks for sharing your experience~