cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
754
Views
0
Helpful
6
Replies
Highlighted
Beginner

MSE API Large Query Question

Trying to gather all client history information so we can load it into mongoDB and do some crunching on it.I can run various queries successfully(using postman) but when I run the /location/history/clients query without an id I get a 502 error. Usually with other queries when they are malformed you just get a blank page. The query without any id runs for a couple of minutes and returns the following error(basically a 502). I suspect that the time to get all the data( a location history count returns a count of 11.9 million) is tripping a timeout on the web server - or too much data perhaps.  Just wondering if there is anything on my end I can do before appealing to the web server admin for help.Note: I am a database admin and any web development work is about 10 to 15 years in my past! I did a quick check through my Chrome browser settings and didn't see anything for a timeout setting on my end.

The query is of the form

https://XXX.XXX.XXX.XX/api/contextaware/v1/location/history/clients

The results are:

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">

<html>

    <head>

        <title>502 Proxy Error</title>

    </head>

    <body>

        <h1>Proxy Error</h1>

        <p>The proxy server received an invalid

response from an upstream server.

            <br />

The proxy server could not handle the request

            <em>

                <a href="https://community.cisco.com/api/contextaware/v1/location/history/clients">GET /api/contextaware/v1/location/history/clients</a>

            </em>.

            <p>

Reason:

                <strong>Error reading from remote server</strong>

            </p>

        </p>

    </body>

</html>

6 REPLIES 6
Highlighted
Frequent Contributor

Hi Jon,

According to the API docs for CMX, there is nothing to prevent using /api/location/v1/history/clients to get history on all clients, whether there is 1 or several hundred clients.  However, according to the CMX engineers, they will suggest using it with mac address and query conditions such as date. 

Matt

Try something like this for CMX 10 & 8 respectively,

https://xx.xx.xx.xx/api/location/v1/history/clients?date=2016/03/14&fromHour=0&toHour=1

https://xx.xx.xx.xx/api/contextaware/v1/location/clients/$id/count?<query condition>=<value>

where

QUERY PARAMETERS

ParameterDescription
idMacaddress (all), IP Address (client) or Username (client) - optional

QUERY CONDITIONS

ParameterDescription
mapHierarchyString containing campus>building>floor
currentlyTrackedBoolean if client is currently found by the MSE
locatedAfterTimeTimestamp
locatedBeforeTimeTimestamp
sortByString parameter:<asc
pageInteger
pageSizeInteger (default 5000)
Highlighted

Hey, thanks for responding Matthew!


Tried some things:

     https://xx.xx.xx.xx/api/location/v1/history/clients?date=2016/04/02&fromHour=0&toHour=1

Had the same 502 error.

When I ran the count version with the same conditions I just got a count of all the history and the qualifiers had no effect.

     https://xx.xx.xx.xx/api/contextaware/v1/location/clients/count?date=2016/03/14&fromHour=0&toHour=1


It returned virtually(of course more data is added every second) the same results as simply:

     https://xx.xx.xx.xx/api/contextaware/v1/location/clients/count


Get the same 502 when I try and narrow things down a bit with a mapHeirarchy specified:

     https://xxx.xxx.xxx.xx/api/contextaware/v1/location/history/clients?mapHierarchy=xxxxxxxx>xxxx>xxxx

A count in this case does count correctly, but when I remove the count I get errors trying to get all history for the floor. Also, adding a date to the floor and count version has no effect and just returns a count of the entire history.


If I go after just current clients with a floor qualifier it works fine and I get all the data for that. I add the history and it fails. For example:

     https://xxx.xxx.xxx.xx/api/contextaware/v1/location/clients?mapHierarchy=xxxxx>xxxxx>xxxxx

works fine.

     https://xxx.xxx.xxx.xx/api/contextaware/v1/location/history/clients?mapHierarchy=xxxxx>xxxxx>xxxxx

fails with a 502 error

Is it possible to get credentials set up to query any database underneath MSE or is the API the only way to get data out? I am in a chicken and egg situation where they don't have clients going in. They want to analyze all client movement. We'd like to dump this all into MongoDB rather than have to navigate using the API which is very tedious and we have yet to decide the best approach even if we have to write a program that uses the API to get all the information we want.



Highlighted
Frequent Contributor

Jon,

Let me find out if a process already exists to do this.  I'll follow-up soon.

Matt

Highlighted

Thanks Matthew. What we are trying to do is see if this data can help to improve patient experience by providing insights of what might be possible to reduce their waiting time or order procedures in a way that might be more comfortable for them... etc.

Highlighted
Frequent Contributor

Jon,

Here is a CMX engineer's reply:

Hi Experts,

Regarding CMX 8.0, the customer would like to know if a process exists to query any database underneath MSE or another way (not API) to get data out.  He is running into timeouts with client history information.

Here is his questions, below.

Thanks,

Matt

...

Matthew,

With MSE 8.0, there is an underlying Oracle database and the customer could theoretically go directly into the Database, but this is not something BU supports or recommends.

The API method is the recommended method and if they are getting too much data they should select data records only for specific day at a time.

Regards,

Darryl Sladden

Senior TME Manager, Wireless and Location

Cisco

Highlighted

After trying a variety of things it appears that pagesize is the key to not overwhelming anything. For example a pagesize of 10 seems to work consistently.  Paring that way down seems to work ok to get results. Also the mapHierarchy if an entire area is not required. So I think we can proceed by writing some code that is fed in map information and then goes through history fetching the pages in a loop. However I did notice that not all the parameters seem to have been carried forward in the next URL that is constructed near the top of each result page - so I will need to look more at that. Not sure that is because of the API or the tools I was using to prototype calls to the API.

This widget could not be displayed.