cancel
Showing results for 
Search instead for 
Did you mean: 
cancel

whereis Spark Bot - Integrating Spark and CMX

1786
Views
10
Helpful
1
Comments
Beginner

What is whereis?

Earlier this year I created the whereis Spark Bot (whereis@sparkbot.io) to address a challenge I faced in the new Cisco North Sydney office (see - The Australian). While the space gives maximum flexibility in work style, and the technology allows for anywhere, anytime communication, sometimes you still need to find someone in the office. This could be for an ad-hoc discussion that would benefit from being face-to-face, meeting up before heading out to a customer meeting, or simply to have a coffee with a colleague. In an Activity Based Workspace (or a Cisco Connected Workspace), finding a colleague can be a challenge. whereis solves this by providing on-demand colleague location in Spark, leveraging Spark Webhooks and the Cisco Connected Mobile Experience (CMX) API.

whereis.png

In this blog, I'll explain how the bot works.

Architectural Overview

whereis consists of 3 python scripts:

- whereis.py which is the webhook receiver. It receives the POST from Spark, GETs the message, and then POSTs the queried username to the other scripts.

- see-em-x.py which takes the POSTed username, queries a list of CMX servers to find the user (based on 802.1X username), and then renders the location on the floor image.

- see-em-x-mse.py which has a similar function to see-em-x.py but queries the legacy MSE API.

whereis_overview.png

The CMX API is the preferred method as it provides a list of all devices with the queried username e.g. for users with more than one device (isn't that all of us? ). The MSE API is used for buildings that aren't synced with CMX, and as a backup for CMX. It is limited in that it only returns the first device with that username.

The modular nature is mainly due to historical reasons which you will see in the code - it was initially designed for a web front-end so see-em-x.py also accepts input from an HTML form and responds with the data in HTML format.

The Initial Request

When a user enters a username (either just the username in a 1:1 "People" Spark conversation, or after mentioning the bot in a "Space"), the following happens:

whereis-1.png

When whereis.py receives the POST from the webhook, it requests the content of the message with the sendSparkGET function.

def sendSparkGET(url):

    request = urllib2.Request(url,

                            headers={"Accept" : "application/json",

                                     "Content-Type":"application/json"})

    request.add_header("Authorization", "Bearer "+bearer)

    contents = urllib2.urlopen(request).read()

    return contents

@post('/')

def index(request):

    webhook = json.loads(request.body)

    print webhook['data']['id']

    result = sendSparkGET('https://api.ciscospark.com/v1/messages/{0}'.format(webhook['data']['id']))

    result = json.loads(result)

It then does some formatting (e.g. removing the "whereis" from the mention and changing everything to lowercase) as well as handling "help" input before sending the username to the other scripts.

The grequests module is used to send a POST with the username to both scripts.

import grequests as async

        payload = {'person':person, 'clientCurrent':'0', 'source':'spark', 'clientList':'[]'}

        headers = {'User-Agent': 'Mozilla/5.0'}

        async_dict = {}

        request = (async.post(url = u, headers=headers, data=payload) for u in cmx_script_urls)

        responses = async.map(request)

The see-em-x.py script stores the username as 'person' and sends a GET for '/api/location/v2/clients?username=' to every CMX server with all domain-username combinations e.g. 'matfowle', 'CISCO\matfowle', 'cisco\matfowle'.

        # Add each CMX URL with each username, domain prefix combination to the list.

        for u in urls:

          urlClientByUsernameWithDomain.extend((u+urlClientByUsername+p) for p in prefixes)

        # Dictionary where we will store the responses from CMX in a format of URL:JSON.

        async_dict = {}

        # Use the grequests module to send requests to all the CMX servers at the same time with all domain prefix and username combinations.

        request = (async.get(url = u+person, auth = HTTPBasicAuth(cmxUser,cmxPass),verify=False, timeout=2) for u in urlClientByUsernameWithDomain)

        responses = async.map(request, exception_handler=exception_handler)

A similar process is done in the see-em-x-mse.py script with some changes in the API URL.

The Initial Response


At the same time that the above was happening, whereis.py has communicated back to the room that the bot is looking for the user. This is to give the user some quick feedback that the request has been received.

whereis-2.png

If the use is found, the response from the CMX server is a list of devices found with the requested username. If the user is not found, an empty JSON response is returned by CMX.

The responses from every CMX server is stored in a dictionary in a 'url : json' format which is then looped through to see which CMX server has found the user.

        # Check that the returned responses are all valid and if so, put them in the list in the format of URL:JSON.

        for result in responses:

          if result:

            async_dict[result.url] = json.loads(result.content)

        # From the dictionary, find which server responded and set that as the CMX server so that we can do the image call.

        for k in async_dict:

          if async_dict[k]:

            for u in urls:

              if u in k:

                cmxAddr = u

            clientList = async_dict[k]

            break

          else:

            clientList = async_dict[k]

From this response, the username, map hierarchy (Campus>Building>Floor), floor image name, floor dimensions, the x,y coordinates for the device, the SSID name and the status (associated/probing) will be used.


Get the Floor Image


Now that the floor image name is known, the see-em-x.py script will go and get the actual image file from the CMX server using '/api/config/v1/maps/imagesource/' and then cache it on the local disk. This means if we see this floor again in a request we don't have to waste bandwidth getting it every time from the CMX server.


whereis-3.png


    # Check if we have already got the floor image file stored locally.

    if os.path.isfile(image_path + client["mapInfo"]["image"]["imageName"]) == True:

      file = image_path + client["mapInfo"]["image"]["imageName"]

      fh = open(file, "rb")

      image = storeMemory(fh.read()).encode("base64").strip()

      fh.close()

    else:

    # Get the image file of the floor the client is on from CMX, save it to a file, then load it in StingIO so we can work with it in memory.

    # We save it to a file first so that next time we don't have to pull an image from CMX. Speed things up and reduces load on CMX.

      image = cmxContent(cmxAddr+urlFloorImage + client["mapInfo"]["image"]["imageName"])

      file = image_path + client["mapInfo"]["image"]["imageName"]

      fh = open(file, "w+")

      fh.write(image)

      fh.close()

      fh = open(file, "rb")

      image = storeMemory(fh.read()).encode("base64").strip()

Return the Location

At this stage, see-em-x.py has all the information it needs to return the location, but it needs to render the location on the floor image. This is done using pyplot by creating a plot on top of the image and marking the x,y of the client with the scatter function. The first scatter line plots the x,y with a dot and the next 3 draw concentric circles around the point.

import matplotlib.pyplot as plt

implot = plt.imshow(convertedim, extent=[0, client["mapInfo"]["floorDimension"]["width"], 0, client["mapInfo"]["floorDimension"]["length"]], origin='lower', aspect=1)

    # Mark the client's coordinates that we received from CMX.

    # The first line will draw a dot at the x,y location and the second and third lines will draw circles around it.

    plt.scatter([str(client["mapCoordinate"]["x"])], [str(client["mapCoordinate"]["y"])], facecolor='r', edgecolor='r')

    plt.scatter([str(client["mapCoordinate"]["x"])], [str(client["mapCoordinate"]["y"])], s=1000, facecolors='none', edgecolor='r')

    plt.scatter([str(client["mapCoordinate"]["x"])], [str(client["mapCoordinate"]["y"])], s=2000, facecolors='none', edgecolor='r')

    plt.scatter([str(client["mapCoordinate"]["x"])], [str(client["mapCoordinate"]["y"])], s=3500, facecolors='none', edgecolor='r')

The plot is then formatted correctly using the gca function.

    # Currently the plot is the same size as the image, but the scale is off so we need to correct that.

    ax = plt.gca()

    ax.set_ylim([0,client["mapInfo"]["floorDimension"]["length"]])

    ax.set_xlim([0,client["mapInfo"]["floorDimension"]["width"]])

    # The plot starts 0,0 from the bottom left corner but CMX uses the top left.

    # So, we need to invert the y-axis and, to make it easier to read, move the x axis markings to the top (if you choose to show them).

    ax.set_ylim(ax.get_ylim()[::-1])

    ax.xaxis.tick_top()

    # Use this to decide whether you want to show or hide the axis markings.

    plt.axis('off')

    # Save our new image with the plot overlayed to memory. The dpi option here makes the image larger.

    plt.savefig(buff, format='png', dpi=500)

see-em-x.py now has everything it needs to send back to whereis.py.

whereis-4.png

The username, status, SSID, building name, floor name, final image location (which is written to disk with a UUID filename), number of devices for this user, and the JSON response from CMX are returned to whereis.py.

    # If the request came from Spark, save the image to a file so that the Spark Bot can send it as a file. It also sends some text about the user and their location.

    if source == "spark":

      file = image_path + str(uuid.uuid4()) +".png"

      fh = open(file, "w+")

      fh.write(newimage.decode('base64'))

      fh.close()

      print "Content-type: application/json"

      print

      response = {'text': client["userName"] + ' is '+ client["dot11Status"] +' to '+ client["ssId"] +' in '+ hierarchy[1] +' on level '+ hierarchy[2], 'image': file, 'clientCount': clientCount, 'clientList': clientList}

      print (json.JSONEncoder().encode(response))

      return

Similar is done in see-em-x-mse.py but as mentioned, only one device is returned for the specified username.

whereis.py will receive responses from both scripts and put them in a dictionary. It will first check if see-em-x.py found the user (by checking if an image was sent), if not it will check if see-em-x-mse.py found the user. If it also doesn't find the user, the not found message is sent using sendSparkPOST(), otherwise it will send the found message and the image. It will also clean up the plotted image from the disk.

        for result in responses:

            if result:

                async_dict[result.url] = json.loads(result.content)

        if async_dict[cmx_script_url]['image'] == False:

            data = async_dict[cmx_script_url2]

            text = data['text']

            filepath = data['image']

            clientCount = int(data['clientCount'])

            clientCurrent = 0

            clientNext = clientCurrent

            filetype = 'image/png'

            print text

            if filepath == False:

                sendSparkPOST("https://api.ciscospark.com/v1/messages", {"roomId": webhook['data']['roomId'], "text": text})

                return "true"

            sendSparkPOST("https://api.ciscospark.com/v1/messages", {"roomId": webhook['data']['roomId'], "text": text, "files": ('location', open(filepath, 'rb'), filetype)})

            os.remove(filepath)

            return "true"

        if async_dict[cmx_script_url2]['image'] is not False:

            os.remove(async_dict[cmx_script_url2]['image'])

If see-em-x.py does find the user then it will use this data instead as it has support for multiple devices per user. However, this means that whereis.py needs to check the number of devices (clientCount). If it is greater then one, first, send the first device location to the Spark room.

        data = async_dict[cmx_script_url]

        text = data['text']

        filepath = data['image']

        clientCount = int(data['clientCount'])

        clientList = data['clientList']

        clientCurrent = 0

        clientNext = clientCurrent + 1

        filetype = 'image/png'

        print text

        if clientCount > 1:

            sendSparkPOST("https://api.ciscospark.com/v1/messages", {"roomId": webhook['data']['roomId'], "text": person +" has "+ str(clientCount) +" devices. I'll get their locations for you."})

            sendSparkPOST("https://api.ciscospark.com/v1/messages", {"roomId": webhook['data']['roomId'], "text": text, "files": ('location', open(filepath, 'rb'), filetype)})

            os.remove(filepath)

Then, for every extra device with this username, POST the JSON received from CMX to see-em-x.py but set the currentClient to the next device in the list. By passing the CMX JSON between see-em-x.py and whereis.py and back again, we can skip see-em-x.py having to query CMX again.

whereis-5.png

            while clientNext < clientCount:

                payload = {'person':person, 'clientCurrent':str(clientNext), 'source':'spark', 'clientList':clientList}

                headers = {'User-Agent': 'Mozilla/5.0'}

                msg = requests.post(cmx_script_url, headers=headers, data=payload)

                data = json.loads(msg.content)

                text = data['text']

                filepath = data['image']

                filetype = 'image/png'

                sendSparkPOST("https://api.ciscospark.com/v1/messages", {"roomId": webhook['data']['roomId'], "text": text, "files": ('location', open(filepath, 'rb'), filetype)})

                os.remove(filepath)

                clientNext += 1

And that's it!

whereis.png

What Next?


First, whereis@sparkbot.io will only work for Cisco employees but you can grab the code for whereis and adapt it for your environment here - GitHub - matfowle/whereis. Note, I am no developer so my code can probably use some work!


In the future I would like to add corporate directory support so that users don't have to remember usernames and can instead enter "Firstname Lastname". Until then, let me know what you think below in the comments. Also, be sure to check out the CMX API over at Cisco DevNet: CMX Mobility Services for yourself, and post below if you have created something leveraging the API!

1 Comment
Cisco Employee

Amazing leverage of CMX APIs to leverage all that data in a real world scenario!

CreatePlease to create content
Content for Community-Ad
August's Community Spotlight Awards
This widget could not be displayed.