cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2801
Views
1
Helpful
13
Replies
ricsonchua
Beginner

Python Sample Notification Listener Not working

I would like to ask a few questions about the Python sample codes in the link below

https://msesandbox.cisco.com:8081/apidocs/code

Rest API has been working properly.

However we are unable to make Notification Listener code no matter what we do.

We are connected to https://msesandbox.cisco.com:8081

I would like to list down the following steps that our organization has done to try and make the script work.


CMX Side

1. Added notifications to push to our public server's IP as new notification for our IP one is public IP and the other is the IP that we use to connect via SSH-PuTTy. (SEE MANAGE_NEW_NOTIFICATION.PNG)

Host side

1. On our server we have Python 2.6 installed and OS is RHEL6.

2. We have changed iptables to allow connection 8080.

3. for the Python Sample Notification Listener code. We had used 0.0.0.0 8080 (we understand this to be listen all)

4. We had tested that socket works locally.

Nothing happens when we execute the socket. May we kindly know if there is something we need to do on CMX to be able to get a stream?

I also suspect that we have no data coming in in the 1st place that is why the socket is not getting anything. (Please see CONNECT & ENGAGE SCREENSHOT)

We do appreciate your kind response.

We also appreciate if Cisco can offer local assistance IF possible as I am based in Singapore.

Best regards,

Ricson

13 REPLIES 13
matfarre
Frequent Contributor

Hi,

I have moved the question toMobility Services API

We have opened a DevNet trouble ticket, also.

Regarding the Connect & Engage metrics, those look normal since the Connect feature is not being used, currently.  Please, look at the Analytics reports for wireless client activity.

Also, try using Notification Movement resource instead of Location Update.

Also, try using a different Listener sample code, such as the Node.js Sample Notification Listener.

We will verify the Python Sample Notification Listener code, and follow up soon.

Thanks for the question,

Matt

matfarre
Frequent Contributor

+--------------------------------+------------+------------+------------+------------+-----------------+-----------------+-----------------+

|            EndPoint            | Total Sent |  Success   |  Failure   |  SendRate  |   SuccessRate   |   FailureRate   |  Latency (ms)   |

+================================+============+============+============+============+=================+=================+=================+

|     http://110.92.127.230:8080 |   549101   |     0      |   549029   |   17.842   |   0.00(0.0%)    |  17.84(99.99%)  |        0        |

+--------------------------------+------------+------------+------------+------------+-----------------+-----------------+-----------------+

|     http://110.92.127.218:8080 |   555524   |     0      |   555452   |   19.819   |   0.00(0.0%)    |  19.82(99.99%)  |        0        |

+--------------------------------+------------+------------+------------+------------+-----------------+-----------------+-----------------+

|    http://110.92.127.218:18080 |   11161    |     0      |   11161    |   0.403    |   0.00(0.0%)    |  0.40(100.0%)   |        0        |

+--------------------------------+------------+------------+------------+------------+-----------------+-----------------+-----------------+

|       http://110.92.127.218:80 |    930     |     0      |    930     |   0.036    |   0.00(0.0%)    |  0.04(100.0%)   |        0        |

+--------------------------------+------------+------------+------------+------------+-----------------+-----------------+-----------------+

|       http://110.92.127.230:80 |   12386    |     0      |   12386    |   0.483    |   0.00(0.0%)    |  0.48(100.0%)   |        0        |

+--------------------------------+------------+------------+------------+------------+-----------------+-----------------+-----------------+

|     http://110.92.127.230:8000 |   13432    |     0      |   13432    |   0.416    |   0.00(0.0%)    |  0.42(100.0%)   |        0        |

+--------------------------------+------------+------------+------------+------------+-----------------+-----------------+-----------------+

| http://110.92.127.230:6677/tes |   10122    |     0      |   10122    |   0.319    |   0.00(0.0%)    |  0.32(100.0%)   |        0        |

|                              t |            |            |            |            |                 |                 |                 |

+--------------------------------+------------+------------+------------+------------+-----------------+-----------------+-----------------+

There is a CMX 10.2.2 CLI command that will show you some of the notification statistics.

  cmxctl metrics notification 

This shows that the notification are being sent to EndPoint.  This indicates that the Listener code or network access may be the problem.

Matt

matfarre
Frequent Contributor

Ricson,

I am getting good Location Update notification api results to RequestBin.

Another confirmation that the Python Listener or network is the issue.

I will look at the Python code, next.

Thanks,

Matt

{

  "notifications": [{

  "notificationType": "locationupdate",

  "subscriptionName": "Location Update Test",

  "entity": "WIRELESS_CLIENTS",

  "deviceId": "7d:ff:ef:5d:ff:ff",

  "lastSeen": "2016-06-08T17:27:38.448+0100",

  "eventId": 7630679,

  "locationMapHierarchy": "DevNetCampus>DevNetBuilding>DevNetZone",

  "locationCoordinate": {

  "x": 203.21071,

  "y": 24.652853,

  "z": 0.0,

  "unit": "FEET"

  },

  "geoCoordinate": {

  "latitude": -999.0,

  "longitude": -999.0,

  "unit": "DEGREES"

  },

  "confidenceFactor": 40.0,

  "apMacAddress": "00:2b:01:00:06:00",

  "associated": true,

  "ipAddress": ["10.10.20.199"],

  "ssid": "test",

  "band": null,

  "floorId": 723413320329068590,

  "batteryInfo": null,

  "vendorData": null,

  "timestamp": 1465403258448

  }]

}

matfarre
Frequent Contributor

python3 listener:

MATFARRE-M-X2AK:~ matfarre$ python3 python_listener.py

<_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'> using 192.168.0.7 port 8000

<_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'> waiting for a connection

<_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'> client connected: ('64.103.26.57', 35301)

<_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'> received "b'POST / HTTP/1.1\r\nAccept-Encoding: gzip\r\nContent-Type: application/json\r\nHost: 184-96-217-156.hlrn.qwest.net:8000\r\nUser-Agent: Jetty/9.1.5.v20140505\r\nContent-Length: 650\r\n\r\n{"notifications":[{"notificationType":"locationupdate","subscriptionName":"Location Update Test","entity":"WIRELESS_CLIENTS","deviceId":"be:ff:ef:5d:ff:ff","lastSeen":"2016-06-08T18:02:34.438+0100","eventId":7761409,"locationMapHierarchy":"DevNetCampus>DevNetBuilding>DevNetZone","locationCoordinate":{"x":154.64381,"y":48.90403,"z":0.0,"unit":"FEET"},"geoCoordinate":{"latitude":-999.0,"longitude":-999.0,"unit":"DEGREES"},"confidenceFactor":64.0,"apMacAddress":"00:2b:01:00:04:00","associated":true,"ipAddress":["10.10.20.179"],"ssid":"test","band":null,"floorId":723413320329068590,"batteryInfo":null,"vendorData":null,"timestamp":1465405354438}]}'"

Only needed to change this line to local address and port:

     server_address = ('192.168.0.7', 8000)                         # change this to your ip address

and be sure the port forwarding is to allow all traffic through port 8000 for the firewall security settings:

Port Forwarding List
LAN IPProtocolPorts
Forwarded
Remote
Ports
Remote DefinedEdit
192.168.0.7TCP8000N/AN/A

and be sure the notification receiver address is your public IP address (Google "what is my ip"):

184.96.217.156

Your public IP address

Thank for all the help Matthew. I am able to connect now using a 4G dongle (no fw/routing setup)

Will inquire with local Cisco support to setup connection to a corporate network in the near future

Best regards,

Ricson

Hi Matthew,

2 things I want to clarify

SETUP GUIDE

Given the private setup as below. Am I right to say that. I need to do the following

1. Add notifications to public IP X.X.X.X port 8000 in CMX

2. Execute listener instance using IP 192.168.0.3 port 8000 in Client machine

3. Configure router to enable port forwarding for IP 192.168.0.3 port 8000

To extend the image below. If CMX and and Client machine are on different routers. How do I configure port forwarding for both routers?

CMX.jpg

CMX stream

Another question with CMX stream.

I noticed that CMX will send 2 streams every time it connects. I was told that sandbox will stop sending stream after 2~3minutes. However the sandbox did not close gracefully which gives me "Connection reset by peer" error. Is this expected from sandbox?

Another observation I had is that CMX sends 2 packets one is a unique charatcer that tells client that it is the end of the stream.

However when I change the python script to if data is not None it results in a infinite loop.

I would like to know then what the character or the value is. Because I would prefer to handle timeouts gracefully.

client connected: ('64.103.26.57', 37224)

received "POST / HTTP/1.1

Accept-Encoding: gzip

Content-Type: application/json

Host: 27.104.198.151:8000

User-Agent: Jetty/9.1.5.v20140505

Content-Length: 649

{"notifications":[{"notificationType":"locationupdate","subscriptionName":"LAPTOP ON DONGLE","entity":"WIRELESS_CLIENTS","deviceId":"fb:ff:ef:5d:ff:ff","lastSeen":"2016-06-11T08:43:55.658+0100","eventId":24764543,"locationMapHierarchy":"DevNetCampus>DevNetBuilding>DevNetZone","locationCoordinate":{"x":53.728783,"y":56.867233,"z":0.0,"unit":"FEET"},"geoCoordinate":{"latitude":-999.0,"longitude":-999.0,"unit":"DEGREES"},"confidenceFactor":112.0,"apMacAddress":"00:2b:01:00:09:00","associated":true,"ipAddress":["10.10.20.223"],"ssid":"test","band":null,"floorId":723413320329068590,"batteryInfo":null,"vendorData":null,"timestamp":1465631035658}]}"

received ""

Best regards,

Ricson

matfarre
Frequent Contributor

Hi Ricson,

The setup guide is correct for using the DevNet CMX sandbox, since your residential gateway (DSL modem, Cable modem) must deal with NAT and port forwarding.  If you have a CMX on the same subnet (192.168.0.0, netmask 255.255.255.0) as in diagram then direct communication is possible, no port forwarding necessary.  If you have two routers then you may have two different subnets, say 192.168.0.0 and 192.168.1.0 both netmask 255.255.255.0, or configured on the same subnet.  In both instances direct communication is possible, no port forwarding necessary.

Regarding duplicate messages, stop after 2-3 minutes, and connection reset by peer error, I will have to investigate.  Also, the MSE 8 sandbox version 8.0.120 is due for upgrade, since 130 is available now, and 140 soon.  So, first thing I'll do is upgrade the sandbox to 130.  I'll let you know when it is complete.

Thanks,

Matt

matfarre
Frequent Contributor

Ricson,

Regarding duplicate notifications, there is a bug identified in 10.2.2.  Expect this to be fixed in the final release of 10.2.2 or bug fix package soon after.

Thanks,

Matt

Bug Id: CSCuy12468

Title: 10.2.2: Duplicate location notifications

Status: Open

Severity: 3 Moderate

Description: *

Symptom: The same location notification sent twice.

Conditions: None

Workaround: None

Further Problem Description:

Last Modified: 13-JUN-2016

Known Affected Releases: 10.2(2.0)

Thanks for the update Matthew.

Does this fix also solve the problem where I get null packets?

These are the 2 things I am experiencing right now using the stream both on sandbox and on actual CMX server.

1. Using the sample code from the API docs without modification will give me "connection reset by peer" error in Python.

2. After modification, script can now run for longer periods but I get around 5% of null packets. This was why I was asking if CMX has a logic for telling socket that the stream has ended like EOF or some other invisible character.

Best regards!

Ricson

Matthew Farrell wrote:

Ricson,

Regarding duplicate notifications, there is a bug identified in 10.2.2.  Expect this to be fixed in the final release of 10.2.2 or bug fix package soon after.

Thanks,

Matt

Bug Id: CSCuy12468

Title: 10.2.2: Duplicate location notifications

Status: Open

Severity: 3 Moderate

Description: *

Symptom: The same location notification sent twice.

Conditions: None

Workaround: None

Further Problem Description:

Last Modified: 13-JUN-2016

Known Affected Releases: 10.2(2.0)

Hi Matthew,

I had found the solution to the problem of the socket receiving blanks and connection reset by peer issue.

I am no longer receiving blanks records. May ask for your help to update the sample accordingly so as not to trouble future customers who plan to use the sample.

Below is the edited version that works best for me.

Sample code

#!/usr/bin/python

import socket

import sys

import os

# this listens to notification stream from MSE and displays results in console.

# Create a TCP/IP socket

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

# Bind the socket to the address given on the command line

server_address = ("0.0.0.0", 9107)                                # change this to your ip address

# Section below removes previous instances of sockets running on the machine

try:

    os.unlink("0.0.0.0")                #change to your address without port (not a tuple)

except OSError:

    if os.path.exists("0.0.0.0"):            #change to your address without port (not a tuple)

        raise

#bind socket after old instance is removed.

sock.bind(server_address)

print >>sys.stderr, 'using %s port %s' % sock.getsockname()     # use this print statement for python2.7

sock.listen(1)

while True:

   print >>sys.stderr, 'waiting for a connection'               # use this print statement for python2.7

   connection, client_address = sock.accept()

   try:

       print >>sys.stderr, 'client connected:', client_address   # use this print statement for python2.7

       while True:

           data = connection.recv(1024)                           # change this value, based on your needs.

           if data is '' :                      #connection reset by peer no longer happens when code is written this way.

               connection.sendall(data)

           else:

           print >>sys.stderr, 'received "%s"' % data            # use this print statement for python2.7

               break

   finally:

       connection.close()

Thank you for the fast response

How do I access cmxctl on the sandbox, rather how do I access CLI to execute cmxctl? Do I need to install something on my server?

I am currently using a different machine now that is connected to a public wifi.

I am still unable to get connection from CMX cloud. Do I need to setup portfowarding as well even if I am connected to a public wifi?

Finally, the goal of what I am doing is to just get connection from any CMX servers moving forward. Therefore this laptop/server/host will almost always never connect to a router/firewall.

Also, would like to clarify that the IP that I should put in the python code is the IP that I get from ifconfig command is that right?

Best regards,

Ricson

matfarre
Frequent Contributor

Ricson,

The DevNet CMX sandboxes are not accessible directly by non-admin users.

Note from Cisco Connected Mobile Experiences (CMX) Command Reference Guide, Release 10.2 - Cisco - In the context of Cisco CMX, in order to access the CLI, use either the console to the virtual machine (VM) or an appliance or the Secure Shell (SSH) to the server when it has IP connectivity.

Regarding most residential networks, port forwarding will need to be configured at the residential gateway/router (DSL modem, cable modem) firewall security settings.

Yes to the last question, the server_address is the ifconfig ip address interface eth0.

Matt


matfarre
Frequent Contributor

Hi Ricson,

Attempting to understand your question further, a VPN to your corporate network and MSE server would solve your problem with remote connectivity.

Also, setting up your own MSE on a VM is free to try, and only requires a single AP to start test/demo.  Cisco has sources for discounted hardware for developers, search on cisco.com, such as The Standard Not for Resale (Standard NFR) Program, and Meraki CMX Try Cisco Meraki Cloud Networking.

Another option is CMX Cloud Cisco CMX Cloud and dCloud :: Welcome to Cisco dCloud :: using your own Cisco router and AP.

Matt

Content for Community-Ad

This widget could not be displayed.