Introduction As a person working with Cisco's portfolio of IoT products you have probably realized that some new platforms are ditching away the old RJ45-based RS232 serial communication console in favour of the newer USB-based serial communication port. If this a good news for technicians as they are now able to connect a computer straight to the router console using just a mini-USB cable, this is not such a great news for those who need to operate serial connections remotely.   The purpose of a commserver is to map a serial console port to TCP port, so that using Telnet to the commserver one can actually talk directly to the equipment console port. This is very handy to have out-of-band management this way, as it works even when the main network is down, an access-list has been misconfigured, a router is stuck in ROMMON mode and plenty other hair pulling scenario.   With the classic RJ45 serial it was a simple task to use a router with a number of serial lines and connect them to console ports for out of band access. But you can't do that with USB... unless you want to use an active USB-to-RJ45 converter, which is messy and they are not that easy to find.    Proposed Solution    I had an old Raspberry Pi 2B+ lying around that I wanted to leverage, but it only has USB ports. You have four USB ports as standard, but I wanted more therefore added an additional board called UUGear BIG7 USB HUB but I'm sure other alternatives will work equally well. The purpose is really to extend the number of USB ports available from 4 up to 11.   Making serial USB ports predictable and persistent   I have used Raspbian as a base operating system, and on my setup and by default the first USB-to-serial connection will be device /dev/ttyACM0. This is not desirable! What we want is to be able to predict which Cisco device console is connected to which USB port, to map a predictable serial device name and ultimately map it to a specific TCP port. For example, if we connect a gateway console to the first USB, we want to map it to /dev/csco1, and to TCP/2001. We will do this my modifying the udev rules, but first we need to uniquely identify the USB ports.   To identify how your ports are mapped: plug in a Serial USB console and watch using "dmesg" on which port this new serial connection has been mapped. For example in my case it shows ttyACM0:   [16201.694459] usb 1-1.3.1: new full-speed USB device number 17 using dwc_otg [16201.851950] usb 1-1.3.1: New USB device found, idVendor=05a6, idProduct=0009, bcdDevice= 0.00 [16201.851975] usb 1-1.3.1: New USB device strings: Mfr=1, Product=2, SerialNumber=0 [16201.851987] usb 1-1.3.1: Product: Cisco USB Console [16201.851997] usb 1-1.3.1: Manufacturer: Cisco [16201.883380] cdc_acm 1-1.3.1:1.0: ttyACM0: USB ACM device Now that I know the device name is ttyACM0, let's check for some unique identifier using:   udevadm info --name=/dev/ttyACM0 --attribute-walk You should see a long list of attributes but the beginning is what is interesting for us. It could be something like:   looking at device '/devices/platform/soc/20980000.usb/usb1/1-1/1-1.3/1-1.3.1/1-1.3.1:1.0/tty/ttyACM0': KERNEL=="ttyACM0" SUBSYSTEM=="tty" DRIVER=="" [...] looking at parent device '/devices/platform/soc/20980000.usb/usb1/1-1/1-1.3/1-1.3.1': KERNELS=="1-1.3.1" SUBSYSTEMS=="usb" DRIVERS=="usb" ATTRS{manufacturer}=="Cisco" Since there can only be one USB driver in 1-1.3.1 this is enough for us to write the rules. Try other ports and see how the numbers are changing, for me this was the first USB port and then it goes 1-1.3.2, 1-1.3.3, ... 1-1.3.7. The udev mapping can be done by editing (or creating) the file in /etc/udev/rules.d/99-usb-serial.rules with the following content:   SUBSYSTEM=="tty",DRIVERS=="usb",KERNELS=="1-1.3.1",SYMLINK+="csco1" SUBSYSTEM=="tty",DRIVERS=="usb",KERNELS=="1-1.3.2",SYMLINK+="csco2" SUBSYSTEM=="tty",DRIVERS=="usb",KERNELS=="1-1.3.3",SYMLINK+="csco3" SUBSYSTEM=="tty",DRIVERS=="usb",KERNELS=="1-1.3.4",SYMLINK+="csco4" SUBSYSTEM=="tty",DRIVERS=="usb",KERNELS=="1-1.3.5",SYMLINK+="csco5" SUBSYSTEM=="tty",DRIVERS=="usb",KERNELS=="1-1.3.6",SYMLINK+="csco6" SUBSYSTEM=="tty",DRIVERS=="usb",KERNELS=="1-1.3.7",SYMLINK+="csco7"  Make sure that you restart udev with:   sudo /etc/init.d/udev restart From this point on, when a serial-to-usb device is connected to USB port 1, it will be mapped to a serial device /dev/csco1, the USB port 2 will be mapped to /dev/csco2, and so on. This will make it very predictable for the next step.     Using ser2net to map serial ports to Telnet TCP ports   Now in order to map specific USB / Serial ports to TCP / Telnet connections there is a very handy software called ser2net. We are very lucky as this is in the Raspbian repository and it can therefore be installed with:   sudo apt install ser2net   Ater the previous where we made the serial ports predictable this will be very simple. One can configure ser2net by editing the file /etc/ser2net.conf with the following:   2001:telnet:600:/dev/csco1:9600 8DATABITS NONE 1STOPBIT -XONXOFF RTSCTS banner 2002:telnet:600:/dev/csco2:9600 8DATABITS NONE 1STOPBIT -XONXOFF RTSCTS banner 2003:telnet:600:/dev/csco3:9600 8DATABITS NONE 1STOPBIT -XONXOFF RTSCTS banner 2004:telnet:600:/dev/csco4:9600 8DATABITS NONE 1STOPBIT -XONXOFF RTSCTS banner 2005:telnet:600:/dev/csco5:9600 8DATABITS NONE 1STOPBIT -XONXOFF RTSCTS banner 2006:telnet:600:/dev/csco6:9600 8DATABITS NONE 1STOPBIT -XONXOFF RTSCTS banner 2007:telnet:600:/dev/csco7:9600 8DATABITS NONE 1STOPBIT -XONXOFF RTSCTS banner   Make sure to restart ser2net for the configuration to take effect (sudo systemctl restart ser2net).   Testing    You can then telnet straight to a given console by using the corresponding TCP port number, for example for the console connected to my first USB port:   etychon@ETYCHON-M-D2ZR~ telnet 192.168.2.7 2001 Trying 192.168.2.7... Connected to 192.168.2.7. Escape character is '^]'. ser2net port 2001 device /dev/csco1 [9600 N81] (Debian GNU/Linux) ic3k>   That's it - now I'm on the IC3000 serial-USB console using Telnet on a Raspberry Pi!   Thank you for reading, and let us know in the comments should you have any other ways you are willing to share?   ... View more

If you are required to retrieve the gateway serial number from an IOx application, environment variables set by the Cisco Application Framework (CAF) are automatically set in the execution environment in the /data/.env file.   You can access this file from your IOx application directly, or invoke it to set all the variables as environment variables, such as:   / # source /data/.env / # echo $CAF_SYSTEM_SERIAL_ID FGL23362022 To see all the variables available in an IOx container, this can be done by accessing the console with the ioxclient utility. The following command (to get console to the application called iox-serial) and then list all the variables:     etychon@squeeze:~/iox/iox-serial$ ioxclient app con iox_serial Currently active profile : default Command Name: application-console Console setup is complete.. Running command : [ssh -p 2222 -i iox_serial.pem appconsole@192.168.2.231] / # cat /data/.env #!/bin/sh export CAF_APP_PERSISTENT_DISK_SIZE_KB=102400 export CAF_APP_MEMORY_SIZE_KB=51200 export CAF_SYSTEM_PRODUCT_ID=IR829M-2LTE-EA-EK9 export CAF_SYSTEM_NAME=IR800 export CAF_SS_PORT=9443 export CAF_APP_APPDATA_DIR=/data/appdata export CAF_APP_CONFIG_DIR=/data export CAF_SS_IP_ADDR=192.168.1.6 export CAF_APP_CORE_DIR=/tmp/cores export CAF_APP_LOG_DIR=/data/logs export CAF_SYSTEM_SERIAL_ID=FGL23362022 export CAF_APP_PERSISTENT_DIR=/data export CAF_APP_CPU_SHARES=342 export CAF_APP_ID=iox_serial export CAF_APP_USERNAME=root export HOST_DEV1=/dev/ttyACM0 export CAF_SYSTEM_UUID=7581eb7b-8c04-4412-b2bb-503598f06a72/ To learn more about these variables check the "Environment Variables" section of Cisco IOx DevNet Documentation: https://developer.cisco.com/docs/iox/#!application-development-concepts/application-development-concepts ... View more

How to I get the gateway Cellular RSSI from Cisco IOx ? If you have an IOx application that requires tp capture the cellular modem signal strength (for example to include in telemetry data along with GPS position and map signal with location), you will find that there is no direct way to get it from IOx. The reason is that the application hosting environment is virtualized and therefore Cisco IOS metrics are not directly accessible.   Simple Network Management Protocol (SNMP) which is an IOS feature can be leverage to exposed this variable or any other SNMP variable. SNMP can be easily configured on the gateway with just a few lines:   On a Cisco IR829 with guest-os running in the 192.168.1.0/24 range, where 192.168.1.1 is being assigned to the virtual Gigabit Ethernet 5 interface, then this is what it takes to give SNMP access to all the virtual instances running on the gateway:   IR829(config)# access-list 10 permit 192.168.1.0 0.0.0.255 IR829(config)# snmp-server community public RO 10 The IOx application can then poll the right SNMP OIDs straight from the gateway, and this OID may change depending on the particular model. Always refer the device's MIB.   For example in Python to poll both cellular interface's RSSI on a Cisco IR829M:   from pysnmp.entity.rfc3413.oneliner import cmdgen import time SNMP_HOST = '192.168.1.1' SNMP_PORT = 161 SNMP_COMMUNITY = 'public' cmdGen = cmdgen.CommandGenerator() errorIndication, errorStatus, errorIndex, varBinds = cmdGen.getCmd( cmdgen.CommunityData(SNMP_COMMUNITY), cmdgen.UdpTransportTarget((SNMP_HOST, SNMP_PORT)), '1.3.6.1.4.1.9.9.661.1.3.4.1.1.1.20', '1.3.6.1.4.1.9.9.661.1.3.4.1.1.1.21' ) for name, val in varBinds: print('%s = %s' % (name.prettyPrint(), val.prettyPrint())) You should see the RSSI signal (in dBm) returned like so:   SNMPv2-SMI::enterprises.9.9.661.1.3.4.1.1.1.20 = -128 SNMPv2-SMI::enterprises.9.9.661.1.3.4.1.1.1.21 = -128   ... View more

Symptoms You have confirmed that EFM is running, and you can open a broker connection using port 8080 or 8443 from the EFM server. However, from an external broker, not hosted on the same machine, the connection is refused. Diagnosis Is this is your first external broker then probably EFM server is not listening for connection requests on outside interfaces. From a remote destination you can try to telnet to either 8080 (http) or 8443 (https) ports: if the connection is refused, this is likely your problem. It might also be a firewall or access-list problem, but then this is a network issue, not an EFM issue. Solution By default ans starting with EFM 1.6 the EFM broker only listens to the local, loopack interface (127.0.0.1). To confirm this is the case, check the configuration file:   efm-server:~ $ grep "host" /opt/cisco/kinetic/efm_server/server.json "host": "127.0.0.1",   The "host" option tells EFM on which interface it should be listening for incoming connections. 127.0.0.1, called the loopback address, will only permit connection originated by EFM server itself to be successful. Should you want to open this up to the outside world, edit the file and replace 127.0.0.1 (loopback) by 0.0.0.0 (all interfaces):   efm-server:~ $ vi /opt/cisco/kinetic/efm_server/server.json   You need to restart the EFM server for this change to take effect:   efm-server:~ $ sudo systemctl restart efm-server ... View more

Symptoms EFM broker connection to upstream EFM server is failing. You have confirmed that the IP connectivity to the destination port number (tcp/443 by default) is open, yet the connection does not establish from the EFM broker to the EFM server. You have verified that both the broker and the server have their time synchronized. Diagnosis Upon log inspection you can see a message "Verification of certificate failed" and "X509 verification result: certificate has expired and depth 0". It might be that when the certificate was created your system time was not correct. Solution Log in to EFM server as user "efm", and enter the directory where the EFM server SSL certificate is stored:   cd /etc/cisco/kinetic/ssl/efm-server   Before generating new certificates, makes sure this is indeed the issue. Check your current certificate expiration date with: openssl x509 -in selfsigned.cert -text | grep Not   First we need to generate a new server key, remember the passphrase used to encrypt that key: openssl genrsa -des3 -out server.pass.key 2048   Remove the passphrase from that key: openssl rsa -in server.pass.key -out selfsigned.key   Create a Certificate Signing Request: openssl req -new -key selfsigned.key -out selfsigned.csr   Generate a new certificate using the CSR and key: openssl x509 -req -sha256 -days 365 -in selfsigned.csr -signkey selfsigned.key -out selfsigned.cert   Check if your key is valid: openssl rsa -in selfsigned.key -check   Check if your certificate is valid (pay attention to dates): openssl x509 -in selfsigned.cert -text   Additional notes:   The dglux server (the Dart server) needs the certificate at (citing the default configuration in server.json)   "certName": "/etc/cisco/kinetic/ssl/efm-server/selfsigned.cert", "certKeyName": "/etc/cisco/kinetic/ssl/efm-server/selfsigned.key",   From 1.6 on the connection should be made via the EFM manager (openresty). This has the certificates configured in file "/usr/local/openresty/nginx/conf/conf.d/efm-manager.conf" which contains:     ssl_certificate /etc/cisco/kinetic/ssl/efm-manager/nginx-selfsigned.crt;   ssl_certificate_key /etc/cisco/kinetic/ssl/efm-manager/nginx-selfsigned.key;   You may need to restart EFM server and broker to wipe out any remains of a previous cached certificate.   ... View more

The purpose of this article is to sort out and clarify how licenses are effectively working in Cisco Kinetic Edge & Fog Processing Module (EFM). It does not address ordering licenses, prices, or anything related to the Cisco Kinetic Cloud (GMM and DCM).    Type of License   There is fundamentally only one license in Cisco EFM, but it can activate two separate features: the EFM broker (the logic engine that runs the data flow amongst other things), and   the visualizations (DGLux5). Initial license: during the EFM installation process, the installer package will automatically create a license for the EFM broker. This means that a new license will never need to be generated again unless the graphical data flow editor is being used.   Since there is only one license, if you ever want to start using the data visualisation, you'll be offered an option to request a trial version. If your instance has internet access it is granted immediately using an on-line activation process. However now the entire license will now be 180-days. After 180-days your license will expire and you won't have a broker, you won't have a data flow editor, and you won't have visualizations.    Workaround 1: Use separate users for EFM broker and visualization During the installation, install only the broker as one user, and the visualization as another user. This way you have separate licenses per user, and expiration of one will not affect others. This is described in: https://www.cisco.com/c/dam/en/us/td/docs/cloud-systems-management/cisco-edge-fog-fabric/1_2_0/Kinetic-EFM-1-2-0_Install_EFM_DGLux5.pdf Workaround 2: Reinstall EFM. This very effective approach will wipe all of your installation, including tokens, data flow, etc... unless you backup them manually. Since the WebUI is not working this is quite a long process that is not recommended if all you want is a new working license. However if you do re-install make sure you separate EFM broker and visualization components on two users as see before. Workaround 3: Regenerate License Remove the user's ~/.dglogik directory, and perform an in-place upgrade of EFM. You will essentially reinstall the same version of EFM, but the installer will generate a new EFM broker license. You'll get access to the data flow editor, and none of your data will be lost.   ... View more