Why do we need Edge Data Processing?
EFM is based on open source DSA Project
DSA Architecture
Broker Highlights
* Transparent multi-hop routing between message brokers
* Reliable and flexible data delivery between nodes
* Runs at the edge and has a very small footprint
* Fine grained publish–subscribe and request-reply message exchange
* Providing guaranteed QoS delivery
* Persistence (on/off) – prevents data loss when broker is disconnected (in-disk store in case of broker outage)
* Durability (on/off) – protects against data loss when client is disconnected (in-memory store and forward)
C++ and Dart based Brokers
* DART (open source) and C++ (Cisco developed) are functionally compatible
* DART Broker was higher performance than the C Broker. C++ Broker now replaces C Broker.
* The C++ Broker is a multi-threaded high-performance broker with very low footprint in order to leverage the multi-core capability of different platforms.
* The C++ broker scales and performs so well that we recommend to use it on all other levels, except when UI is required (System Monitor, DGlux5, System Administrator, Asset Manager), in a multi-tier architecture.
QoS (Quality of Service)
* QoS is defined by the subscriber
* QoS (Quality of service) – refers to the delivery of messages from the source to the subscriber.
+ QoS 0: No guarantee
+ QoS 1: In memory queue (circular buffer) of size 1 that does not survive client disconnect or broker restart.
Also known as “Volatile”.
+ QoS 2: In memory queue (circular buffer) up to a configured maximum size that survives a client disconnect, but not a broker restart.
Also known as “Durable.”
+ QoS 3: On disk queue (circular buffer) up to a configured maximum size that survives both a client disconnect and a broker restart.
Also known as “Durable and Persistent.”
QoS Considerations
* Performance impact every time you increase your QoS level
* eMMC drives can easily wear out (burn through the lifecycle)
* QoS usage best practices
* Use QoS 0 for UIs
* Use QoS 1 for most telemetry
* Use QoS 2 for important telemetry
* Use QoS 3 for alarms
DSLinks
* Enable communication with edge devices and message brokers
* Data acquisition
* Communicate to the device using its native language
* Bridges from device’s native protocol into EFM protocol
* Originators of data streams
* Always pass data upstream
* May pass commands downstream
* Support for multiple languages
* Dart, Java, JavaScript, Python, Ruby, C, C++, Scala
* EFM base links include:
* System, Dataflow, DQL, and ParStream
Kinetic Connector Catalog
https://developer.cisco.com/docs/kinetic-connector-catalog/
EFM Components
