Generic Protocol Adapter
The Generic Protocol Adapter enables retrieving data from industrial assets that expose data using non-standard protocols. It decodes received messages into objects understandable by the Gateway Agent.
The Generic Protocol Adapter is modular and extensible. Its core part is the data translation engine. Implementations for particular protocols are added as plugins.
Currently, the Generic Protocol Adapter supports:
Data sources:
- TCP data source
- Data source for Kessler's Stream2 protocol
Data decoders:
- Decoder supporting fixed-size binary frames
- Decoder with a frame begin and end sequence detection
- Decoder for Kessler's Stream2 protocol
Architecture Diagram
The diagram below shows modules of the Generic Protocol Adapter responsible for the data flow:
Terminology
Module | Description |
---|---|
Data Source | Downstream source of data, exposed by the device's interface, providing data to the decoder, e.g.: a socket, a pipe or a file. |
Decoder | Message processor that translates from a protocol data structure to the Gateway Agent's data structures. |
Message Sender | Interface for upstream message sending. Data understandable by the Gateway Agent is provided by the decoder. |
Usage
Normally, the output of the Generic Protocol Adapter requires further processing. It can be redirected to the local Rule Engine or the GWA Analytics module. It can be also stored on a disk for offline analysis using the Storage Service.
Configuring the Generic Protocol Adapter
There are two configuration files of the Generic Protocol Adapter:
Deployment configuration file - Configuration of the MQTT connection, PMQ connection and health monitoring.
Application configuration file - Protocol-specific configuration of a data source and a decoder.
When you modify the Generic Protocol Adapter configuration, restart the adapter to apply the changes by means of the following command:
systemctl restart gwa-generic-protocol-adapter-cpp
The adapter configuration is then updated.
Deployment Configuration File
The deployment configuration file of the Generic Protocol Adapter is located at:
/etc/relayr/gwa-generic-protocol-adapter-cpp/gwa-generic-adapter-config.json
The following configuration settings are available:
transport
- Communication channel used by the adapter. Available options:mqtt
orpmq
. By default,mqtt
.mqtt_connection
- Connection settings of the MQTT message broker:
Configuration Setting | Description | Default Value |
---|---|---|
host | IP address of the MQTT message broker. | localhost |
port | Port number for connecting to the MQTT broker. | 1883 |
keepalive | Keepalive value of the MQTT connection. | 20 |
max_inflight_messages | Maximum number of QoS 1 or 2 messages that can be in the process of being transmitted simultaneously. This includes messages currently going through handshakes and messages that are being retried. If set to 1 , this guarantees in-order delivery of messages. | 100 |
qos_levels | Quality of Service level for each traffic type. Currently, this setting is configurable for measurements only. Other Gateway Agent message types are not supported by the Generic Adapter. Messages sent to topics other than measurements are always sent with the QoS 2 . See here for information on how to send Gateway Agent-compatible measurements. | measurements: 0 , alerts: 1 . configuration: 1 , metadata: 1 |
health_monitor
- Health monitoring settings:
Configuration Setting | Description | Default Value |
---|---|---|
heartbeat_interval | Time interval at which the periodic heartbeat messages are sent, given in seconds. | 60 |
pmq_connection
- Configuration of the High Speed Bus (PMQ) connection. Configure this section to use thepmq
transport
:
Configuration Setting | Description | Default Value |
---|---|---|
output_queues | List of output PMQ queues, where the adapter reports all data, except for health monitoring messages, which are always reported via MQTT. Queue names must begin with / . | ["/gpa_out_queue"] |
input_queues | List of PMQ queues to monitor for inbound messages, e.g. configuration update tasks. Queue names must begin with / . | ["/gpa_in_queue"] |
message_size | Maximum size of a PMQ message, given in bytes. | 8192 |
message_count | Maximum number of messages in a PMQ queue. | 10 |
For an example of the deployment configuration file, see here.
Application Configuration File
The application configuration file customizes the Generic Protocol Adapter's options for a specific protocol.
The file allows you to configure a data source (a downstream source of data) and a decoder (a message processor). It has the following structure:
{
"data_sources": [],
"data_decoders": []
}
The file is located at:
/etc/relayr/gwa-generic-protocol-adapter-cpp/gwa-generic-protocol-adapter-application-config.json
This file is protocol-specific. See sections below for options available for the current version of the Generic Protocol Adapter:
For an example of the application configuration file, see here.
Data Sources
Configuring a TCP Data Source
This section explains how to configure the application configuration file to support TCP data sources.
In the data_sources
section of the configuration file, configure the following settings:
Configuration Setting | Description | Example of Value |
---|---|---|
connection_id | ID of the connection specifying which data source is connected to which decoder. | Example |
name | Name of the data source. Each supported data source has a predefined value you can enter. | GA::DataSourceTCP |
config.host | IP address of the TCP Client. | 127.0.0.1 |
config.service | Port number or service name of the TCP Client. | 1337 |
config.retry_ms | Number of milliseconds after which the lost connection is retried. | 100 |
config.buffer_size | Buffer size for the TCP socket. | 256 |
Example:
{
"data_sources": [
{
"connection_id": "Example",
"name": "GA::DataSourceTCP",
"config": {
"host": "127.0.0.1",
"service": "1337",
"retry_ms": 100,
"buffer_size": 256
}
}
Configuring a Data Source for Kessler Stream2 Protocol
This section explains how to configure the application configuration file to support a data source for Kessler's Stream2 protocol. In this data source, the adapter communicates via a character device with up to four sensor boards.
In the data_sources
section of the configuration file, configure the following settings:
Configuration Setting | Description | Example of Value |
---|---|---|
connection_id | ID of the connection specifying which data source is connected to which decoder. | Kessler-relayr_stream2 |
name | Name of the data source. Each supported data source has a predefined value you can enter. | Elevator::DataSourceKessler |
config.dev | Path to the character device, typically /dev/device_name . | /dev/ttyUSB5 |
config.baudRate | Transmission speed, given in bauds. | 500000 |
config.req_freq | Frequency at which the gateway attempts to send requests, given in Hz. | 200 |
config.buffer_size | Size of the internal buffer, given in bytes. | 1024 |
config.active_slaves | Array of integers defining which slaves are addressed. Supported values: 1 , 2 , 3 , 4 . | [1, 2] |
Example:
{
"data_sources": [
{
"connection_id": "Kessler-relayr_stream2",
"name": "Elevator::DataSourceKessler",
"config": {
"dev": "/dev/ttyRS485",
"baudRate": 500000,
"req_freq": 200,
"buffer_size": 1024,
"active_slaves": [
1,
2
]
}
}
]
Data Decoders
Configuring a Decoder for Fixed-size Binary Frames
This section explains how to configure the application configuration file to support a decoder for fixed-size binary frames.
In the data_decoders
section of the configuration file, configure the following settings:
Configuration Setting | Description | Example of Value |
---|---|---|
connection_id | ID of the connection specifying which data source is connected to which decoder. | Example |
name | Name of the data decoder. Each supported decoder has a predefined value you can enter. | GA::DecoderFixedFrame |
config.frame_endianness | Arrangement of bytes while reading data from a data stream. little sets a little-endian byte arrangement, big sets a big-endian byte arrangement. See here for more information. | little |
config.frame_size | Size of a frame, in bytes. | 28 |
config.frame | Data type and a topic to which data is reported, specified for each byte of a frame. Here, you can also define which bytes from a frame to skip and what is the expected field value. | See below for more information. |
Frame Configuration
The data_decoders.config.frame
setting of the application configuration file allows you to specify a data type and a topic where values are reported, for each byte of a frame.
Optionally, you can also define:
Based on the topic format, the Generic Protocol Adapter deduces the message type. To report data as Gateway Agent-compatible measurements, the topic must have the following format:
<prefix>/<did>/measurements/<id>
, for examplev1/sb/abcd/measurements/Temperature
.
The following data types are supported:
uint8
uint16
uint32
uint64
float
double
int8
int16
int32
int64
The Generic Protocol Adapter verifies if you have provided MQTT topics in the correct format. Topics must comply with the MQTT specification and cannot contain
#
and+
characters.
Frame Configuration Example
"frame":
[
["uint8", "/Counter"],
2,
["uint32", "Device/TimeStamp"],
["uint64", "Random"],
["uint8", "v1/sb/abcd/measurements/Temperature"],
["float", "Float"],
["double", "Double"]
]
In this example, values of the byte described by ["uint8", "/Counter"]
are of the uint8
type and are reported to the /Counter
topic. Values of the byte described by ["uint8", "Temperature"]
are of the uint8
type and are reported to the v1/sb/abcd/measurements/Temperature
topic.
See below for examples of output values.
Skipping Bytes in the Frame Configuration
In the frame configuration, you can optionally define which bytes to skip while reading data.
To skip some bytes, enter a number in a desired position in the list, instead of a value type and a topic. The number specifies how many bytes to skip.
In the example above, 2
means that the Generic Protocol Adapter skips two bytes after the ["uint8", "/Counter"]
byte.
Defining the Expected Field Value
In the frame configuration, you can optionally configure the expected value for a field.
To do so, define the expected value as the third field in the frame
array, after the value type and the topic.
Example:
"frame":
[
["uint8", "/Counter/val"],
2,
["uint32", "TimeStamp"],
["uint64", "Random"],
["uint8", "Zero", 0],
["float", "v1/sb/abcd/measurements/Multiple"],
["double", "Double", 0.2]
]
This example contains two expected values:
0
, as configured in["uint8", "Zero", 0]
.0.2
, as configured in["double", "Double", 0.2]
.
If the received value is different from the expected value, the Generic Protocol Adapter logs an error that informs what the expected value and the received value were.
MQTT Output Example
The Generic Protocol Adapter reports data as measurements to topics specified in the frame configuration. Each reported measurement has a value, a timestamp and an ID.
The measurement ID is the last part of the topic to which the measurement is reported. For example, for topic
v1/sb/abcd/measurements/Temperature
, the measurement ID isTemperature
.
Here is an example of a reported Temperature
measurement:
v1/sb/abcd/measurements/Temperature {"value": 16, "id": "Temperature", "ts": 15000343}
For each byte of a frame, the Generic Protocol Adapter converts values read from a device into value types specified in the configuration (e.g. uint8
or float
).
The table below is an example of how the adapter converts values received in one frame, as configured in the frame configuration example above.
The frame size in this example is
28 bytes
. Consecutive bytes of a frame read from a device are given in the third table column.
Topic | Output value | Bytes read from a device (Hex format) |
---|---|---|
/Counter | 1 | 01 |
02 03 | ||
Device/TimeStamp | 67438087 | 04 05 06 07 |
Random | 8834916224013 | 08 09 0A 0B 0C 0D 0E 0F |
v1/sb/abcd/measurements/Temperature | 16 | 10 |
Float | 1.15232E-28 | 11 12 13 14 |
Double | 4.3003407051602E-207 | 15 16 17 18 19 1A 1B 1C |
Endianness Configuration
The Generic Protocol Adapter allows you to configure the arrangement of bytes read from a data stream.
To set the endianness, configure the data_decoders.config.frame_endianness
setting in the decoder configuration with one of the following values:
big
- big-endian byte arrangement, which places the most significant byte first and the least significant byte last in the result value.little
- little-endian byte arrangement, in which the first byte of the data payload becomes the least significant byte in the result value.
The
frame_endianness
configuration is mandatory.
If your platform uses a different format for saving floating point numbers than for saving integers, the floating point numbers won't be interpreted correctly.
Configuration Example
{
"data_decoders": [
{
"connection_id": "Example",
"name": "GA::DecoderFixedFrame",
"config":
{
"frame_endianness": "little",
"frame_size" : 28,
"frame":
[
["uint8", "Counter"],
2,
["uint32", "TimeStamp"],
["uint64", "Random"],
["uint8", "v1/sb/abcd/measurements/Temperature"],
["float", "Float"],
["double", "Double"]
]
}
}
]
Configuring a Decoder with a Frame Beginning and End Sequence Detection
This section explains how to configure the application configuration file to support a decoder that detects the beginning and end of a frame, based on sequences of bytes provided in the configuration file.
In the data_decoders
section of the configuration file, configure the following settings:
Configuration Setting | Description | Example of Value |
---|---|---|
connection_id | ID of the connection specifying which data source is connected to which decoder. | Example |
name | Name of the data decoder. Each supported decoder has a predefined value you can enter. | GA::DecoderBeginEndSequence |
config.frame_begin | Sequence of bytes at the frame beginning, at least 1 byte. | [0, 1] |
config.frame_end | Sequence of bytes at the frame end, at least 1 byte. | [255, 255] |
config.mqtt_topic | Topic to which data is reported. | v1/sb/deadbeef-1234-5678-abcd-deadbeef/array8/example |
config.buffer_size | Maximum size of messages in a buffer, given in bytes. | 1024 |
config.type | Format of the published data. Supported types: array8 and hex_string . If set to array8 , data is sent as a JSON array of bytes, e.g. [255,77] . If set to hex_string , data is sent as a string of bytes in the hex format, e.g. ff4d . | array8 |
Based on the provided sequences of bytes (frame_end
and frame_begin
settings), the Generic Protocol Adapter detects the beginning and end of a frame. All bytes that are between those two sequences (excluding the beginning and end sequences themselves) are sent to the configured mqtt_topic
. Depending on the configured data type
, the bytes are sent either as a JSON array or as a hex string.
A frame can have any length, but there is a limit to the buffer size, configured in the config.buffer_size
setting.
It is recommended that the buffer size is at least twice as big as the expected frame length, plus the beginning and end sequences.
Configuration Example
{
"data_decoders": [
{
"connection_id": "Example",
"name": "GA::DecoderBeginEndSequence",
"config": {
"frame_begin": [0, 1],
"frame_end": [255, 255],
"mqtt_topic": "v1/sb/deadbeef-1234-5678-abcd-deadbeef/array8/example",
"buffer_size": 1024,
"type": "array8"
}
}
Configuring a Decoder for Kessler Stream2 Protocol
This section explains how to configure the application configuration file to support a decoder for Kessler's Stream2 protocol.
In the data_decoders
section of the configuration file, configure the following settings:
Configuration Setting | Description | Example of Value |
---|---|---|
connection_id | ID of the connection specifying which data source is connected to which decoder. | Kessler-relayr_stream2 |
name | Name of the data decoder. Each supported decoder has a predefined value you can enter. | Elevator::DecoderKessler |
config | JSON object where you can configure objects named slave1 , slave2 , slave3 , slave4 , one for each sensor board, containing information about the reported data. | See below for more information. |
Slave Configuration
In the data_decoders.config
section of the application configuration file, you can configure objects named slave1
, slave2
, slave3
, slave4
, one for each sensor board, containing information about the reported data.
For each of the slaves, the following settings are available:
Configuration Setting | Description | Example of Value |
---|---|---|
samples | Number of data samples in a complex measurement. If set to 1 , simple measurements are sent. If you don't configure the samples setting, the adapter sends simple measurements, by default. You can override this global setting by configuring samples for individual measurements. | 200 |
In the slave configuration, include a JSON object for each measurement you want to configure.
Below is the list of supported measurement types. For detailed information about them, see here.
startByte
sensorId
firmwareRev
errorByte
humidity
temperature
pressure
tof
crc
endByte
For the object names listed above, you can configure the following settings:
Configuration Setting | Description | Example of Value |
---|---|---|
topic | String with an MQTT topic where the adapter reports data. | v1/sb/sensor_board_1/measurements/humidity |
samples | Optional setting. Number of data samples in a complex measurement. If set to 1 , simple measurements are sent. This setting overrides the global samples setting. | 100 |
acc
magnetic
gyro
For the object names listed above, you can configure the following settings:
Configuration Setting | Description | Example of Value |
---|---|---|
x | String with an MQTT topic where the adapter reports data from the X axis. | v1/sb/sensor_board_1/measurements/gyro.x |
y | String with an MQTT topic where the adapter reports data from the Y axis. | v1/sb/sensor_board_1/measurements/gyro.y |
z | String with an MQTT topic where the adapter reports data from the Z axis. | v1/sb/sensor_board_1/measurements/gyro.z |
samples | Optional setting. Number of data samples in a complex measurement. If set to 1 , simple measurements are sent. This setting overrides the global samples setting. | 100 |
magnetNode
- for this object name, you can configure the following settings:
Configuration Setting | Description | Example of Value |
---|---|---|
x | String with an MQTT topic where the adapter reports data from the X axis. | v1/sb/sensor_board_2/measurements/magnet.x |
y | String with an MQTT topic where the adapter reports data from the Y axis. | v1/sb/sensor_board_2/measurements/magnet.z |
z | String with an MQTT topic where the adapter reports data from the Z axis. | v1/sb/sensor_board_2/measurements/magnet.z |
t | String with an MQTT topic where the adapter reports data from the T axis. | v1/sb/sensor_board_2/measurements/magnet.t |
samples | Optional setting. Number of data samples in a complex measurement. If set to 1 , simple measurements are sent. This setting overrides the global samples setting. | 100 |
All the object names listed above are optional, so you can leave their configuration empty or skip them altogether.
Configuration Example
{
"data_decoders": [
{
"connection_id": "Kessler-relayr_stream2",
"name": "Elevator::DecoderKessler",
"config": {
"slave1": {
"samples": 200,
"startByte": {},
"humidity": {
"topic": "v1/sb/sensor_board_1/measurements/humidity"
},
"temperature": {
"topic": "v1/sb/sensor_board_1/measurements/temperature"
},
"acc": {
"x": "v1/sb/sensor_board_1/measurements/cacc.acc.x",
"y": "v1/sb/sensor_board_1/measurements/cacc.acc.y",
"z": "v1/sb/sensor_board_1/measurements/cacc.acc.z"
}
},
"slave2": {
"samples": 200,
"startByte": {},
"sensorId": {
"topic": "v1/sb/sensor_board_2/metadata/sensor_board_id"
},
"firmwareRev": {
"topic": "v1/sb/sensor_board_2/metadata/firmware_revision"
},
"magnetNode": {
"x": "v1/sb/sensor_board_2/measurements/magnet.x",
"y": "v1/sb/sensor_board_2/measurements/magnet.y",
"z": "v1/sb/sensor_board_2/measurements/magnet.z",
"t": "v1/sb/sensor_board_2/measurements/magnet.t"
},
"tof": {},
"crc": {},
"endByte": {}
}
}
}
]
}
Examples of Configuration Files
Deployment Configuration File
{
"transport": "mqtt",
"mqtt_connection": {
"host": "localhost",
"port": 1883,
"keepalive": 20,
"max_inflight_messages": 100,
"qos_levels": {
"measurements": 0,
"alerts": 1,
"configuration": 1,
"metadata": 1
}
},
"health_monitor": {
"heartbeat_interval": 60
},
"pmq_connection":
{
"output_queues": ["/gpa_out_queue"],
"input_queues": ["/gpa_in_queue"],
"message_size": 8192,
"message_count": 10
}
}
Application Configuration File
{
"data_sources": [
{
"connection_id": "Example",
"name": "GA::DataSourceTCP",
"config": {
"host": "127.0.0.1",
"service": "1337",
"retry_ms": 100,
"buffer_size": 256
}
}
],
"data_decoders": [
{
"connection_id": "Example",
"name": "GA::DecoderFixedFrame",
"config":
{
"frame_endianness": "little",
"frame_size" : 28,
"frame":
[
["uint8", "Counter"],
2,
["uint32", "TimeStamp"],
["uint64", "Random"],
["uint8", "v1/sb/abcd/measurements/Temperature"],
["float", "Float"],
["double", "Double"]
]
}
}
]
}