Azure Cloud Adapter
The Azure Cloud Adapter enables a communication between the Gateway Agent components and Azure Cloud. It is suitable for projects that involve onboarding devices to the relayr Cloud with help of IoT Hub.
The adapter supports device registration and the reporting of both simple and complex measurements, as well as alerts, configuration parameters, logs, and metadata. It also supports the handling of business parameters, commands, and package management.
By providing advanced data batching and caching capabilities, the adapter assures a reliable data exchange with the Cloud.
For a step-by-step guide on how to install the Azure Cloud Adapter, see the Installation Guide.
Configuring the Azure Cloud Adapter
There are two configuration files of the Azure Cloud Adapter:
Deployment configuration file - Communication settings of the Gateway Agent and Azure Cloud, including the Cloud connection, Gateway Agent MQTT and HTTP connections, health monitoring, data batching and caching.
Application configuration file - Configuration of devices connected to the adapter.
After changing the configuration settings, restart the relayr Cloud Adapter to apply the changes:
systemctl restart gwa-azure-cloud-adapter-cpp
For an example of the Azure Cloud Adapter configuration files, see here.
Deployment Configuration File
The deployment configuration file of the Azure Cloud Adapter is located at:
/etc/relayr/gwa-azure-cloud-adapter-cpp/gwa-azure-cloud-adapter-config.json
The file consists of several sections with the following configuration options available:
transport
- Communication channel used by the adapter. By default,mqtt
.
mqtt_connection
- configuration of the Gateway Agent's northbound interface MQTT broker:
Configuration Setting | Description | Default Value |
---|---|---|
host | Hostname or IP address of the Gateway Agent's northbound interface MQTT broker. | localhost |
port | TCP port number of the Gateway Agent's northbound interface MQTT broker. | 1883 |
keepalive | Keepalive value for the Gateway Agent's northbound interface MQTT connection. | 20 |
max_inflight_messages | The maximum number of QoS 1 or 2 messages that can be in the process of being transmitted simultaneously. | 100 |
qos_levels.measurements | Quality of Service level for measurements. | 0 |
qos_levels.alerts | Quality of Service level for alerts. | 1 |
qos_levels.configuration | Quality of Service level for configuration. | 1 |
qos_levels.metadata | Quality of Service level for metadata. | 1 |
cloud.connection
- configuration of the default connection to IoT Hub:
Configuration Setting | Description | Default Value |
---|---|---|
transport | Transport protocol used to communicate with IoT Hub. Available options: mqtt , websocket-mqtt , amqp , websocket-amqp , Setting it to websocket-mqtt may facilitate a communication through firewalls in case they interrupt the default mqtt transport. | mqtt |
host | Hostname or IP address of the default IoT Hub. You can override it for every device by setting a hub-id in the application configuration file or during the device provisioning. | relayr-test-hub.azure-devices.net |
port | TCP port of the default IoT Hub. | 8883 |
ca_file | Path to the Azure IoT Hub (public) X.509 certificate file | IoTHubRootCA.pem |
id_scope | Identifier of Device Provisioning Service to connect to. | 0ne000A72E1 |
use_compression | If set to true , the adapter compresses messages sent to and received from the Cloud. If set to false , the compression is disabled entirely. | true |
compression_level | Strength of data compression, between 0 and 22 . The 0 level still compresses the data. To disable the compression, set the use_compression setting to false . | 15 |
nb_configs_connection
- configuration of the HTTP connection to the Rule Engine and GWA Analytics to receive configuration updates of business parameters from the Cloud:
Configuration Setting | Description | Default Value |
---|---|---|
gwa_rule_engine.host | Host name or IP address of the Rule Engine's business-params REST endpoint. To disable the Rule Engine's interface, leave this value empty. | localhost |
gwa_rule_engine.port | Port number of the Rule Engine's business-params REST endpoint. | 25080 |
gwa_analytics.host | Host name or IP address of the GWA Analytics business-params REST endpoint. To disable the GWA Analytics interface, leave this value empty. | localhost |
gwa_analytics.port | Port number of the GWA Analytics business-params REST endpoint. | 27080 |
Configuration Setting | Description | Default Value |
---|---|---|
host | Hostname or IP address of the Rule Engine. | localhost |
port | TCP port of the Rule Engine. | 25080 |
database
:
Configuration Setting | Description | Default Value |
---|---|---|
path | Path to the adapter's database file. | /var/cache/relayr/gwa-azure-cloud-adapter-cpp/cloud.sqlite3 |
message_batching
- for more information, see the Data Batching and Caching section.
Configuration Setting | Description | Default Value |
---|---|---|
log_sending_timeout_ms | How long a log message can wait to be sent when there's not enough data for a full batch. -1 means indefinitely. | 60000 |
alert_sending_timeout_ms | How long an alert message can wait to be sent when there's not enough data for a full batch. -1 means indefinitely. | 100 |
measurement_sending_timeout_ms | How long a measurement message can wait to be sent when there's not enough data for a full batch. -1 means indefinitely. | 60000 |
task_update_sending_timeout_ms | How long a task update message can wait to be sent when there's not enough data for a full batch. -1 means indefinitely. | 100 |
config_response_sending_timeout_ms | How long a configuration response message can wait to be sent when there's not enough data for a full batch. -1 means indefinitely. | 100 |
locations_sending_timeout_ms | How long a location message can wait to be sent when there's not enough data for a full batch. -1 means indefinitely. | 60000 |
analytics_event_sending_timeout_ms | How long an analytics event message can wait to be sent when there's not enough data for a full batch. -1 means indefinitely. | 100 |
blob_sending_timeout_ms | How long a blob token request message can wait to be sent when there's not enough data for a full batch. -1 means indefinitely. | 10000 |
sending_buffer_size | For memory usage optimization, the adapter buffers the batched messages and sends only one batch at a time. This parameter defines the maximum number of batches buffered. If there are too many, the oldest batch is removed. | 64 |
compression_ratio_a | Heuristic estimation of a compression ratio as output_size = compression_ratio_a * input_size + compression_ratio_b * log(input_size) + compression_ratio_c . It is recommended to leave the default value unchanged. | 0.05 |
compression_ratio_b | Heuristic estimation of a compression ratio as output_size = compression_ratio_a * input_size + compression_ratio_b * log(input_size) + compression_ratio_c . It is recommended to leave the default value unchanged. | 1.5 |
compression_ratio_c | Heuristic estimation of a compression ratio as output_size = compression_ratio_a * input_size + compression_ratio_b * log(input_size) + compression_ratio_c . It is recommended to leave the default value unchanged. | 12 |
use_compression | If set to true , the adapter compresses messages sent to and received from the Cloud. If set to false , the compression is disabled entirely. This setting overrides the global option from the cloud.connection section. If not configured, the global setting is used. | true |
compression_level | Strength of data compression, between 0 and 22 . The 0 level still compresses the data. To disable the compression, set the use_compression setting to false . This setting overrides the global option from the cloud.connection section. If not configured, the global setting is used. | 15 |
message_cache
- for more information, see the Data Batching and Caching section.
Configuration Setting | Description | Default Value |
---|---|---|
queue_size | Number of messages to store in the RAM cache. When you set a very low value, the actual queue size might be larger than the setting shows. | 500 |
persistent_measurements_limit | Database measurement messages limit. Use -1 to disable the limit. Use 0 to disable the persistent caching. | 1000 |
persistent_alerts_limit | Database alert messages limit. Use -1 to disable the limit Use 0 to disable the persistent caching. | 1000 |
persistent_logs_limit | Database log messages limit. Use -1 to disable the limit. Use 0 to disable the persistent caching. | 1000 |
persistent_task_updates_limit | Database task update messages limit. Use -1 to disable the limit. Use 0 to disable the persistent caching. | 1000 |
persistent_locations_limit | Database location messages limit. Use -1 to disable the limit. Use 0 to disable the persistent caching. | 1000 |
persistent_analytics_events_limit | Database analytics event messages limit. Use -1 to disable the limit. Use 0 to disable the persistent caching. | 1000 |
persistent_blob_requests_limit | Limit of token request messages used in preparation for a blob upload. Use -1 to disable the limit. Use 0 to disable the persistent caching. It is recommended to set it to the same value as the persistent_blobs_limit setting. | 5 |
persistent_blobs_limit | Limit of the number of blobs sent concurrently. Use -1 to disable the limit. Blobs must be stored persistently, so setting this to 0 disables the caching, but also disables the blob sending feature. | 5 |
task_history_limit | Database task history limit. Use -1 for unlimited history. Use 0 to disable the task history persistence. | 1000 |
log
:
Configuration Setting | Description | Default Value |
---|---|---|
log_severity | Minimum severity of messages to be logged. | DEBUG |
health_monitor
- settings related to the health monitoring functionality:
Configuration Setting | Description | Default Value |
---|---|---|
heartbeat_interval | Time interval at which heartbeat messages are sent, given in seconds. | 60 |
Application Configuration File
The application configuration file of the Azure Cloud Adapter is located at:
/etc/relayr/gwa-azure-cloud-adapter-cpp/gwa-azure-cloud-adapter-application-config.json
This file lets you configure edge devices connected to the adapter.
In the devices
section, the following settings are available for each device:
Configuration Setting | Description | Example of Value |
---|---|---|
id | Device identifier used in the Gateway Agent. In most cases, it is the same as cloud_id . | sw-iot-device01 |
cloud_id | Device identifier used by Azure Cloud. This is the name given to the device in the IoT Hub. | sw-iot-device01 |
azure-key | Shared secret (symmetric key) used to authenticate the device with the cloud. Not recommended. See here for more information. | UdVtl3ZH+7h/e+K34c4MZraY=" |
ssl-certificate | Device X.509 public certificate file used to authenticate the device with the cloud. Recommended. See here for more information. | certs/sw-iot-device03.cert.pem |
ssl-private-key | Device X.509 private key file used to authenticate the device with the cloud. See here for more information. | certs/sw-iot-device03.key.pem |
hub-id | Hostname or IP address of the device's IoT Hub. This setting overrides the global IoT Hub host you set in the cloud.connection section of the deployment configuration file. | relayr-test-hub.azure-devices.net |
provisioning-payload | Optional. JSON object with additional device provisioning data. |
Configuring Device Authentication
While configuring edge devices in the application configuration file, you can use the following authentication methods:
- Shared secret - a device is authenticated with the cloud by means of a symmetric key that you enter in the
azure-key
setting in the application configuration file. You can generate a device's key in IoT Hub. This method is less secure than X.509 certificates, so it is suitable for testing purposes only.
Example:
"azure-key": "UdVtl3ZH+7h/e+KLRL011p1n9VAdyYjGgA34c4MZraY="
- X.509 certificates - to use this method, you need to generate a public and private signed certificate, containing IoT Hub's identity. Then, configure the
ssl-certificate
andssl-private-key
settings in the application configuration file. This is a recommended method of device authentication.
Example:
"ssl-certificate": "certs/sw-iot-device03.cert.pem",
"ssl-private-key": "certs/sw-iot-device03.key.pem"
Data Batching and Caching
The Azure Cloud Adapter provides data batching, caching, and compression mechanisms to optimize the amount of data sent to the cloud and ensure the reliable data exchange.
Message Batching
The adapter always sends data in batches. The minimum data batch size is 4 KB and the maximum size is 256 KB. Before sending data to the cloud, the adapter waits until it collects at least the minimum batch.
For each message type, e.g. measurements, alerts, or task updates, you can configure a timeout. It is a number of milliseconds the adapter waits if there's not enough data for a full batch. If the timeout expires, the adapter sends the collected data, no matter how much data it has gathered.
You can configure the batching timeouts in the message_batching
section of the deployment configuration file. If you don't want to set any timeout, set the setting value to -1
. It means that the adapter always waits for a full batch before sending the data.
It's not recommended to set a timeout to
-1
. When you disable the timeout and the adapter stops receiving data due to some problems, the data collected so far may never reach the cloud.
Example:
"message_batching": {
"measurement_sending_timeout_ms": 60000,
"task_update_sending_timeout_ms": 2000,
In this example, the timeout for sending measurements is 1 minute. The timeout for sending task updates is 2 seconds. After this time, the adapter sends the data even if it hasn't collected a full batch.
Message Compression
Apart from batching the sent data, the Azure Cloud Adapter also supports data compression. By default, the compression is enabled. If you want to disable it, set the use_compression
setting to false
in the cloud_connection
section of the deployment configuration file.
You can also configure the compression strength in the compression_level
setting. Available levels are between 0
and 22
with the default value of 15
.
The compression settings in the
cloud_connection
section are global and refer to all types of data sent to and received from the Cloud, e.g. to both complex measurements and data batching. To set a different compression for the data batching, configure local compression settings in themessage_batching
section of the deployment configuration file. If configured, the localuse_compression
andcompression_level
settings override the global ones.
Message Caching
To prevent data losses, the adapter provides a data caching mechanism. For each message type, e.g. measurements, alerts, or task updates, you can set a limit of messages, up to which the adapter stores data persistently in its database. Even if the connection between the Gateway Agent and the cloud is interrupted, the adapter sends the stored messages once the connection is restored. When a caching limit is crossed, the oldest messages are deleted.
You can configure the caching limits in the message_cache
section of the deployment configuration file. To disable the persistent caching, set the limit value to 0
. For an unlimited caching, set it to -1
.
Example:
"message_cache": {
"persistent_measurements_limit": 0,
"persistent_alerts_limit": 1000,
"persistent_task_updates_limit": -1,
In this example, the caching of measurements is disabled. The adapter stores persistently up to 1000 alert messages and an unlimited number of task updates.
If you disable the caching of blob messages by setting the
persistent_blobs_limit
to0
, you disable the whole blob sending feature, as blobs must be stored persistently.
Examples of Configuration Files
Deployment Configuration File
{
"transport": "mqtt",
"mqtt_connection": {
"host": "localhost",
"port": 1883,
"keepalive": 20,
"max_inflight_messages": 100,
"qos_levels": {
"measurements": 0,
"alerts": 1,
"config": 1,
"metadata": 1
}
},
"cloud_connection": {
"transport": "mqtt",
"host": "relayr-test-hub.azure-devices.net",
"port": 8883,
"ca_file": "IoTHubRootCA.pem",
"id_scope": "0ne000A72E1",
"repository": "https://mystorage.blob.core.windows.net",
"use_compression": true
},
"nb_configs_connection": {
"gwa_rule_engine": {
"host": "",
"port": 25080
},
"gwa_analytics": {
"host": "",
"port": 27080
}
},
"database": {
"path": "/var/cache/relayr/gwa-azure-cloud-adapter-cpp/cloud.sqlite3"
},
"message_batching": {
"alert_sending_timeout_ms": 100,
"log_sending_timeout_ms": 60000,
"measurement_sending_timeout_ms": 60000,
"task_update_sending_timeout_ms": 100,
"config_response_sending_timeout_ms": 100,
"locations_sending_timeout_ms": 60000,
"analytics_event_sending_timeout_ms": 100,
"blob_sending_timeout_ms": 10000,
"sending_buffer_size": 64
},
"message_cache": {
"queue_size": 500,
"persistent_measurements_limit": 1000,
"persistent_alerts_limit": 1000,
"persistent_logs_limit": 1000,
"persistent_task_updates_limit": 1000,
"persistent_locations_limit": 1000,
"persistent_analytics_events_limit": 1000,
"persistent_blob_requests_limit": 5,
"persistent_blobs_limit": 5,
"task_history_limit": 1000
},
"log": {
"log_severity": "DEBUG"
},
"health_monitor": {
"heartbeat_interval": 60
}
}
Application Configuration File
{
"devices": [
{
"id": "sw-iot-device01",
"cloud-id": "sw-iot-device01",
"azure-key": "UdVtl3ZH+7h/e+KLRL011p1n9VAdyYjGgA34c4MZraY=",
"hub-id": "relayr-test-hub.azure-devices.net"
},
{
"id": "sw-iot-device03",
"cloud-id": "sw-iot-device03",
"ssl-certificate": "certs/sw-iot-device03.cert.pem",
"ssl-private-key": "certs/sw-iot-device03.key.pem",
"hub-id": "relayr-test-hub.azure-devices.net"
}
]
}