This page contains several advanced patterns that may help you with processing events, enriching raw messages, and synchronizing activities with devices.
A common pattern is to pass normalized event data to other events for common processing, this can be achieved with exec.now().
graph LR
A1[Device #1] --> C1[LORA Service]
A2[Device #2] --> C2[LWM2M Server]
subgraph iotb [Tartabit IoT Bridge]
C1 --> D1{LORA Uplink Trigger - #T1}
C2 --> D2{LWM2M Data Trigger - #T2}
D1 -->|"exec.now()"| E{Send to App Trigger - #T3}
D2 -->|"exec.now()"| E
end
E --> H[(Application)]
T1 may be a trigger that receives LORA uplink messages and decodes them to a common format.
var obj = {
temperature: event.data.decoded.temp,
humidity: event.data.decoded.rh
}
exec.now('decoded-data',obj)
T2 may be a tirgger that receives LWM2M data events and decodes them to a common format.
var obj = {
temperature: event.data.temperature_3303['0'].sensor_value_5700,
humidity: event.data.humidity_3304['0'].sensor_value_5700,
}
exec.now('decoded-data',obj)
T3 is a trigger filtering on "Generic" events with key "decoded-data", thus it will execute with data anytime #T1 or #T2 execute sync.now().
var obj = {
temperature: event.data.temperature_3303['0'].sensor_value_5700,
humidity: event.data.humidity_3304['0'].sensor_value_5700,
}
var queueMessage = {
deviceId: event.endpoint.key,
ts: event.ts,
data: {
temp: obj.temperature,
hum: obj.humidity
}
}
// as an example we send the data to a service bus queue.
servicebus.queue_publish('azure_servicebus', 'myqueue', queueMessage)
- The general plan is to decode the device data, enrich the data with location or network data, send the data to the application.
- The best tool to use for this process is the
exec.now() functionality that lets you generate events with user defined data models.
- #T1 and #T2: These trigger receives events from devices, and decodes the messages to a common javascript object. Once decoded, it uses
exec.now() to forward the decoded event to #T3 for further enrichment.
- These triggers should not be aware of specific information regarding the target application. The reason being that you can multiplex the data and send the normalized events to more than one application, but the more specific you get in these triggers the harder to integrate to different backend applications.
- These trigger should handle normalizing of units, data types, event codes and such so that the #T3 trigger does not have "device specific" logic.
- #T3: This trigger is a "Generic" trigger that catches events from #T1 and #T2, and handles all enrichment of the raw data packets.
- It should not be aware of specific capabilities of the devices generating its events.
- This trigger is optional, if there is no enrichment, you can go directly from the normalized events coming from devices to the application integration trigger #T4.
- #T4: This trigger is responsible for sending data to the backend application, it could implement any of the enterprise service connectors, but the key is that it consumes
Generic events from #T3 and submits them to the backend application.
A common pattern is to need to integrate multiple different device types to the same backend application. It may be that you have devices that use different connectivity, ie cellular vs LORA, or it could be that you have multiple different vendors supplying devices that interface to the same application.
graph LR
A1[LORAWAN Device] --> B1[LORA Network Server]
A2[LWM2M Device] --> C2[LWM2M Server]
B1 --> C1[LORA Service]
subgraph iotb [Tartabit IoT Bridge]
C1 --> D1{LORA Uplink Trigger - #T1}
C2 --> D2{LWM2M Data Trigger - #T2}
D1 -->|"exec.now()"| E{Enrichment Trigger - #T3}
D2 -->|"exec.now()"| E
E -->|"location.reverse_geocode()"| E
E -->|"net.lookupMccMnc()"| E
E -->|"exec.now()"| G{Send to App Trigger - #T4}
end
G --> H[(Application)]
- The general plan is to decode the device data, enrich the data with location or network data, send the data to the application.
- The best tool to use for this process is the
exec.now() functionality that lets you generate events with user defined data models.
- #T1 and #T2: These trigger receives events from devices, and decodes the messages to a common javascript object. Once decoded, it uses
exec.now() to forward the decoded event to #T3 for further enrichment.
- These triggers should not be aware of specific information regarding the target application. The reason being that you can multiplex the data and send the normalized events to more than one application, but the more specific you get in these triggers the harder to integrate to different backend applications.
- These trigger should handle normalizing of units, data types, event codes and such so that the #T3 trigger does not have "device specific" logic.
- #T3: This trigger is a "Generic" trigger that catches events from #T1 and #T2, and handles all enrichment of the raw data packets.
- It should not be aware of specific capabilities of the devices generating its events.
- This trigger is optional, if there is no enrichment, you can go directly from the normalized events coming from devices to the application integration trigger #T4.
- #T4: This trigger is responsible for sending data to the backend application, it could implement any of the enterprise service connectors, but the key is that it consumes
Generic events from #T3 and submits them to the backend application.
IoT devices spend long periods of time asleep and not reachable by the server. For these devices one pattern is to use the queues built into the IoT Bridge to store data for devices until the next time they are available.
sequenceDiagram
participant Device
participant Server as IoT Bridge
participant App
Device->>Server: Sends data
Note right of Server: Trigger T1 executes
Server->>App: Server sends data to application
Note right of Device: Device goes to sleep!
App->>Server: Configure device
Note right of Server: Trigger T2 executes and calls queue.put() to save request from App
Device->>Server: Sends data
Server->>App: Server sends data to application
activate Server
Note right of Server: Trigger T1 checks queue.get() for pending configurations
Server->>Device: Sends configuration
Server->>App: Configuration acknowledged
Note right of Device: Device goes to sleep!
- Queues are non-blocking, you can't wait on a
queue.get() for work to arrive, and you must not use any kind of spin loops to wait for work. Queues should be checked when data is received by devices which is generally a good indicator that the device is able to receive commands.
- Our queue system is persistent to disk, but not backed up, as such, in certain extreme disaster recovery scenarios, the contents of the queues may be lost. There should always be a process to requeue messages in the event of a critical disaster.
- #T1: This trigger receives data from devices and sends it to the backend application. When it receives a message, it calls
queue.get() to check if there are messages waiting to be delivered. If messages are returned, it will use the device specific functions to send data to the device.
- #T2: This trigger receives data from backend applications, it could be an HTTP server request, or listening on a queue or other enterprise service bus connection. When the message is received it can either try to send it to the device, or queue it for future transmission using
queue.put().
Sometimes order is important, as well as reducing the amount of parallel processing that may be occuring within an application. Using event mode queues allows you to put items on a queue and use events to process the data in-order without complex trigger logic.
sequenceDiagram
participant Device
participant Server as IoT Bridge
participant Queue
participant App
Device->>Server: Sends data
Note right of Server: Trigger T1 executes
Server->>Queue: Trigger adds message to queue (depth:1)
Server->>Queue: Trigger adds message to queue (depth:2)
Queue->>Server: Generates "queue-activity" event (depth: 1)
Note right of Server: Trigger T2 executes
Server->>App: Trigger sends data to application
Server->>Server: Trigger ends with trigger.reply({})
Queue->>Server: Generates "queue-activity" event (depth: 0)
Note right of Server: Trigger T2 executes
Server->>App: Trigger sends data to application
Server->>Server: Trigger ends with trigger.reply({})
Note right of Server: Depth is 0, no more events
- Anytime a message is added to a queue, if a handler trigger is not running, a new
queue-activity event will be generated. If a handler trigger is already running, no event will be generated until the handler returns trigger.reply({}).
- If you do not call
trigger.reply({}) then no additional queue-activity events will be generated.
- #T1: This trigger receives data from devices and puts the messages on the queue to send to the server using
queue.put({data}, {event: true}) to enable event mode.
- #T2: This trigger receives the
queue-activity events and can use custom filters on the queue name to limit the scope of the trigger. It will process the queued data item and it must call trigger.reply({}) when complete.