Device Orchestration Service (DOS)

1 Device Orchestration Service Overview
2 Requirements for DOS
3 Getting Started with DOS
4 DOS Supported Handlers and Functions
- 4.1 Message Handlers
  - 4.1.1 Writing customer handlers
    - 4.1.1.1 Standard Event Overall Structure
    - 4.1.1.2 Standard Event Schema Definition
    - 4.1.1.3 Example of Generated Standard Event
      - 4.1.1.3.1 Multiple different entities with the same entity
      - 4.1.1.3.2 An array of Similar Entity Types
    - 4.1.1.4 Creating Standard Event
- 4.2 Filter Functions
5 Description of DOS Configuration Parameters
6 Service End-2-End Runtime Flow
- 6.1 Runtime Flow
7 Service Monitoring and Metering
- 7.1 ServiceRunHistory

Device Orchestration Service Overview

A-Stack Device Orchestration Service (DOS) is a component that allows device data from a number of device providers to bridge into the device models. As part of bridging multiple devices, data streams are created leveraging stream processing engines with the ability to process the raw device data into the selected device processors functions. Device Orchestration Service can be thought of three major components -

Smart Bridge Framework (SBF) - A subcomponent of Device Orchestration Service responsible for ingesting data from a number of configured Device Providers. Device Providers can either push the data to the transport enabled by Smart Bridge Service or Smart Bridge Service can pull the data a scheduled interval from the Device Provider.
Stream Processing Engine (SPE) - A subcomponent of Device Orchestration Service responsible for converting the Device Provider data bridged via Smart Bridge Service into a stream. Stream Processing Engine maintains the device data in memory with periodic disk writes in case of overflow. Interaction between Smart Bridge Service (SBF) and Stram Processing Engine (SPE) is either local (colocated) or remote mode (via HTTP interface)
Device Processor Framework (DPF) - A subcomponent of Device Orchestration Service responsible for processing the device data and store into Device models.

Requirements for DOS

The latest version of A-Stack TQLStudio

Connections to HTTP, WS, AMQP, MQTT, Perif, and OPC protocol handlers are supported out of the box.

Getting Started with DOS

This tutorial provides a hands-on introduction to Atomiton DOS. You create a DOS configuration to move data between a provider data source and your models.

Tasks:

Pre-requisites
Step 1: Create a new TQLStudio Project
Step 2: Prepare DOS Configuration files
Step 3: Map data fields
Step 4: Add filters, custom handlers (if any)
Step 5: Review and Create DOS Configuration
Step 6: Activate and Listen to Data

Step 1: Create a new TQLStudio Project

Step 2: Prepare DOS Configuration files

Step 3: Map data fields

Step 4: Add filters, custom handlers (if any)

Step 5: Review and Create DOS Configuration

Step 6: Activate and Listen to Data

DOS Supported Handlers and Functions

Message Handlers

DOS provides the following out-of-the-box message handlers

Handler Name	Description

Handler Name

Description

AST_DOS_MessageProcessingHandler_Activity

This handler does a simple 1:1 mapping of source fields to the destination field.

AST_DOS_MessageProcessingHandler_OPC_Activity

This handler parses the OPC data and maps each parsed NodeID structure into attribute names. It takes values from NodeValue.

NodeID Structure:

#ns=2;s=Channel1.Device2.\USSAV1S01HST\AirCompressor_CMP02_Current_Average

The algorithm is to look for last “\” and take the string from this location to the end of the string. In the above example, it takes AirCompressor_CMP02_Current_Average
Extracted last field is further broken down based user-provided SourceFieldFilterPattern

Writing customer handlers

Standard Event Overall Structure

Standard Event Schema Definition

#
xs:element(name: updatesaveentity):
    xs:complexType:
      xs:sequence:
        xs:element(name: request):
          xs:complexType:
            xs:sequence:
              xs:element(name: entity, maxOccurs: unbounded):
                xs:complexType:
                  xs:all:
                    xs:element(name: find):
                      xs:complexType:
                        xs:sequence:
                          xs:element(name: entity):
                            xs:complexType:
                              xs:all:
                                xs:element(name: entitytype):
                                  xs:simpleType:
                                    xs:restriction(base: "xs:string"):
                                      xs:pattern(value: "([a-zA-Z])+");
                                xs:element(name: filter):
                                  xs:complexType:
                                    xs:sequence:
                                      xs:any(maxOccurs: unbounded, minOccurs: 1, processContents: skip);
                    xs:element(name: entitytype):
                      xs:simpleType:
                        xs:restriction(base: "xs:string"):
                          xs:pattern(value: "([a-zA-Z])+");
                    xs:element(name: body):
                      xs:complexType:
                        xs:sequence:
                          xs:any(maxOccurs: unbounded, minOccurs: 1, processContents: skip);

Example of Generated Standard Event

Multiple different entities with the same entity

When different entities can be packed within a body; EntityType will be ignored and not used.

UpdateSaveEntity:
  Request:
    Entity:
      EntityType: Not used
      Body:
        AirCompressor:
          AirCompId: CMP02
          PowerAttrs:
            CurrentAverage: 0.0
            LineFrequency: 57.6
            PowerConsumption: 0.0
            PowerFactor: 57.6
            VoltageAverage: 57.6
        Boiler:
          BoilerId: B001
          PowerAttrs:
            CurrentAverage: 0.0
            LineFrequency: 57.6
            PowerConsumption: 0.0
            PowerFactor: 57.6
            VoltageAverage: 57.6
        HTPanel:
          HTPanelId: HT02
          PowerAttrs:
            CurrentAverage: 0.0
            LineFrequency: 0.0
            PowerConsumption: 0.0
            PowerFactor: 0.0

An array of Similar Entity Types

In the example below an array of AirQuality models are packaged with EntityType as EnvironmentSensor

#
UpdateSaveEntity:
  Request:
    Entity:
      entityType: EnvironmentSensor
      body:
        airQuality:
          reading:
            o3:
              Value: 4440.0
            pm1:
              Value: 15
            pm2p5:
              Value: 16
            so2:
              Value: 1110.0
            pm10:
              Value: 17
        Label: 15000024
        barometer:
          reading:
            pressure:
              Value: 111111
        ambientHumidity:
          reading:
            relativeHumidity:
              Value: 44.77
        geocoordinates:
          latitude: 55.724915
          longitude: 12.4730643
        ambientTemperature:
          reading:
            temperature:
              Value: 70
        parentDomain: Environment
        applicableDomain: Environment
        thirdPartyId: 15000024
        ambientNoise:
          reading:
            noise:
              Value: 55.33
        providerDetails:
          provider: uRad
          providerId: uRadEnvironmentProvider
        tenantId: ckcaws.com
        LastUpdated: 1548913485000
        deviceState:
          connState:
            connected: true
            Since: 1575297029602
    Entity:
      entityType: EnvironmentSensor
      body:
        airQuality:
          reading:
            no2:
              Value: 6080.0
            o3:
              Value: 9050.0
            pm1:
              Value: 21
            pm2p5:
              Value: 22
            pm10:
              Value: 23
        Label: 15000025
        barometer:
          reading:
            pressure:
              Value: 98989
        ambientHumidity:
          reading:
            relativeHumidity:
              Value: 88.77
        geocoordinates:
          latitude: 55.724917
          longitude: 12.4730644
        ambientTemperature:
          reading:
            temperature:
              Value: 98.14
        parentDomain: Environment
        applicableDomain: Environment
        thirdPartyId: 15000025
        ambientNoise:
          reading:
            noise:
              Value: 99.88
        providerDetails:
          provider: uRad
          providerId: uRadEnvironmentProvider
        tenantId: ckcaws.com
        LastUpdated: 1548913444000
        deviceState:
          connState:
            connected: true
            Since: 1575297029605

Creating Standard Event

$stdeventcontent = MLP.newMl();  #Create a new Map List Structure to hold the Standard Event
$stdeventcontent.EntityType = $F.context.EntityType; # Set the entity Type from either the context (if available) to derived from fields.
foreach (value: var $mapfield, in: $msghandlerInfo.MapFields.Field):           # while looping through the mapfileds and generating desitnation fields..
    ......
    $stdeventcontent.Body.{$mapfield.DestinationFieldName} = $srcValue;        # Set the Body of the event; All the mapped fields will be added within the body 
$stdevent.UpdateSaveEntity.Request.Entity = $stdeventcontent;                  # Now Set the overall container i.e. UpdateSaveEntity with generated standard event content

Filter Functions

Description of DOS Configuration Parameters

Parameter Name	Description

Parameter Name	Description
Service	An overall container of ServiceInstance
ServiceName	Unique name of the ServiceInstance
ServiceDescription	Description of ServiceInstance
Tags	Choose key-value pairs to tag your ServiceInstance. Use tags to organize, track, or control access for this ServiceInstance. For example, a tag can include CustomerName and license agreement
DataEncrpytion	Encryption ensures that your data and your sensitive information are not accessible by any user or application without a valid key. Encrypting data in transit and data at rest is vital for regulatory compliance. Service always encrypts your data and sensitive information during transit and at rest. You can either choose a customer master key (CMK) managed by Atomtion (default) or bring your own key to encrypt your data and sensitive information.
ServiceTrigger	A trigger causes ServiceInstance to run. You can choose from one of three triggers types: Run on-schedule. Use when you want the ServiceInstance to run automatically at a scheduled frequency. Event-driven. Use when you want the ServiceInstance to run when an external event happens (As described by the Provider) Run on-demand. Use when you want to run the ServiceInstance
SBFOptions	An overall container to hold SBF Options
ProviderInfo	An overall container to hold Provider Information
ProviderName	Name of the Provider
ProviderDescription	A detailed description of the provider
ProviderInteractionName	Provider interaction name. Combination of ProviderName and ProviderInteractionName must be unique
Interaction	An overall container to hold the provider interaction
CustomInteractionAction	Users can specify their own actions than can handle the complete provider interaction. This is useful when there are multiple steps involved in an interaction.
InteractionType	Interaction can be a “READ” or “WRITE” type. Read is for reading from the provider and write is to write back to the provider.
Protocol	An overall container to hold the provider protocol communication information
ProtocolHandlerName	Protocol Handler name - must be one of the A-Stack provided protocol handler name (HttpServerExtensionArgs) or short prefix like - HTTP or OPCUA, etc.
ClientServer	The interaction can be Client i.e. ServiceInstance initiates a connection at specific trigger interval OR Server i.e. provider pushes data to ServiceInstance
ConnectionInfo	An overall container to hold protocol connection information. It includes hostname, port, URI constructed via an array of name-value pairs, Request Message Value to be sent as a payload. Header parameters, Protocol Method (GET/PUT/POST..)
MessageProcessingHandler	An overall container to hold information on how to handle and process the message received from the provider using a given interaction.
MessageProcessingHandlerAction	Action name that can handle the message. Values can be `ASTFW_Service_MessageProcessingHandler_Action` or provided by the user. If it is provided by the user, it must take care of handling the mapping of source and destination fields.
MapFields	An overall container to hold source to the destination field mapping. From the Studio one can Upload a .csv file with mapped fields - Use a comma-separated values (CSV) file to specify the field mappings. Each line in the CSV file contains the source field name, followed by a comma, followed by the destination field name.
Field	An overall container to hold a single field mapping information
SourceFieldName	Source filed name from the response of a provider interaction
DestinationFieldName	Destination field name pointing to the user Thing Model attribute
Formula	This step is optional. To add a formula that concatenates fields, select two fields from Mapped fields, and then provide a custom formula to chose one of the standard formulas provided. Example: Concatenation.
DataModificationAction	This step is optional. Some of the data modifications are truncate a string value. Users can specify custom action to handle data modification.
Validation	Overall container to hold singe data validation. This can be used to discriminate if the same value is sent by the provider.
ValidationCondition	For Validations, add validations to check whether a field has bad data. For each field, choose the condition that indicates bad data. Service provided conditions are: Value is null Value is 0 Value is missing Value is negative Value contains text Users can provide their validation own condition.
ValidationAction	What action ServiceInstance should take when a field in a record is bad. Users can provide their own validation action

Service End-2-End Runtime Flow

Runtime Flow

Service Monitoring and Metering

ServiceRunHistory

Column	Description

Column	Description
ServiceName	ServiceInstance Name whose history is being maintained.
DataCreated	Data time when ServiceInstance was created
Status	The Status column tells you if the ServiceInstance is active, in the draft, or has been suspended. ServiceInstances triggered on a schedule that has been created but not yet activated are in “draft” mode. You can suspend ServiceInstances that is triggered on a schedule or by an external event. Suspended ServiceInstances can be resumed by calling StartListen action again
RunStatus	Run Status column shows the status of the last ServiceInstance run indicating if it was a success or if there were errors.
LastRunDate	Date time when ServiceInstance last run