Page Comparison

...

Key Concepts

Tip

icon	false

	Related docs
For loop
Show query

Background

Device Management

Bulk initialization of Things is closely tied to the device management aspect of IoT Platform. The devices layer is the most important component of an IoT solution. A mature IoT platform comes with comprehensive device management features that let customers:

On-board existing and new devices with rich metadata.
Identical devices that share the same metadata are grouped together. This feature makes it possible to search devices based on their capabilities.
The platform also provides per-device authentication and authorization to enforce enhanced security.
It makes it easy to define which devices can connect, send, and receive messages.
Devices can be easily blacklisted and whitelisted through declarative policies.

TQLEngine platform provide mechanisms to easily address above device management requirements.

Bulk Initialization Design Pattern

At the time of writing this tutorial Atomiton TQLEngine does not provide specific language constructs to perform bulk initialization of Things or Data. But the existing Atomic domain languages capabilities can be exploited to achieve bulk initialization requirements. The bulk initialization design pattern can be broken down into two steps:

Image Added

Bulk Initialization Steps:

Step #

Name

Description

Output

Implementation Flow

1

Gather Metadata

Input to gathering of metadata step can be:

Manual entires
Code that generates metadata
Reading metadata from external device management source

The output of gathering metadata step could be:

Generated CSV, JSON Files
Generated Create / Save TQL Query statement

...

	@TQL Macro, AppModel, AppFacet Gather and Initialize can be combined into same flow. Decoupled implementation from TQLEngine (Example: NodeJS generating series of JSON files)
2	Initialize	Initialize step

...

includes:

Load the data into TQLEngine.
Metadata gathered from step-1 serves as input

...

Bulk Initialization Steps:

Image Removed

Simulating Temperature Sensor

Simulation Logic

Image Removed

GHCurrentTemp is the general temperature inside the greenhouse
It is affected by CurrentTemp (external), the vent and fans, and all the heaters
The heaters are controlled by the application
The vent and fans controls are not exposed to the application. So vent and fan behavior are simulated by BT, described in the next section.
GHCurrentTemp is the default value for ZoneTemp if the heater is not on in that zone (if the heater is on in that zone, ZoneTemp will be higher than GHCurrentTemp)
Value updates every 5 (?) minutes
GHTempAdj On Sunnyday: formula to calculate GHTempAdj in Celcius: GHTempAdj = (100/(CurrentTemp+237.1))³ * 100
For example, when (external) CurrenTemp = -5 Celcius, GHTempAdj = 5.186; when CurrentTemp = 28 Celcius, GHTempAdj = 3.661
On NOT Sunnyday: formula to calculate GHTempAdj in Celcius: GHTempAdj = (100/(CurrentTemp+237.1))³ * 60
For example, when (external) CurrenTemp = -5 Celcius, GHTempAdj = 3.111; when CurrentTemp = 28 Celcius, GHTempAdj = 2.197
Since temperature has an effect on humidity, always determine temperature value before determining humidity value

Add Greenhouse and Heater Model

Code Block

language	xml
title	Greenhouse DataModel and Types
linenumbers	true

<Def Name="EnvInfo">
      <Number Name="Temperature"/>
      <Number Name="Humidity"/>
      <Number Name="Light"/>
      <Number Name="Pressure"/>
      <Number Name="SoilMoisture"/>
      <Number Name="Wind"/>
      <String Name="LastUpdatedByProvider"/>
    </Def>

    <Def Name="BoundingBox">
      <GeoLocation Name="Vertex1"/>
      <GeoLocation Name="Vertex2"/>
      <GeoLocation Name="Vertex3"/>
      <GeoLocation Name="Vertex4"/>
    </Def>

    <Def Name="GeoLocation">
      <Number Name="latitude"/>
      <Number Name="longitude"/>
    </Def>

    <DataModel Name="Greenhouse">
      <Sid Name="GreenhouseID"/>
      <String Name="GreenhouseName"/>
      <GeoLocation Name="Location"/>
      <String Name="LocationName"/>
      <BoundingBox Name="Boundary"/>
      <Integer Name="LaneCount"/>
      <Double Name="LaneWidth"/>
      <Double Name="ZoneLength"/>
      <Integer Name="ZoneInLane"/>
      <Integer Name="ZoneCount"/>
      <Double Name="GHLength"/>
      <Double Name="GHWidth"/>
      <EnvInfo Name="ExternalEnv"/>
      <EnvInfo Name="InternalEnv"/>
      <String Name="VentOnOffState"/>
      <String Name="FansOnOffState"/>
      <Boolean Name="SunnyDay"/>
      <Boolean Name="Running" default="false"/> <!-- true / false -->
    </DataModel>

    <ThingModel Name="Heater">
      <Sid Name="HeaterID"/>
      <String Name="OnOffState"/>
      <Integer Name="HeatingLevel"/>
    </ThingModel>

Pipeline Macros

Execute Internal Query
Schedule and UnSchedule TQL Code

Code Block

language	xml
title	Pipeline Macros
linenumbers	true

<Macro Name="executeQuery">
  <Argument>
    <QueryString>
      <Query/>
    </QueryString>
  </Argument>
  <Result>
    <OnRequest Target="[:RuntimeParams.FacetIDName:]" Disable="CMD_SERVER">
      <Process Return="CMD_NOP">
        <Message>
          <Value>
            [:$Macro.Argument.QueryString:]
          </Value>
        </Message>
      </Process>
    </OnRequest>
  </Result>
</Macro>


<Macro Name="DeleteScheduleJob">
  <Argument>
    <JobName/>
  </Argument>
  <Result>
    <DoRequest target="[:RuntimeParams.SequenceFacetIDName:]">
      <Process>
        <Message type="xml">
          <Value>
            <Remove eid="[:$Macro.Argument.JobName:]"/>
          </Value>
        </Message>
      </Process>
    </DoRequest>
  </Result>
</Macro>

<Macro Name="ScheduleJob">
  <Argument>
    <ScheduleInterval>
    </ScheduleInterval>
    <StartTime>
      0
    </StartTime>
    <EndTime>
    </EndTime>
    <ActionCode>
    </ActionCode>
    <Name>
    </Name>
  </Argument>
  <Result>
    <Log Message="Inside ScheduleJob macro"/>
    <DoRequest target="[:RuntimeParams.SequenceFacetIDName:]">
      <Process>
        <Message>
          <Value>
            <Execute eid="[:$Macro.Argument.Name:]" schedule="[[:$Macro.Argument.StartTime:]..[:$Macro.Argument.EndTime:]/[:$Macro.Argument.ScheduleInterval:]]">
              <Action>
                <ExecuteQuery>
                  <QueryString>
                    [:$Macro.Argument.ActionCode:]
                  </QueryString>
                </ExecuteQuery>
              </Action>
            </Execute>
          </Value>
        </Message>
      </Process>
    </DoRequest>
  </Result>
</Macro>

Macro to Simulate Sensor

Code Block

language	xml
title	Simulate Sensor Logic
linenumbers	true

<Macro Name="SetGHCurrentTemp">
      <Argument>
        <TempSensorId></TempSensorId>
        <GreenhouseID></GreenhouseID>
      </Argument>
      <Result>
        <Log Message="*********Calculating Temperature..."/>
        <executeQuery>
          <QueryString>
            <Query>
              <Find>
                <Greenhouse>
                  <GreenhouseID>[:$Macro.Argument.GreenhouseID:]</GreenhouseID>
                </Greenhouse>
              </Find>
            </Query>
          </QueryString>
        </executeQuery>
        <if condition="$Response.Message.Value/Find/Status eq 'Success'">
          <then>
            <SetLocalData key="CurrentTemp">
              <Value>[:$Response.Message.Value.Find.Result.Greenhouse.ExternalEnv.Temperature:]</Value>
            </SetLocalData>
            <SetLocalData key="GHCurrentTemp">
              <Value>[:$Response.Message.Value.Find.Result.Greenhouse.InternalEnv.Temperature:]</Value>
            </SetLocalData>
            <SetLocalData key="Vent">
              <Value>[:$Response.Message.Value.Find.Result.Greenhouse.VentOnOffState:]</Value>
            </SetLocalData>
            <SetLocalData key="Fans">
              <Value>[:$Response.Message.Value.Find.Result.Greenhouse.FansOnOffState:]</Value>
            </SetLocalData>
            <SetLocalData key="SunnyDay">
              <Value>[:$Response.Message.Value.Find.Result.Greenhouse.SunnyDay:]</Value>
            </SetLocalData>
            <executeQuery>
              <QueryString>
                <Query>
                  <Find>
                    <Heater>
                      <OnOffState eq='On'/>
                    </Heater>
                  </Find>
                </Query>
              </QueryString>
            </executeQuery>

            <Javascript>
              <resp>
                <Include>$Response.Message.Value</Include>
              </resp>
              var resultList = resp.Find;
              var heaterLevels = [];
              var heaterLevelsSum = 0;
              for each (var result in resultList.iterEntries("Result")){
              var heaters =
              result.getValue();
              heaterLevels.push(heaters.Heater.HeatingLevel);
              heaterLevelsSum = heaterLevelsSum + heaters.Heater.HeatingLevel;
              }
              var GHTempAdj =
              0;
              if([:$LocalData.SunnyDay:]){
              GHTempAdj = Math.pow((100/([:$LocalData.CurrentTemp:]+237.1)), 3) * 100;
              }else{
              GHTempAdj =
              Math.pow((100/([:$LocalData.CurrentTemp:]+237.1)), 3) * 60;
              }
              var GHBaseTemp = [:$LocalData.CurrentTemp:] + GHTempAdj;
              var TempVentImp =
              ([:$LocalData.CurrentTemp:] - [:$LocalData.GHCurrentTemp:]) * 0.03;
              var HeaterEffectRatio = heaterLevelsSum / 90;
              var TempHeaterImp =
              HeaterEffectRatio * (Math.sqrt(Math.max(0,(45-[:$LocalData.GHCurrentTemp:]))));
              var GHCurrentTemp = GHBaseTemp + TempVentImp + TempHeaterImp;
              sffContext.execute("SetContextData","key","GHCurrentTemp","value",GHCurrentTemp);
            </Javascript>

            <executeQuery>
              <QueryString>
                <Query>
                  <Find format="Version">
                    <TempSensor>
                      <sensorId>[:$Macro.Argument.TempSensorId:]</sensorId>
                    </TempSensor>
                  </Find>
                  <SetResponseData>
                    <Key>Message.Value.Find.Result.TempSensor.TempValue.Value</Key>
                    <Value>[:$ContextData.GHCurrentTemp:]</Value>
                  </SetResponseData>
                  <Save>
                    <from>Result</from>
                    <Include>$Response.Message.Value.Find</Include>
                  </Save>
                </Query>
              </QueryString>
            </executeQuery>

            <executeQuery>
              <QueryString>
                <Query>
                  <Find format="Version">
                    <Greenhouse>
                      <GreenhouseID>[:$Macro.Argument.GreenhouseID:]</GreenhouseID>
                    </Greenhouse>
                  </Find>
                  <SetResponseData>
                    <Key>Message.Value.Find.Result.Greenhouse.InternalEnv.Temperature.Value</Key>
                    <Value>[:$ContextData.GHCurrentTemp:]</Value>
                  </SetResponseData>
                  <Save>
                    <from>Result</from>
                    <Include>$Response.Message.Value.Find</Include>
                  </Save>
                </Query>
              </QueryString>
            </executeQuery>
          </then>
        </if>
      </Result>
    </Macro>

Add Action to Start Simulate

...

language	xml
title	Action to Start Simulation
linenumbers	true

...

.
This steps has to be repeatable.

...

Simple Sensors and Actuators

...

JavaScript to capture simulation logic
Schedule frequency of simulation

The output of initialize steps are:

Data and Things loaded into TQLEngine.
May or may not note trigger actions.

@TQL Query
Can be integrated into Gather Metadata step.

Greenhouse UseCase

Please refer to 8. Simulating sensors, actuators, and devices for the description of Greenhouse usecase. The data initalization requirement for this case is shown below

Greenhouse Configurator Application Flow

Image Added

Greenhouse Configuration Requirements

Ability to add greenhouses on-the-fly
Each greenhouse must be able to dynamically generate number of lanes, zones per lane
Add Crop type per lane

Bulk Initialization Implementation

Import baseline Greenhouse Data and Thing Models

Create a new Project in TQLStudio by importing content from Baseline models.

Pipeline Macros

Execute Internal Query

Code Block

language	xml
title	Pipeline Macros
linenumbers	true

<Macro Name="executeQuery">
  <Argument>
    <QueryString>
      <Query/>
    </QueryString>
  </Argument>
  <Result>
    <OnRequest Target="[:RuntimeParams.FacetIDName:]" Disable="CMD_SERVER">
      <Process Return="CMD_NOP">
        <Message>
          <Value>

...

          [:$Macro.Argument.QueryString:]
          </

...

Value>
        </Message>
      </Process>

...

</OnRequest>
  </Result>
</Macro>

Main CreateGreenhouse Macro

Code Block

language	xml
title	Main Create Greenhouse Macro
linenumbers	true

<Macro Name="CreateGreenhouse">
  <Argument>
    <Greenhouses></Greenhouses>
  </Argument>
  <Result>
    <SetLocalData key="Greenhouses">

...

      <Value>
        <Include>$Macro.Argument.Greenhouses</Include>
      </Value>

...

  </SetLocalData>
    <for each="record" in="Greenhouse" from="$LocalData.Greenhouses" using="$ProcessData">
       <JavaScript>

...

     var NumberOfLanes = [:$ProcessData.record.NumberOfLanes:];
        var ZonesPerLane

...

= [:$ProcessData.record.ZonesPerLane:];
        var NumberOfZones = NumberOfLanes * ZonesPerLane;
        sffContext.execute("SetLocalData","key","NumberOfZones","value",NumberOfZones);

...

     </JavaScript>
       <!-- Create Query for Greenhouse -->

...

      <SetContextData key="FirstLane">
           <Value>Yes</Value>

...

       </SetContextData>
          <For each="laneRecord" in="Lane" from="$ProcessData.record.Lanes" using="$LocalData">

...

          <!-- Add Lanes -->
            <addLanes>

...

              <GreenhouseID>[:$ContextData.GreenhouseID:]</GreenhouseID>
              <LaneName>[

...

$LocalData.

...

laneRecord.

...

LaneName:

...

]</LaneName>
              <LaneLength>110</LaneLength>

...

 <LaneWidth>40</LaneWidth>
              <ZoneInLane>[:$ProcessData.record.ZonesPerLane:]</ZoneInLane>
              <BorderLane>No</BorderLane>

...

            </addLanes>

...

<if condition="$Response.Message.Value/Create/Status eq 'Success'">
              <then>

...

     <SetLocalData key="LaneID">

...

<Value>[:$Response.Message.Value.Create.Lane.LaneID:]</Value>

...

</

...

SetLocalData>
                <loadZoneModels>

...

<LaneID>[:$LocalData.LaneID:]</

...

LaneID>

...

<ZonesPerLane>[:$ProcessData.record.ZonesPerLane:]</ZonesPerLane>

...

<CropType>[:$LocalData.laneRecord.CropType:]</CropType>

...

<FirstLane>[:$ContextData.FirstLane:]</FirstLane>
                </loadZoneModels>

...

            </then>

...

</if>
            <SetContextData key="FirstLane">

...

<Value>No</Value>
            </

...

SetContextData>
          </For>
        </

...

then>
      </if>
    </for>
  </Result>
</Macro>

Load Zone Models

Data is randomized and generated

Code Block

language	xml
title	Load Zone Models
linenumbers	true

<Macro

...

Name="loadZoneModels">
	<Argument>

...

<ZonesPerLane></ZonesPerLane>

...

<LaneID></LaneID>

...

<CropType></CropType>

...

    <FirstLane></FirstLane>
	</Argument>
	<Result>
      <JavaScript>

...

           var ZonesPerLane

...

= 0;

...

if(!isNaN([

...

$Macro.Argument.

...

ZonesPerLane:

...

])){
                ZonesPerLane = [:$Macro.Argument.ZonesPerLane:];

...

		var loadContent = ListMap.static.newInstance();
      		for (var row=1;

...

row &lt;= ZonesPerLane; row++) {

...

			var svi = loadContent.instanceAdd("addZone");

...

			svi.put("ZoneName", "Zone"+row);

...

Changes to TempFacetSerial

...

language	xml
title	Changes to Thing Facet
linenumbers	true

...

 			svi.put("LaneID", "[:$Macro.Argument.LaneID:]");

...

 			svi.put("ZoneLength", 10);

...

 			svi.put("ZoneWidth", 10);
      			svi.put("Vertex1", 5);

...

 			svi.put("Vertex2", 2);

...

 			svi.put("Vertex3", 15);

...

 			svi.put("Vertex4", 12);

...

 			svi.put("EndZone", "No");

...

 svi.put("ZoneAvgSM", 45);

...

svi.put("CropType", "[:$Macro.Argument.CropType:]");

...

Queries

Let's start writing some queries to create and read the data.

Create Greenhouse

...

language	xml
title	Create Greenhouse
linenumbers	true

...

if("[:$Macro.Argument.FirstLane:]" == "Yes" &amp;&amp; row==1){

...

  svi.put("Live", "Yes");

...

  }else{

...

svi.put("Live", "No");

...

   			svi.put("GridOnLength", 3);

...

    			svi.put("GridOnWidth", 3);

...

     			svi.put("GridCount", 9);

...

			svi.put("GridLength", 3.33);

...

			svi.put("GridWidth", 3.33);

...

 			svi.put("ZoneTemperature", "0");

...

  			svi.put("ZoneHumidity", "0");

...

svi.put("ZoneAvgLight", "0");

...

       svi.put("NoOfPlants",

...

Math.floor((Math.random() * 100) + 1));

...

		sffLog.info("load Zone Content ======== "+loadContent);
      		loadContent;

...

    </JavaScript>
	</Result>
</Macro>

Queries

Let's start writing some queries to create and read the data.

Reset All Data

Code Block

language	xml
title	Delete Greenhouse
linenumbers	true

<DeleteAllModelsData/>

Create Greenhouse

Macro can be called directly

Code Block

language	xml
title	Create Greenhouse
linenumbers	true

<CreateGreenhouse>

...

<Greenhouses>

...

	<Greenhouse>
		<GreenHouseName>GreenHouse-1</GreenHouseName>
		<Location>
			<latitude>1.11</latitude>
			<longitude>1.12</longitude>
		</Location>
		<ZonesPerLane>5</ZonesPerLane>
		<NumberOfLanes>2</NumberOfLanes>
		<Lanes>
			<Lane>
				<LaneName>Lane-1</LaneName>
				<CropType>Tomato</CropType>
			</Lane>

...

	<Lane>
				<LaneName>Lane-2</LaneName>
				<CropType>Tomato</CropType>
			</Lane>
		</Lanes>
	</Greenhouse>
  </

...

Greenhouses>
</

...

CreateGreenhouse>

...

Show Query

Code Block

title
language	xml

...

Show Query
linenumbers	true

<Query>

...

<Show>

...

<Entity

...

inherits="DataModel" not.qname="*.TqlSystem.*"/>
  </Show>
</Query>

Find Greenhouse / TempSensor

Code Block

language	xml
title	Find Greenhouse
linenumbers	true

<Query>
  <Find>

...

 <Greenhouse>

...

<GreenhouseID ne=""/>

...

</

...

Greenhouse>
  </Find>

...

</

...

Query>

...

Start Simulation

...

language	xml
title	Start Simulator
linenumbers	true

...

<!-- Custom container -->
<Query as="Khans.GreenDen">
  <Find>

...

<Greenhouse>

...

<GreenhouseID ne=""/>
    </

...

Greenhouse>
  </Find>
</

...

Query>
 
<!-- Filter the result -->
<Query as="Khans.GreenDen">
  <Find

...

only="

...

Greenhouse:ExternalEnv">
    <Greenhouse>
      <GreenhouseID ne

...

=""/>
    </Greenhouse>
  </Find>

...

Find Greenhouse / TempSensor

...

language	xml
title	Find Greenhouse
linenumbers	true

...

</Query>
 
<!-- Filter the result -->
<Query as="$none:Greenhouse.Names"> <!-- What happens if we don't use $none -->
 <Find only="Greenhouse:GreenhouseName,Greenhouse:ZoneCount">
   <Greenhouse>

...

<GreenhouseID ne=""/>     
   </Greenhouse>

...

</Find>
</Query>
 
<!--

...

Filter the

...

result -->
<Query as="

...

$none:Greenhouse.Names"> <!-- What happens if we don't use $none -->
 <Find only="Greenhouse:GreenhouseName,Greenhouse:ZoneCount" as="$none:$none:$none"> <!-- Knock off Containers -->

...

<Greenhouse>

...


     <GreenhouseID ne=""/>     
   </Greenhouse>

...

</Find>
</Query>

Complete Model

You can import this model into your engine from this location

Versions Compared

Old Version 6

New Version Current

Key

Key Concepts

Background

Bulk Initialization Design Pattern

Simulating Temperature Sensor

Simulation Logic

Add Greenhouse and Heater Model

Pipeline Macros

Macro to Simulate Sensor

Greenhouse UseCase

Greenhouse Configurator Application Flow

Greenhouse Configuration Requirements

Bulk Initialization Implementation

Import baseline Greenhouse Data and Thing Models

Pipeline Macros

Main CreateGreenhouse Macro

Load Zone Models

Queries

Create Greenhouse

Queries

Reset All Data

Create Greenhouse

Show Query

Find Greenhouse / TempSensor

Start Simulation

Find Greenhouse / TempSensor

Complete Model