1. Serial Temperature Sensor

Key Concepts



Related docs
TQLEngine AnatomyTQLEngine Anatomy
Development lifecycleDevelopment lifecycle
Model and ThingModelModels, Model lifecycle


 I. Hardware Setup

In order to run the tutorial, you need to set up the hardware. If you are using Raspberry Pi and Arduino, the instructions are here. If you are using Libelium, the instructions are here.

II. Abstract the device using ThingFacet

Write a ThingFacet 

To abstract the interactions with the temperature/humidity sensors setup in the previous section, we create a TempFacetSerial ThingFacet. 

 ThingFacet

ThingFacet is a reusable TQL component that defines the software interactions with a "thing" and represent the "thing" in the software. A "thing" can be sensors, actuators, devices, or a particular aspect or function of a more complex machine. A ThingFacet has a name, Model Attributes and Actions.

Attributes

Attributes are containers of values. They can be used for the following purposes in ThingFacets:

(1) Store parameters required to make a protocol specific invocation to the thing, such as <String Name="InterfacePort"/>

(2) Store information received from the thing, such as <Number Name="TempValue"/> (The TempValue from the temperature sensor)

(3) Hold fixed information about the thing, such as <String Name="Unit" default="Celsius"/>

(4) Act as control variables - variable that can activate certain actions. They are called "actionable attributes". Actionable attributes are linked with one or more actions in the same ThingFacet by a modifier "KnownBy". For example: <Number Name="TempValue" KnownBy="SerialReadAction"/>

Actions

Action is where the processes to interact with things are defined.


An example of Camera ThingFacet.

Go to page


We add two types of attributes in this ThingFacet (1) parameters required to make a protocol specific invocation (2) attribute(s) to store the information received from the sensor. For the purpose of this tutorial we have protocol specific parameters as (Unit, Peripheral, InterfacePort, Interface, etc). The full message received on the serial interface will be stored in the PerifMsg attribute and the Celsius Temperature Value will be stored in  the TempC Parameter. 

TempFacetSerial attributes
<ThingFacet Name="TempFacetSerial">
	<String Name="PerifMsg"/>
	<String Name="TempC"/>
    <String Name="Unit" default="Celsius"/>
	<String Name="Peripheral"/>
    <String Name="InterfacePort"/>
    <String Name="Interface"/>
    <String Name="UniqueId" default="56789"/>
    <String Name="Baudrate"/>
    <String Name="Format" default="ascii"/>
    <String Name="Operation"/>
    <String Name="Payload"/>
	...
</ThingFacet>

Write an Action with Workflow

The next step in ThingFacet is writing a Action which contains a workflow responsible for making external protocol specific call and store the output in the appropriate ThingFacet attribute.  An important decision while writing Action is correct protocol selection. See the /wiki/spaces/TQLDocs/pages/1179871 in the Working with Things section. For this serial sensor we need the perif:// protocol handler, which is part of the TQLEngine.

1. Write an Action name as SerialReadAction

SerialReadAction
<ThingFacet Name="TempFacetSerial">
	...
	<Action Name="SerialReadAction">
		...
	</Action>
</ThingFacet>
 Action

An Action is a named element that describes a unit of executable functionality.

An Action has a name:  

Action Named Element
#
Action(Name: "RotateCameraAction", Documentation: "Simple Action to Rotate IP-based Camera by x degrees") 

An Action must define executable code in any Atomic Domain Languages (Workflow, FacetScript, TQL, Subscribe, Sequence) or any of their combinations, within one of the following elements:

NOTE: In the case of ThingFacet or AppFacet an Action must start with a Workflow.

Action is associated with attribute(s)

An Action must be associated or attached to a model attribute. In ThingFacets or AppFacets, an Action is associated with an model attribute of the same ThingFacet or AppFacet using the "KnownBy" modifier as below. The associated attribute becomes an actionable attribute.

Action Named Element
#
Double(Name: "RotateValue", KnownBy: "RotateCameraAction")

Please refer to <subscribe> and <sequence> for their respective action associations.

See also Associate multiple thing actions.

Action results in changes of the "Known" value(s) of any attributes

From the Action, using the output of the workflow, you can update any attributes of the model. Note that It is always the "Known" value of the attribute that will be updated by Actions. In this example, Action is changing the State and Image attributes.

Action Result
#
Output(name: "Result", as: "ActionResult"):
  Value:
    State: "[%:[%:@Output:%]/if([:IOK:]) then 'ON' else 'OFF':%]"
    Image: "[%:[%:@Output:%]/if([:IOK:]) then Invoke/GetImage/Message/Value/text() else '/img/no-image.jpg':%]"

Note: the output of the workflow can update any attribute of the model. This is not limited to the actionable attribute.

Action is triggered by the modification of associated (KnownBy) attribute values

Trigger is an execution phase of an Action. Execution of an Action is asynchronous and can be triggered in any number of ways. Please refer to <subscribe> and <sequence> for their respective action triggers.

In ThingModels and AppModels any updates to the associated attribute(s) values will trigger the Action. This include the initialization of the attributes (i.e. when it first acquires its value). The attribute value updates can only be achieved via TQL queries, in one of the following forms: 

  • External applications using TQL queries to update the value of the attributes. This includes the following example of a Save query to instantiate the model. Here giving the attribute RFID a value of $Null( ) will trigger the Action "KnownBy" this attribute.
  • Internally TQL queries updating the value of the attributes (e.g. TQL queries can be used by models to update values of other model's attributes).

Instantiate a Device using TQL Query
#
Query:
  Save(format: "version,current"):
    RfidReader(ReaderId: "R1", URL: "perif://"):
        RFID(value: "$Null()", version: "1")

See also Automatic Action trigger.

Go to page


2. Create a Workflow within the Action, with one single continuous running Task and waiting for the Event with its ActionArgument.

 Workflow

A workflow consists of a sequence of tasks that are orchestrated and repeatable. Each task may process events, generate events, or process information. A workflow can be defined as part of an Action or as part of a Query.

Workflows are defined by Workflow Definition Language (WDL). Workflow can have a WorkflowId (wid). In run time, a workflow can have multiple instances (or called processes). Each process or instance is a single incarnation of the workflow definition.

TQL workflow should not be confused with business workflows. Business workflow is a set of business activities driven by data exchanges, work executions and people interactions. TQL workflow is a sequence of tasks that are strictly computer system based and purely data driven. So TQL workflow can execute very "low level" device communications such as messaging, as well as very "high level" application logic. 

A task can have the following structure: 

Input - any data which is required for the task to be started. Unless all inputs are available, the task will wait.

Output - any data which is generated by the task for externals to consume. One task's output may be the input for another task in the same workflow.  

Invoke - the process part of the task. Invoke may execute script, listen to protocol handler or send data to protocol handlers.

The sequence between multiple tasks in the same workflow is driven by the dependencies between their inputs and outputs. Since tasks generally only start on receiving their inputs, their sequences are only determined instance by instance at runtime.

Workflow example
<Workflow Limit="1" Live="1" Timeout="-1">
	<Task name="Main" while="true">
		<Event name="Argument" as="ActionArgument" />
		<Invoke name="InvokeSerialRead" waitFor="Argument" get="perif://">
			<Message>
				<Value>
					...
				</Value>
			</Message>
		</Invoke>
		<Output name="Result" as="ActionResult">
			<Value>
				...
			</Value>
		</Output>
	</Task>
</Workflow>

Go to page

 Workflow structure

Workflow contains one or more sequential or parallel, single time execution or repeatable tasks.

 

Go to page

Workflow
<Action Name="SerialReadAction">
	<Workflow Limit="1" Live="1" Timeout="-1">
    	<Task name="Main" while="true">
        	<Event name="Argument" as="ActionArgument"/>
			...
    	</Task>
	</Workflow>
</Action>

Here we used three modifiers for this workflow. Limit = "1" means there can be at most one instance of this workflow waiting. Live = "1" means there can be at most one instance of this workflow running. Timeout ="-1" means this workflow will never be timed out. We used a modifier while = "true" with the Task to make the workflow running in a continuous loop, because it needs to run repeatedly, not just once. For more details, refer to workflow modifiers and the lifecycle of a workflow.

The task will be activated by the event handler Event called ActionArgument. "ActionArgument" is the event generated whenever the attribute(s) associated with this Action is modified (See Associate Action with a ThingFacet Attribute). ActionArgument carries all the current values of the ThingFacet attributes, which can be used in the task if needed.


 ActionArgument

ActionArgument is automatically passed argument to the action of a ThingModel or AppModel (instance) by A-Stack at the action trigger step.


  • ActionArgument contains both internal and user-defined attributes of the ThingModel / AppModel instance that triggered the action.
  • ActionArgument is local and transient - any changes made to the ActionArgument does not get reflected in the original copy of ThingModel / AppModel.


Example: Here is the ThingModel and ThingFacet (Partial Action) Definition for a Phidget Servo Motor

TempFacetSerial attributes
#
ThingFacet(Name: "PhidgetServoFacet"):
  String(Name: "ServoProtocolURL", Default: "phid://")
  String(Name: "DeviceInterfaceIndex")
  String(Name: "PhidgetDeviceType")
  Integer(Name: "ServoAngle", KnownBy: "PhidgetServoAction")
  Action(Name: "PhidgetServoAction", Documentation: "Control Servo Motor"):
    Workflow Limit: "1", Live: "1", Timeout: "-1"):
      Task(Name: "Main", while: "true"):
        Event(Name: "Argument", as: "ActionArgument")
          #...
ThingModel(Name: "PhidgetServoModel", Combines: "PhidgetServoFacet"):
  Sid(Name: "PhidServoId")

Here is the Phidget Servo Motor Instance creation TQL Query that triggers an Action

PhidgetServo Motor Instance Creation
#
Query:
  DeleteAll:
    PhidgetServoModel:
      PhidServoId(ne: "")
  Create:
    PhidgetServoModel:
      ServoProtocolURL: phid://
      PhidgetDeviceType: PhidgetAdvancedServo
      DeviceInterfaceIndex: 0
      ServoAngle: 110

Here is the ActionArgument Structure that is passed by A-Stack

ActionArgument Value of Phidget Servo Motor
#
Event:
  Argument:
    PhidServoId: KNI5JYVPAAAAUAABA4ONNHF2
    QName: 
      Atomiton.PhidgetServos.PhidgetServoModel
    $Guid:
      "#f3guorsmvr4qhomryacl5m3dha3ctaoe"
    $sid:
      KNI5JYVPAAAAUAABA4ONNHF2
    PhidgetDeviceType(Value: "PhidgetAdvancedServo", Known: "PhidgetAdvancedServo", Order: "", Version: "1", Timestamp: "1457366819505", DateTime: "2016-03-07 08:06:59.505"
      QName: "Atomiton.PhidgetServos.PhidgetServoModel.PhidgetDeviceType", FName: "PhidgetDeviceType")
    ServoAngle(Value: "110", Known: "110", Order: "", Version: "1", Timestamp: "1457366819505", DateTime: "2016-03-07 08:06:59.505", QName: "Atomiton.PhidgetServos.PhidgetServoModel.ServoAngle"
      FName: "ServoAngle")
    DeviceInterfaceIndex(Value: "0", Known: "0", Order: "", Version: "1", Timestamp: "1457366819505", DateTime: "2016-03-07 08:06:59.505"
      QName: "Atomiton.PhidgetServos.PhidgetServoModel.DeviceInterfaceIndex", FName: "DeviceInterfaceIndex")
    ServoProtocolURL(Value: "phid://", Known: "phid://", Order: "", Version: "1", Timestamp: "1457366819503", DateTime: "2016-03-07 08:06:59.503"
      QName: "Atomiton.PhidgetServos.PhidgetServoModel.ServoProtocolURL", FName: "ServoProtocolURL")

The following system generated tags are for each attribute:


A-Stack automatically generates tags as part of every attribute that makes up the model definition. System tags can be read (using FIND) with 

Format="all" modifier with the <Find> Query.

System Tag NameDescriptionPersisted?Query Constraint?
ValueThe value of the attribute is stored in "Value". This is often specified by the user at the time of create query (instantiation)YesYes
Known

Applicable to ThingModel/AppModel - this is the value that is only set by the output of Actions. Known CANNOT be

set using CRUD operations (Create / Update) via TQL Queries. The system also maintains a "Known" for DataModel attributes, for consistency purposes, but it will always be the same as Value.

YesNo
VersionThe version number of the attribute. This number is automatically incremented each time the value changes.Yes

No. You can request the value

using Format on <Find>

TimestampUnix epoch time at the time (in millisecond) of the attribute creation. For example: Timestamp value
1457663102004 is same as Fri, 11 Mar 2016 02:25:02 GMT
YesNo
DatetimeFormatted data time value of the attribute creation. For example:
2016-03-10 18:25:02.004
YesNo
QName

Fully qualified name of the attribute. <NamSpace>.<Domain>.<ModelName>.<AttributeName>. For example

SimpleModel.MyModels.VendorInfo.vendorName given that VendorInfo is defined within

Namespace - SimpleModel, with Domain as MyModels and DataModel Name as VendorInfo

YesNo
FNameGiven name of the attribute name.YesNo

Go to page

$Guid here is a global unique identifier of this model definition

$Sid is a unique Identifier of this Model Instance. Note that this value will be same as PhidServoId that is defined by PhidgetServoModel Thing Model using type Sid.


Accessing ActionArgument:

ActionArgument values can be accessed using standard Template Processor (TP) Notation. For example:

Access to PhidgetDeviceType Value
[%:Event.Argument.PhidgetDeviceType.Value:%]

Go to page

3. Invoke PERIF call: Method of PERIF is GET. We use Invoke modifier Get for this purpose. The parameters required PERIF are passed as part of <Message> and <Value>. More details of the perif:// handler are in the Working with Things Section.

Invoke serial handler
<Invoke name="InvokeSerialRead" waitFor="Argument" scope="process" get="perif://">
    <Message>
        <Value>
            <InterfacePort>[%:Event.Argument.interfacePort.Value:%]</InterfacePort>
            <Baudrate>[%:Event.Argument.Baudrate.Value:%]</Baudrate>
            <Interface>[%:Event.Argument.Interface.Value:%]</Interface>
            <UniqueId>[%:Event.Argument.UniqueId.Value:%]</UniqueId>
            <Operation>[%:Event.Argument.Operation.Value:%]</Operation>
            <format>[%:Event.Argument.Format.Value:%]</format>
            <Payload>[%:Event.Argument.Payload.Value:%]</Payload>
            <Peripheral>[%:Event.Argument.Peripheral.Value:%]</Peripheral>
        </Value>
    </Message>
</Invoke>

Here Invoke gets the attribute values that the Event ActionArgument (or Argument) carries. It uses them as the parameter values for the PERIF by substitution. For example, 

<Baudrate>[%:Event.Argument.baudrate.Value:%]</Baudrate>

means replacing the value of Message.Value.Baudrate with the value of Event.Argument.baudrate.Value.

Value substitution (rather than assignment) is very common in TQL, given that it is a declarative language. The notation [%: is part of the /wiki/spaces/TEACH/pages/21170773. More details on /wiki/spaces/TEACH/pages/21170773 can be found in the Developer's Guide.

4. Process the message received from the sensor

Message parsing using TP and XPath
<Output Name="Result" As="ActionResult">
	<Value>
		<PerifMsg>
			[%:Invoke.InvokeSerialRead.Message.Value.received:%]
		</PerifMsg>
    	<TempC>
        	[%:Invoke.InvokeSerialRead.Message.Value/substring("[%:Invoke.InvokeSerialRead.Message.Value.received:%]",7,5):%]
    	</TempC>
	</Value>
</Output>

Here we used an XPath statement to parse the TempC value out of the message. The sketch that is installed on the Arduino when it comes with the IoT Kit writes a message on the serial interface that looks as follows: TEMPC:25.94#TEMPF:78.69#HUMPCT:0.0#AMB:618. So in the processing of the output two things are done here. The "Known" of the <PerifMsg> attribute of the facet is set to the entire message string. And with the XPath substring function is used to extract the Celsius Temperature value (in the example 25.94). Instead of the substring function that uses a fixed start and length one could of course use other XPath functions to make this code more robust.

Alternatively, you can use Javascript. An example of using JavaScript for message processing within an Action can be found in another tutorial here.

Associate Action with a ThingFacet Attribute

We will now have to associate a ThingFacet Attribute (PerifMsg) with the Action using the KnownBy modifier. When PerifMsg is "KnownBy" SerialReadAction, any PerifMsg value changes will activate the SerialReadAction. We do the same for the TempC attribute to set its "Known" in the output of the action.

Attribute TempValue
<ThingFacet Name="TempFacetSerial">
	...
	<String Name="PerifMsg" update="auto" KnownBy="SerialReadAction">
	<String Name="TempC" update="auto" KnownBy="SerialReadAction"/>
</ThingFacet>
 Actionable attributes

An actionable attribute is a model attribute with a modifier of "KnownBy" pointing to one or multiple Action(s). For example:



Example
String(name: "Image", KnownBy: "SyncImage")

String(name: "State", KnownBy: "SyncImage,SyncPreset")



An actionable attribute defines the active property of a ThingFacets or AppFacets.

In runtime, an actionable attribute will activate the associated Action(s) and their workflows whenever its value is modified.

For more on the relationship between Actions and actionable attributes, see here.

Go to page

The attribute modifier update= "auto" makes sure that once the action associated with this attribute is triggered, its workflow continues to run and wait for subsequent sensor events (not just the first event). This modifier is only used with actionable attributes. For more details, refer to Automatic Action trigger.

III. Combine ThingFacet with a ThingModel to persist data

Finally, in order to use ThingFacet we have to combine it with a ThingModel. We define ThingModel to contain only a unique system identifier.

ThingModel TempSensor
<ThingModel Name="TempSensor" combines="TempFacetSerial">
	<Sid Name="sensorId"/>
</ThingModel>
 Models

A model is a definition of a container, which represents the functionality or knowledge about some kind of entity. At run time, a model can be instantiated into model instances. Models are defined by Thing Definition Language (TDL). 

 In general, there are three types of models in an IoT application:

  • ThingModel - representing any entity external to the system that can have interactions with the system (e.g. exchanging messages). These "things" can be sensors, actuators, machines, or other devices.
  • DataModel - representing a data structure.
  • AppModel - representing any unit of application logic which can contain one or more processes. These processes may interact with other components within the system (e.g. other Models) or with external applications.

The basic structures of Models

  • A name
  • A list of Attributes
  • A number of Actions (only ThingModels and AppModels have Actions)
  • A number of Constraints (including Unique)

All models by default contain an Attribute of type SystemId (or Sid). If Sid is not defined by the author of the model, system will assign Sid to the model.

Model "combines" model facets

To increase reusability, model facets are adopted as reusable units.

There are two types of model facets:

  • ThingFacets
  • AppFacets

Model facets have the same structural components of Models (name, attributes, actions, constraints). However, model facets are not instantiated or persisted. They can only be "combined"  as part of a model, with the model being instantiated, and persisted.

  • ThingModel can combine ThingFacets;
  • AppModel can combine AppFacets;
  • There is no DataFacets.

As a good and recommended practice pattern, actions and attributes of a particular device logic or application logic are always defined in the model facets, and the model will acquire those attributes and actions via the keyword "combines". A model can combine multiple model facets. And vice versa, model facet can by combined by multiple model definitions.

For example:

ThingModel
#
ThingModel(Name: "TempSensor", combines: "TempFacetSerial"):
	Sid(Name: "sensorId");

Go to page

 ThingModels

ThingModel is one kind of Model (the other kinds are DataModels and AppModels). Therefore it has all the properties of Models (Lifecycle of modelsModel AttributesUnique and Constraints).

A ThingModel has

However, as a recommended practice, the Attributes and Actions related to the interactions with an external thing (sensors, actuators, machines) are often defined in a reusable structure called ThingFacet, and combined into a ThingModel. This relationship was explained in the concept of Models and Model Facets. Application related information are often defined directly as attributes of the ThingModel.

In the following example, the ThingModel PhidgetServoModel combines the ThingFacet PhidgetServoFacet, where: 

  • PhidgetServoFacet contains the device interaction specific Attributes (e.g. ServoProtocolURL) and Actions (e.g. PhidgetServoAction);
  • PhidgetServoModel contains application related Attributes: PhidServoId and InstalledAtZone (assuming the Phidget servo motor is installed in a particular zone within a Greenhouse environment, which has multiple zones).
  • Combines is used as a modifier of ThingModel

In the simplest case, a ThingModel may only have one single directly defined attribute of type Sid. Although not required, it is best practice to always create a ThingModel with one attribute that is of type SystemId or Sid.

PhidgetServo Thing Model
#
ThingModel(Name: "PhidgetServoModel", Combines: "PhidgetServoFacet"):
    Sid(Name: "PhidServoId")
    String(Name: "InstalledAtZone")
    
ThingFacet(Name: "PhidgetServoFacet"):
    String(Name: "ServoProtocolURL", Default: "phid://")
    String(Name: "DeviceInterfaceIndex")
    String(Name: "PhidgetDeviceType")
    Integer(Name: "ServoAngle", KnownBy: "PhidgetServoAction")
    Action(Name: "PhidgetServoAction", Documentation: "Control servo motor"):
      #...

This pattern allows the ThingFacet to be a reusable artifact, and potentially combined by multiple ThingModels.

Note: One can combine multiple ThingFacets within a single ThingModel using comma separated list of ThingFacets.

For Example Esky ThingModel combining EskyPreset and EskyImage Thing Facets

PhidgetServo Thing Model Part of a Zone
#
ThingFacet(Name: "EskyPreset"):
  String(Name: "State", KnownBy: "SyncPreset")
  String(Name: "Preset", KnownBy: "SyncPreset")
  Action(Name: "SyncPreset", documentation: "Synchronize camera state and preset"):
    Workflow Limit: "1", Live: "1", Timeout: "PT20S"):
      Task(Name: "Main", while: "true"):
        Event(Name: "Argument", as: "ActionArgument")
          #...

ThingFacet(Name: "EskyImage", combines: "Login"):
  String(Name: "State", KnownBy: "SyncImage")
  Clob(Name: "Image", KnownBy: "SyncImage")
  
  #Actions
  Action(Name: "SyncImage", documentation: "Synchronize camera state and snapshot image"):
    Workflow Limit: "1", Live: "1", Timeout: "PT10S"):
      Task(Name: "Main", while: "true"):
        Event(Name: "Argument", as: "ActionArgument")

ThingModel(Name: "Esky", combines: "EskyPreset,EskyImage", documentation: "actual single instance camera model"):
  Sid(Name: "CameraId")

In summary

  • ThingModels are normally used to represent and define interactions with external physical things. The specifics of interactions are often modeled via ThingFacet.
  • ThingModels combine one or more ThingFacets. ThingFacets are reusable artifacts. In runtime, only instances of ThingModels are created and persisted (not ThingFacets). 
  • ThingModels can contain application related information that are related to things.

When ThingModels do not have any Actions or combine any ThingFacets they are equivalent to DataModel i.e pure Attributes only.

ThingModel can be created, read, update and deleted using TQL queries.


 CRUD operations on ThingModels

Examples of CRUD Operation on a ThingModel:

CREATE:

Create PhidgetServo Thing Model
#
Create: 
    PhidgetServoModel: 
      ServoProtocolURL: "phid://"
      PhidgetDeviceType: "PhidgetAdvancedServo"
      DeviceInterfaceIndex: 0
      ServoAngle: 110

CREATE RESULT:

Create PhidgetServoModel Result
#
Create(Status: "Success"):
  PhidgetServoModel: 
    PhidServoId: "KNJN2YRVAAAAUAABA7LIQ4Y6" 
    ServoProtocolURL(Status: "Success+Created:1:1457384153653;", Value: "phid://")
    DeviceInterfaceIndex(Status: "Success+Created:1:1457384153653;", Value: "0")
    PhidgetDeviceType(Status: "Success+Created:1:1457384153653;", Value: "PhidgetAdvancedServo")
    ServoAngle(Status: "Success+Created:1:1457384153654;", Value: "110")

Note: If a ThingModel is combined with a ThingFacet, creation of a ThingModel results in instantiating (Trigger) of a ThingFacet action.


UPDATE:

PhidgetServo Thing Model
#
Query: 
  Find(format: "version"):
    PhidgetServoModel as: "var.PS"):
      PhidServoId(ne: "")
  SetResponseData: 
    key: Message.Value.Find.Result.PhidgetServoModel.ServoAngle.Value
    value: 120
  Update: 
    from: Result
    Include: $Response.Message.Value.Find


UPDATE Result:

PhidgetServo Thing Model
#
Find(Status: "Success", Format: "version"):
  Result: 
    PhidgetServoModel: 
      PhidServoId: "KNJPIFR6AAAAUAABA67ZRBQO" 
      PhidgetDeviceType(Value: "PhidgetAdvancedServo", Version: "1")
      ServoAngle(Version: "1", Value: "120")
      DeviceInterfaceIndex(Value: "0", Version: "1")
      ServoProtocolURL(Value: "phid://", Version: "1")

Note: If a ThingModel is combined with a ThingFacet updating a ThingModel attribute results in Trigger of a Action if that attribute is associated with an action.


DELETE:

Delete PhidgetServoModel Thing Model
#
Query: 
  DeleteAll: 
    PhidgetServoModel: 
      PhidServoId(ne: "")


DELETE Result

Delete PhidgetServoModel Thing Model Result
#
DeleteAll(Status: "Success"):
  Result: 
    PhidgetServoModel: 
      PhidServoId: "KNJOOOBZAAAAUAABA72MMI6E"
      PhidgetDeviceType: "PhidgetAdvancedServo"
      ServoAngle: 110
      DeviceInterfaceIndex: 0
      ServoProtocolURL: "phid://"

Note: If a ThingModel is combined with a ThingFacet, creation of a ThingModel results in deleting an instance results in cleanup of connections with actual things as per the protocol Handler used.



FIND

PhidgetServo Thing Model
#
Query: 
  Find(format: "all"):
    PhidgetServoModel:
      PhidServoId(ne: "")


Find result: Format="all" - returns all the internal attribute data as well.

PhidgetServoModel ThingModel Find Result
#
Find(Status: "Success", Format: "all"):
  Result: 
    PhidgetServoModel(QName: "Atomiton.PhidgetServos.PhidgetServoModel"):
      PhidServoId: "KNJOOOBZAAAAUAABA72MMI6E" 
      PhidgetDeviceType(Value: "PhidgetAdvancedServo", Known: "PhidgetAdvancedServo", Version: "1", Timestamp: "1457384798266", DateTime: "2016-03-07 13:06:38.266", QName: "Atomiton.PhidgetServos.PhidgetServoModel.PhidgetDeviceType", FName: "PhidgetDeviceType")
      ServoAngle(Value: "110", Known: "110", Version: "1", Timestamp: "1457384798266", DateTime: "2016-03-07 13:06:38.266", QName: "Atomiton.PhidgetServos.PhidgetServoModel.ServoAngle", FName: "ServoAngle")
      DeviceInterfaceIndex(Value: "0", Known: "0", Version: "1", Timestamp: "1457384798266", DateTime: "2016-03-07 13:06:38.266", QName: "Atomiton.PhidgetServos.PhidgetServoModel.DeviceInterfaceIndex", FName: "DeviceInterfaceIndex")
      ServoProtocolURL(Value: "phid://", Known: "phid://", Version: "1", Timestamp: "1457384798265", DateTime: "2016-03-07 13:06:38.265", QName: "Atomiton.PhidgetServos.PhidgetServoModel.ServoProtocolURL", FName: "ServoProtocolURL")


Note:

Go to page


Go to page


Only when the TempSensor ThingModel "combines" TempFacetSerial, can TempFacetSerial be instantiated within the TempSensor and accessible by TQL queries. TempSensor will inherit all the attributes from TempFacetSerial, in addition to its own attributes. The ThingFacet TempFacetSerial hence serves as a reusable component.

More details on the use of "combines" can be found here. Information on Sid can be found here.

IV. Deploy and test via queries and subscriptions

Export, import and deploy

Once the models are built in a TQLStudio project, you can export the project. The URL of the Zip file containing the content of your project will be sent over to your email account. Once you have downloaded the engine – launch the TQLEngine User Interface in the browser and select Import Project. You can copy the URL from Export Project Email and hit import button. Go to ThingSpaces and Deploy the project.

 Deploy and instantiate

Deploy

In TQL, Model development is a three step process -

  1. Definition
  2. Deploy (compilation)
  3. Queries (instantiation, update, delete, subscription)

See also lifecycle of models.

In this section Deploy and Query (which includes Instantiation or Create) is discussed. Deploy and Query is the the runtime phase of the model development. In order to deploy the models you would need a running instance of A-Stack. A-Stack can be downloaded from your TQLStudio Account page and installed to the environment of your choice.

The deployment unit for A-Stack is Package, which has contains definition of all the capabilities (Facets) required as part of the package.

Package File (<Package Name>.mqp.xml) has following structure and it gets created by the system at the time of deploying a project by using TQLConsole.

A-Stack Package Definition

Package contains instances for Facet Types and give the unique Id as fid-<FacerID>. fid-<Facet ID> is used to refer to this instance both from inside the engine from other models and from outside of the engine.

Package Definition Template
#
NewPackage(Name: "Some-Name-for-the-package"): 
	# List of A-Stack Capabilities to deploy using FacetScript
 	NewFacetInstance(Name: "..", fid: "..", Type: "..."):
		#...
Package Definition File Naming

Package definitions are defined in a file and given and extension of "mqp.xml". The file is normally placed in the "deploy" folder subdirectory.

A-Stack Capabilities

Package definition contains instances of capabilities that are offered by A-Stack. Below is the list of the A-Stack Capabilities:

Name

Type NameDescription
MessageSffMsgFacetGeneric Message Communication.
NetworkSffNetworkFacetA-Stack cluster support. Define the list of A-Stack(s) that can participate in syncing their data
TQLSffTqlFacetThing Query Language Support (Thing Definition, Thing Interaction)
WorkflowSffWdlFacetWorkflow Language Support
Behavior TreesSffBdlFacetBehavior Tree Implementation
TopicSffTopicFacetSubscription and Notification of Model changes
SchedulingSffSequenceFacetSchedule tasks / jobs (piece of Model code) to be executed in a repeated sequence of defined interval.
Static FilesSffStaticFileFacetServe static files using A-Stack.

When a package definition is created you are defining the list of A-Stack capability instances that you would like to get started with. The capability instances are defined using FacetScript Language. Please refer to Facescript Introduction Section for more details on using FacetScript.

There are two ways to deploy the models:

  1. TQLStudio ThingSpaces
  2. A-Stack Management API against your running copy of A-Stack on your local machine (Gateway, MicroController, Laptop).

Deploying Project using TQLStudio ThingSpaces

Once the models are defined using Model Editor as part of your project, you can deploy the models by clicking "Deploy" as part of ThingSpaces. The models get deployed to the default A-Stack provided by Atomiton in the cloud. You can overwrite the destination engine by updating Host, where your engine is running.

When you run the A-Stack on your machine, there is default administration UI, which can be accessed through http://<Hostname>:<Port>/fid-TQLConsole/app/index.html. The project can be exported from cloud Studio and imported in the local engine by using this admin UI. The default username and password for this local engine is <TQLEngine@atomiton.com> and password is <tql123>.

Content of Package Definition when deploying using TQLStudio ThingSpaces

Once the project is deployed from ThingSpaces in TQLStudio (or A-Stack UI) default Package Layout is deployed as part of your project. The rational behind default package layout is to provide a instance of every A-Stack with standard capability to start with. Here we list the package definition snippets:

Runtime Parameters Section

Top section of the default package definition maintains all the reusable reference names. As developer, you can extend this list with your own names, which can be referred later.

Default Package Definiton Runtime Parameter Section
#
RuntimeParams:
	DbmLocation(Comment: "Name of the model file"): Model.Location
    ThingFacetLocation(Comment: "Folder for all the Facet definitions"): TF.Location
    MacroLocation(Comment: "Folder for all the macro definitions"): Macros.Location
    SpacesLocation(Comment: "Folder for all instances definition"): Spaces.Location
    TypesLocation(Comment: "Folder for all type definitions"): Types.Location
    WSFacetIDName(Comment: "WebSocket Facet Id for TQL subscriptions"): TQL.FacetWS
    FacetIDName(Comment: "Http Facte Id for TQL Queries"): TQL.Facet
    TQLCacheName(Comment: "Storage name for the current package"): TQL.CacheName
    TopicFacetIDName(Comment: "???"): TQLGenericTopic
    MacroFacetID(Comment: "???"): Macro.Location
    SequenceFacetIDName(Comment: "Unique name for the Sequence Facet Instance"): TQLSequenceFacet
    WdlFacetIDName(Comment: "Unique name for the Workflow Facet Instance"): TQLWdlFacet
    TQLOutputFormat(Comment: "???"): XML
    ProjectSettings_EnforceKey(Comment: "???"): False
    ProjectSettings_ProjectKey(Comment: "???"): TQLStudio.ProjectKey
Workflow - Standalone Use of Workflow using Workflow Definition Language (WDL)

Workflow is one of the basic capability in the engine. It is used in the Thing and App Models by default to support actions. In addition, you can add this capability in the engine by writing new facet Instance of type SffWdlFacet. This means now you can write Facet Script containing workflow definitions, which will be processed by engine at the time of deployment. By linking this to Http Server Facet, workflow can receive inputs on Http Channel and generate response on this channel as well.

Workflow Capability Instance
#
NewFacetInstance(fid: "[:RuntimeParams.WdlFacetIDName:]", Name: "wdl", Type: "SffWdlFacet"):
	OnActivate:
    	ImportFacet: [:RuntimeParams.MacroFacetID:]
    OnOpen(ModifyPipeline="HttpServerExtensionArgs"):
		#...
Topic - Standalone Use of Subscription and Notifications of Model changes. 

In A-Stack, every model artifact (model, attribute, instance) is defaulted to a topic in itself. Means clients can subscribe to any part of the model and start getting update events. In addition to model based subscriptions, independent pub-sub capability can be added in the engine, by writing SffTopicFacet Facet instance.

Topic Capability Instance
#
NewFacetInstance(fid: "[:RuntimeParams.TopicFacetIDName:]", name: "TQLGenericTopic", type: "SffTopicFacet"):
  OnActivate:
    DoRequest:
      Process:
        Message(type: "CDM"):
            Value:
              Subscribe(sid: "GenericTopicID", topic: "TQL.*"):
                Action:
                  logInfo: "CacheUpdate -> [:[:@Log:]$Request:]"
    OnOpen(ModifyPipeline: "HttpServerExtensionArgs"):
      #...
Sequence - Standalone Use of Scheduled jobs/task (Model code snippet)

Sequence provides the time based triggering, which can be against fixed time, delayed time, constant frequency, constant time intervals, etc.

Sequence Capability Instance
#
NewFacetInstance(fid: "[:RuntimeParams.SequenceFacetIDName:]", name: "seq", type: "SffSequenceFacet"):
  OnActivate:
    OnActivate:
      #...
    OnOpen ModifyPipeline: "HttpServerExtensionArgs")
      #...
TQL - Thing Query Language Usage over an WebSocket Transport

All models provide subscription capability on websocket and that is achieved by adding TQLFacet and connecting it with the websocket pipeline. Note that your model file(s) created using Model Editor is referenced in this section.

TQL Capability Instance over WebSocket
#
NewFacetInstance(fid: "[:RuntimeParams.WSFacetIDName:]", Name: "TQL", Type: "SffTqlFacet"):
    OnActivate:
      NewFacetInstance(name: "tqlwfws", type: "SffWdlFacet"):
      TopicFacet: TQLGenericTopic
      ImportFacet: [:RuntimeParams.MacroFacetID:]
      Process:
        Storage(Name: "[:RuntimeParams.TQLCacheName:]", Type: "SqlSff"):
           Comment: "[:RuntimeParams.TQLCacheName:] Database SFF Unstructured SQL database"
        Namespace: # Your Model file is referenced here
          TQLStudio.DbmLocation

    OnOpen(ModifyPipeline: "WsServerExtensionArgs")

    OnRequest:
      DoRequest:
        Process(Return: "CMD_NOP"):
          Message:
            Value:
              Include: $Request.Message.Value
      DoResponse:
        Process:
          Message(type: "[:RuntimeParams.TQLOutputFormat:]"):
            Value:
              Include:$Response.Message.Value
TQL - Thing Query Language Usage over an HTTP Transport

Normal queries are posted on http channel and to achieve this capability, TQLFacet is created with http pipeline extension.

  • Note that your model file(s) created using Model Editor is referenced in this section.
  • In this section there is a default FacetScript code to perform enforcing usage of Project Keys (if set in Project Settings Menu).

TQL Capability Instance over HTTP
#
NewFacetInstance(fid: "[:RuntimeParams.FacetIDName:]", Name: "TQL", Type: "SffTqlFacet"):
    OnActivate:
      Include: [:GetProjectModelsMacro:]
      Include: [:InstantiateModelMacro:]
      
      NewFacetInstance(name: "tqlwf", type: "SffWdlFacet"):
      TopicFacet: TQLGenericTopic
      ImportFacet: [:RuntimeParams.MacroFacetID:]
      Process:
        Storage(Name: "[:RuntimeParams.TQLCacheName:]", Type: "SqlSff"):
          Comment: "[:RuntimeParams.TQLCacheName:] Database SFF Unstructured SQL database"
        Namespace: TQLStudio.DbmLocation
        
    OnOpen(ModifyPipeline: "HttpServerExtensionArgs");
    
    OnRequest:
      SetContextData(key: "projectKey", value: "[:$Request.Arguments.x-atomiton-project-key:]")
      SetContextData(key: "projectSysId", value: "[:$Request.Arguments.x-atomiton-project-id:]")
      
      if("[:RuntimeParams.ProjectSettings_EnforceKey:]" == "true"):
        if($ContextData/projectKey == "[:RuntimeParams.ProjectSettings_ProjectKey:]"):        
              DoRequest:
                Process(Return: "CMD_NOP"):
                  Message:
                    Value:
                      Include: $Request.Message.Value
        else:
          SetResponse:
            Message(type: "[:RuntimeParams.TQLOutputFormat:]"):
              Value:
                Status: Failed
                Message: "Invalid Project Key"
      else:
        DoRequest:
          Process(Return: "CMD_NOP"):
            Message:
              Value:
                Include: $Request.Message.Value
                
      DoResponse:
        Process:
          Include: RESTHeaders
          Message(type: "[:RuntimeParams.TQLOutputFormat:]"):
            Value:
              Include: $Response.Message.Value

Note: The process of deploying Imported projects in A-Stack using User Interface (UI) is same as that of TQLStudio ThingSpaces.

For deployment directly to A-Stack using command line (without UI), see /wiki/spaces/TQLDocs/pages/1179774 in the Advanced Topics Section.

Instantiate

The final phase of Model development is actually using the Model using Queries. In some cases Models (Especially Thing / App Models) needs to be initialized to kick-start the whole process of interacting with Things or external application. In TQL, we use the term "Instantiate" to define the process of creating new instances on models. Model Instances can be created using either

  • Create TQL Query
  • Save TQL Query

Instantiate does the following tasks:

  • It creates an instance, or instances of a Model
  • It initialize the values of the model instances' attributes
  • If any of the attributes is an actionable attribute, the associated Action will be triggered (provided the attribute value is initialized)

For example - Here is the Instantiate Query to start the Phidget Servo Motor Thing Model. Since the Pidget Servo Motor Thing Model is combined with PhidgetServoFacet Thing Facet, and there is a action on ServoAngle attribute waiting for event (create/update) to occur, therefore creating an instance results in triggering of the Action. 

Instantiate Phidget Servo Model
#
Query:
	Create:
    	PhidgetServoModel:
      		ServoProtocolURL: 
				phid: "..."
      	PhidgetDeviceType: "PhidgetAdvancedServer"
      	DeviceInterfaceIndex: 0
      	ServoAngle: 110

You can instantiate in two ways.

(1) Instantiate by writing Save or Create TQL Queries in Query Editor.

(2) Instantiate using ThingSpace

ThingSpace provided easier way to instantiate the models. There are two ways to instantiate in ThingSpaces

a) Manual Entry of values

Once the model is instantiated using ThingSpaces it creates a file with Save Query automatically in Spaces folder of your project.

Save Query from Instantiate
#
Query:
	Save:
    	VendorInfo(instantiated: true):
      		vendorName: Phidget
      		vendorTitle: SensorProvider

b) Import Instantiate data using CSV file format.

CSV file need to have the columns in order, in which the definition is provided. If no value, you can leave the column empty. If no values for trailing attributes, you don't have to create columns.


Go to page

Instantiation

Here we use a query to instantiate the sensor, creating 1 instance of the TempSensor ThingModel. In the query, we first use "DeleteAll" to delete all the current instances of the TempSensor, if there are any. The Save will save the values into the corresponding model attributes  (inherited from the ThingFacet). These values provide the runtime parameters for the perif:// handler, as well as providing an update of the PerifMsg and TempC actionable attributes ($Null ( )), which will trigger the associated action SerialReadAction. The instantiation query below has been used on a setup where the engine is running locally on a Mac Pro and the Arduino board was directly connected to the Mac. 

Sensor instantiation
<Query>
	<DeleteAll format="version,current">
    	<TempSensor>
      		<sensorId ne=""/>
    	</TempSensor>
  	</DeleteAll>
  	<Save format="version,current">
    	<!-- This will read -->
    	<TempSensor>
      		<Peripheral>serial</Peripheral>
      		<Baudrate>9600</Baudrate>
      		<InterfacePort>/dev/cu.usbmodem1411</InterfacePort>
      		<Interface>serial</Interface>
      		<Format>ascii</Format>
      		<Operation>receive</Operation>
      		<UniqueId>76522</UniqueId>
      		<Payload>$Null()</Payload>
			<PerifMsg>$Null()</PerifMsg>
      		<TempC>$Null()</TempC>
    	</TempSensor>
  	</Save>
</Query>

Query and subscription

Now we can try a simple Find Query to get sensor values

Find TempSensor query
<Query>
    <Find format="version,known">
        <TempSensor>
            <sensorId ne="" />
        </TempSensor>
    </Find>
</Query>

You will see the result from the RESULTS window:

Find TempSensor query result
<Find Status="Success" Format="version,current">
    <Result>
        <TempSensor>
            <sensorId>KLHEPDF7AAAAUAABB4JDTW4S</sensorId>
            <interfacePort Value="/dev/cu.usbserial-A7030IWB" Version="1"/>
            <payload Value="" Version="1"/>
            <peripheral Value="serial" Version="1"/>
            <interface Value="serial" Version="1"/>
            <operation Value="receive" Version="1"/>
            <tempValue Value="" Version="1"/>
            <baudrate Value="115200" Version="1"/>
            <uniqueId Value="76522" Version="1"/>
            <format Value="ascii" Version="1"/>
        </TempSensor>
    </Result>
</Find>

 


 Queries

In TQL, a query is any operations on the model instance data (or model data), which includes filtering, joining, and manipulations. Model instances are incarnations of the Models (or model definitions) through instantiation. The A-Stack stores the latest version of the model instance data, which is accessible to TQL Queries. See data storage for more details.

There is a special type of query which allows you to show or list the model definitions themselves. See /wiki/spaces/TQLDocs/pages/1179668 for details.

Like conventional queries, TQL queries perform CRUD operations on data. but unlike conventional database queries, TQL queries are:

(1) Current. The A-Stack maintains the latest versions of any attribute values and attribute known values based on the model definitions. For example, the latest reading of a temperature sensor value. TQL queries always return those latest versions.

(2) Model-driven. Since the model definitions define the structure of the model data, the A-Stack constructs the database queries based on the model definition to retrieve requested data or perform other CRUD operations. The model definition is NOT the data schema. Model instance data are saved in schemaless key value pairs.

(3) Actionable. Modifications of model data via TQL queries, if involving actionable attributes, can result in activation of actions, some of which may subsequently execute device actuations.

Each model attribute has both a value and a known value. Although you can "FIND" both values and known values of model attributes, you can only set values but not known values from queries. This is because the known values are reserved to be updated only by the result of Actions, which comes from external systems or external things.

TQL queries have the following standard structures:

FIND          CREATE          SAVE          UPDATE          DELETE          DELETEALL

In addition,  you can use other operations such as conditions/constraints, response modifications, iterations, etc. within the queries.

Go to page


To subscribe to changes, use subscription is registered by through a query (query, create)


Subscribe to temperature sensor
<Query Storage='TqlSubscription'>
    <Save>
        <TqlSubscription Label='TempSensor' sid='20'>
            <Topic>*Atomiton.Sensor.TempSensor*</Topic>
        </TqlSubscription>
    </Save>
</Query>

Or you can subscribe to the specific attribute of the model.

Subscribe to temperature sensor attribute
<Topic>Atomiton.Sensor.TempSensor.TempC*</Topic>

The Label is the string that will be attached to every message you receive from this subscription. Here we give an sid to this instance of the subscription.

V. Source Code

Import into TQLStudio

ProjectNameImport Link
Temp Sensor SerialTempSensor


The full content of the code is here.

Serial temperature sensor
<Namespace Name="Atomiton">
  <Domain Name="Sensors">
    <ThingFacet Name="TempFacetSerial">
      <String Name="PerifMsg" Update="auto" KnownBy="SerialReadAction"/>
	  <String Name="TempC" Update="auto" KnownBy="SerialReadAction"/>
      <String Name="Unit"/>
      <String Name="Peripheral"/>
      <String Name="InterfacePort"/>
      <String Name="Interface"/>
      <String Name="UniqueId"/>
      <String Name="Baudrate"/>
      <String Name="Format"/>
      <String Name="Operation"/>
      <String Name="Payload"/>
      <Action Name="SerialReadAction">
        <Workflow Limit="1" Live="1" Timeout="-1">
          <Task Name="Main" While="true">
            <Event Name="Argument" As="ActionArgument"/>
            <Invoke Name="InvokeSerialRead" waitFor="Argument" scope="process" Get="perif://">
              <Message>
                <Value>
                  <InterfacePort>[%:Event.Argument.interfacePort.Value:%]</InterfacePort>
                  <Baudrate>[%:Event.Argument.Baudrate.Value:%]</Baudrate>
                  <Interface>[%:Event.Argument.Interface.Value:%]</Interface>
                  <UniqueId>[%:Event.Argument.UniqueId.Value:%]</UniqueId>
                  <Operation>[%:Event.Argument.Operation.Value:%]</Operation>
                  <Format>[%:Event.Argument.Format.Value:%]</Format>
                  <Payload>[%:Event.Argument.Payload.Value:%]</Payload>
                  <Peripheral>[%:Event.Argument.Peripheral.Value:%]</Peripheral>
                </Value>
              </Message>
            </Invoke>
            <Output Name="Result" As="ActionResult">
              <Value>
				<PerifMsg>[%:Invoke.InvokeSerialRead.Message.Value.received:%]</PerifMsg>
                <TempValue>[%:Invoke.InvokeSerialRead.Message.Value/substring("[%:Invoke.InvokeSerialRead.Message.Value.received:%]",7,5):%]</TempValue>
              </Value>
            </Output>
          </Task>
        </Workflow>
      </Action>
    </ThingFacet>
    <ThingModel Name="TempSensor" Combines="TempFacetSerial">
      <Sid Name="sensorId"/>
    </ThingModel>
  </Domain>
</Namespace>