Cloud Service Account Setup
Most of the Cloud Service Providers require you to setup an account with their service, generate authorization key and get access to their APIs. For the purpose of this tutorial we have selected - http://openweathermap.org/current as a cloud service provider to gather external environment data.
OpenWeatherMap provides current weather data for any location on Earth including over 200,000 cities. OpenWeather frequently updates current weather based on global models and data from more than 40,000 weather stations. Data is available in JSON, XML or HTML format.
Pre-registration:
1. Signup for the user account on openweathermap.
2. Make a note of the API key that is provided after successful signup. This key will be used to invoke openweathermap APIs.
3. Select the list of APIs to be integrated in the TQL Model (ThingFacets). For this tutorial we have selected API to get temperature, humidity, light, pressure for a given City and Country.
| API Name | Parameters | Response |
|---|---|---|
| http://api.openweathermap.org/data/2.5/weather | q - Query String. Example, Santa Clara,CA mode - XML or JSON or HTML APPID - Your API Key from step (2) | Cloud Service API Response <current>
<city id="5393015" name="Santa Clara">
<coord lon="-121.96" lat="37.35"/>
<country>US</country>
<sun rise="2016-03-17T14:13:28" set="2016-03-18T02:18:25"/>
</city>
<temperature value="297.67" min="296.15" max="301.15" unit="kelvin"/>
<humidity value="41" unit="%"/>
<pressure value="1012" unit="hPa"/>
<wind>
<speed value="3.1" name="Light breeze"/>
<gusts/>
<direction value="0" code="N" name="North"/>
</wind>
<clouds value="5" name="clear sky"/>
<visibility value="16093"/>
<precipitation mode="no"/>
<weather number="800" value="clear sky" icon="02d"/>
<lastupdate value="2016-03-17T23:30:57"/>
</current>
|
Write a ThingFacet
To abstract the interactions with the OpenWeatherMap Cloud Service we write a ExternalEnvFacet ThingFacet.
There are two kinds of attributes that are part of a ThingFacet.
- Attributes required to make a protocol specific invocation
- Attributes to store the information received from the cloud service API as part of the response.
Protocol Specific Parameters are: Cloud Service Base URL, API Key, Output Format and Query Parameter.
Let's create Type (Using Def) EnvInfo to store the partial list of the API response
<Def Name="EnvInfo"> <Double Name="Temperature"/> <Double Name="Humidity"/> <Double Name="Light"/> <Double Name="Pressure"/> <String Name="LastUpdatedByProvider"/> </Def>
Lets now create a ExternalEnvFacet ThingFacet using the EnvInfo type.
<ThingFacet Name="ExternalEnvFacet"> <String Name="DCBaseURL"/> <String Name="DCAPIKey"/> <String Name="OutputFormat"/> <String Name="DCQueryParam"/> <Integer Name="SensingInterval"/> <EnvInfo Name="EnvDetails"/> </ThingFacet>
Write a Action with Workflow
The next step in ThingFacet is writing a Action which contains a workflow responsible for making external protocol specific call and store the output in the appropriate ThingFacet attribute. An important decision while writing Action is correct protocol selection. See the /wiki/spaces/TQLDocs/pages/1179871 in the Working with Things section. For this OpenWeatherMap Service we need simple HTTP request, which is part of the TQLEngine.
1. Write an Action name as EnvInfoAction
<ThingFacet Name="ExternalEnvFacet"> ... <Action Name="EnvInfoAction"> ... </Action> </ThingFacet>
2. Create a Workflow within the Action, with one single continuous running Task and waiting for the Event with its ActionArgument.
<Action Name="EnvInfoAction">
<Workflow Limit="1" Live="1" Timeout="-1">
<Task name="Main" while="true">
<Event name="Argument" as="ActionArgument"/>
...
</Task>
</Workflow>
</Action>
Here we used three modifiers for this workflow. Limit = "1" means there can be at most one instance of this workflow waiting. Live = "1" means there can be at most one instance of this workflow running. Timeout ="-1" means this workflow will never be timed out. We used a modifier while = "true" with the Task to make the workflow running in a continuous loop, because it needs to run repeatedly, not just once. For more details, refer to workflow modifiers and the lifecycle of a workflow.
The task will be activated by the event handler Event called ActionArgument. "ActionArgument" is the event generated whenever the attribute(s) associated with this Action is modified (See Associate Action with a ThingFacet Attribute). ActionArgument carries all the current values of the ThingFacet attributes, which can be used in the task if needed.
3. Invoke HTTP call: Method of HTTP is Get. We use Invoke modifier Get for this purpose. The parameters required for HTTP Get are passed as directly as the value of Get.
<Invoke Name="ReadValues" WaitFor="Argument" Documentation="Invoke HTTP GET"
Get="[%:Event.Argument.DCBaseURL.Value:%]?q=[%:Event.Argument.DCQueryParam.Value:%]&mode=[%:Event.Argument.OutputFormat.Value:%]&APPID=[%:Event.Argument.DCAPIKey.Value:%]"/>
Note that the Event Parameter values are dereferenced using TP Notation. The notation [%: is part of the /wiki/spaces/TEACH/pages/21170773. More details on /wiki/spaces/TEACH/pages/21170773 can be found in the Developer's Guide.
| Tag | Resolution |
|---|---|
[%:Event.Argument.DCBaseURL.Value:%] | DCBaseURL value passed when creating/updating model instance. |
[%:Event.Argument.DCQueryParam.Value:%] | DCQueryParam value passed when creating/updating model instance. |
& | Escape the XML "&". Remember that the query string is q=xxx&... here & must be escaped. |
[%:Event.Argument.OutputFormat.Value:%] | OutputFormat value passed when creating/updating model instance. |
[%:Event.Argument.DCAPIKey.Value:%] | DCAPIKey value passed when creating/updating model instance. |
4. Process Response.
- Check if HTTP code is 200 else log the error message
Convert the temperature value from Kelvin to Centigrade.
HandleResponse to GET<Invoke Name="HandleResponse"> <FacetScript> <If Condition="/'[%:Invoke.ReadValues.Status:%]' eq '200'"> <Then> <Log Message="Response Status is 200"/> <If Condition="/'[%:Invoke.ReadValues.Message.Value.current.temperature.unit:%]' eq 'kelvin'"> <Then> <Log Message="Temperature Unit is Kelvin.."/> <SetContextData Key="CelTemp" Value="[%:/number([%:Invoke.ReadValues.Message.Value.current.temperature.value:%]-273):]"/> <Log Message="Temperature Value in Centigrade is.. [:$ContextData.CelTemp:]"/> </Then> <Else> <SetContextData Key="CelTemp" Value="[%:Invoke.ReadValues.Message.Value.current.temperature.value:%]"/> </Else> </If> </Then> <Else> <Log Message="Non-200 Response Status. Send Alert to DevOps"/> </Else> </If> </FacetScript> </Invoke>Tag Resolution "/'[%:Invoke.ReadValues.Message.Value.current.temperature.unit:%]' eq 'kelvin'"
This is an XPath Expression to compare string. Example: "/<StringValue>' eq 'StringValue2' SetContextData
This is a temporary key/value storage provided by TQLEngine as part of FacetScript. Here Key is CelTemp, Value is converted value to Centigrade to Original Value (if original value is not in Kelvin). [%:/number([%:Invoke.ReadValues.Message.Value.current.temperature.value:%]-273):]
This is XPath Expression to perform Subtraction.
5. Store the output value in EvnDetails Attribute.
<Output Name="Result" As="ActionResult">
<Value>
<EnvDetails>
<Temperature>[:$ContextData.CelTemp:]</Temperature>
<Humidity>[%:Invoke.ReadValues.Message.Value.current.humidity.value:%]</Humidity>
<Pressure>[%:Invoke.ReadValues.Message.Value.current.pressure.value:%]</Pressure>
<LastUpdatedByProvider>[%:Invoke.ReadValues.Message.Value.current.lastupdate.value:%]</LastUpdatedByProvider>
</EnvDetails>
</Value>
</Output>
Associate Action with a ThingFacet Attribute
We will now have to associate a ThingFacet Attribute (State) with the Action using the KnownBy modifier. When State is "KnownBy" PhilipLightAction, any State value changes will activate the PhilipLightAction.
<ThingFacet Name="ExternalEnvFacet"> ... <EnvInfo Name="EnvDetails" KnownBy="EnvInfoAction"/> </ThingFacet>
The attribute modifier update= "auto" makes sure that once the action associated with this attribute is triggered, its workflow continues to run and wait for subsequent sensor events (not just the first event). This modifier is only used with actionable attributes. For more details, refer to Automatic Action trigger.
Combining ThingFacet with a ThingModel
Finally, in order to use ThingFacet we have to combine it with a ThingModel. We define ThingModel to contain only a unique system identifier.
<ThingModel Name="ExternalEnv" Combines="ExternalEnvFacet">
<Sid Name="ExtEnvId"/>
</ThingModel>
ExternalEnvFacet can only be instantiated (and write TQL Queries) when it is combined with ExternalEnv Thing Model. ExternalEnv will inherit all the attributes from ExternalEnvFacet, in addition to its own attributes. The ThingFacet ExternalEnvFacet hence serves as a reusable component.
More details on the use of "combines" can be found here. Information on Sid can be found here.
Checking Weather Periodically
Let's assume that you would like to check the weather periodically - say every morning or based on some other external factors. We can automatically add the behavior of checking periodically by triggering the EnvInfoAction.
The building blocks to Checking Weather Periodically are:
- Ability to reset the current value of EnvInfo Attribute so that the action is triggered.
- Ability to schedule the reset code at certain periodic interval.
TQLEngine allows us to run TQL Queries from the Model itself using <OnRequest> Tag. We need to provide the Target ID to make this request. Since queries can be called any number of times, it is always a good idea to wrap calling of a TQL Query in a generic Macro Definition. Let's call this ExecuteQuery.
Generic Macro to Execute a TQL Query
<Macro Name="ExecuteQuery">
<Argument>
<QueryString>
<Query/>
</QueryString>
</Argument>
<Result>
<OnRequest>
<Target>[:RuntimeParams.FacetIDName:]</Target>
<Process>
<Message>
<Value>[:$Macro.Argument.QueryString:]</Value>
</Message>
</Process>
</OnRequest>
</Result>
</Macro>
Macro to Reset EnvInfo values i.e. Update Query
We can call the Update Query to reset the EnvInfo parameter. Let's wrap this update query in another Macro called ResetEnvInfo
<Macro Name="ResetEnvInfo">
<Argument>
<ExternalEnvId/>
</Argument>
<Result>
<Log Message="Resetting External Env: [:$Macro.Argument.ExternalEnvId:]"/>
<ExecuteQuery>
<QueryString>
<Query>
<Find Format="all">
<ExternalEnv>
<ExtEnvId>[:$Macro.Argument.ExternalEnvId:]</ExtEnvId>
</ExternalEnv>
</Find>
<SetResponseData>
<Key>Message.Value.Find.Result.ExternalEnv.EnvDetails.Temperature.Value</Key>
<Value>$Null()</Value>
</SetResponseData>
<SetResponseData>
<Key>Message.Value.Find.Result.ExternalEnv.EnvDetails.Humidity.Value</Key>
<Value>$Null()</Value>
</SetResponseData>
<SetResponseData>
<Key>Message.Value.Find.Result.ExternalEnv.EnvDetails.Pressure.Value</Key>
<Value>$Null()</Value>
</SetResponseData>
<SetResponseData>
<Key>Message.Value.Find.Result.ExternalEnv.EnvDetails.LastUpdatedByProvider.Value</Key>
<Value>$Null()</Value>
</SetResponseData>
<Update>
<From>Result</From>
<Include>$Response.Message.Value.Find</Include>
</Update>
</Query>
</QueryString>
</ExecuteQuery>
</Result>
</Macro>
Scheduling ResetEnvInfo Macro
Now we need a mechanism to schedule a piece of Modeling code to run periodically. In this case we schedule RsetEnvInfo Macro. TQLEngine provides Scheduling mechanism using Sequence Capability. Similar to Executing a TQL Query from a Model we need to schedule and unschedule code by firing a request to Sequence Target. Let's write two macros to handle schedule and deleteschedule models.
Generic Macro to Schedule
<Macro Name="ScheduleJob">
<Argument>
<ScheduleInterval>
</ScheduleInterval>
<StartTime>0</StartTime>
<EndTime>
</EndTime>
<ActionCode>
</ActionCode>
<Name>
</Name>
</Argument>
<Result>
<Log Message="Inside ScheduleJob macro"/>
<DoRequest target="[:RuntimeParams.SequenceFacetIDName:]">
<Process>
<Message>
<Value>
<Execute eid="[:$Macro.Argument.Name:]" schedule="[[:$Macro.Argument.StartTime:]..[:$Macro.Argument.EndTime:]/[:$Macro.Argument.ScheduleInterval:]]">
<Action>
<ExecuteQuery>
<QueryString>[:$Macro.Argument.ActionCode:]</QueryString>
</ExecuteQuery>
</Action>
</Execute>
</Value>
</Message>
</Process>
</DoRequest>
</Result>
</Macro>
Generic Macro to Delete Schedule
<Macro Name="DeleteScheduleJob">
<Argument>
<JobName/>
</Argument>
<Result>
<DoRequest target="[:RuntimeParams.SequenceFacetIDName:]">
<Process>
<Message type="xml">
<Value>
<Remove eid="[:$Macro.Argument.JobName:]"/>
</Value>
</Message>
</Process>
</DoRequest>
</Result>
</Macro>
Start / Stop Scheduling Action
So far, we have ability to run a resetEnvInfo Query, Schedule resetEnvInfo, and UnSchedule resetEnvInfo. We need a mechanism to start and stop the schedule. We can achieve this using a attribute and an associated action. For this purpose let's add an attribute called EmulateSensing with an action EmulateSensingAction to EnvInfoFacet. The Pseudo logic to be implemented in this action is:
if EmulateSensing is true
schedule resetInfo Macro
else
unschedule resetInfo Macro
<String Name="EmulateSensing" KnownBy="EmulateSensingAction"/>
<Action Name="EmulateSensingAction">
<Workflow Limit="1" Live="1" Timeout="0">
<Task Name="GetEnvInfo" While="true">
<Event Name="Argument" As="ActionArgument"/>
<Invoke Name="ToggleSchedule" WaitFor="Argument">
<FacetScript>
<If condition="/'[%:Event.Argument.EmulateSensing.Value:%]' eq 'true'">
<then>
<ScheduleJob>
<Name>UpdateEnvDetails</Name>
<ScheduleInterval>[%:Event.Argument.SensingInterval.Value:%]sec</ScheduleInterval>
<ActionCode>
<ResetEnvInfo>
<ExternalEnvId>[%:Event.Argument.ExtEnvId:%]</ExternalEnvId>
</ResetEnvInfo>
</ActionCode>
</ScheduleJob>
</then>
<else>
<Log Message="Inside DeleteScheduleJob else condition ======= "/>
<DeleteScheduleJob>
<JobName>UpdateEnvDetails</JobName>
</DeleteScheduleJob>
</else>
</If>
</FacetScript>
</Invoke>
<Output Name="Result" As="ActionResult">
<Value>
<EmulateSensing>[%:Event.Argument.EmulateSensing.Value:%]</EmulateSensing>
</Value>
</Output>
</Task>
</Workflow>
</Action>
Query and subscription
Queries we need are:
- Find External Env Weather Information
- Initialize External Env Weather Model
- Subscribe to External Env Weather Model
- Start Sensing (i.e. Simulation)
- Stop Sensing (i.e. Simulation)
Queries
<Query>
<Find format="known,version">
<ExternalEnv>
<DCBaseURL ne=""/>
</ExternalEnv>
</Find>
</Query>
<Query>
<DeleteAll>
<ExternalEnv>
<DCBaseURL ne=""/>
</ExternalEnv>
</DeleteAll>
<Create>
<ExternalEnv>
<DCBaseURL>
http://api.openweathermap.org/data/2.5/weather
</DCBaseURL>
<DCAPIKey>
2b8f32bc27def9f56804c03516516f4d
</DCAPIKey>
<OutputFormat>
xml
</OutputFormat>
<DCQueryParam>
Pune,IN
</DCQueryParam>
<EnvDetails>
<Temperature>
0
</Temperature>
<Humidity>
0
</Humidity>
<Light>
0
</Light>
<Pressure>
0
</Pressure>
<LastUpdatedByProvider>
0
</LastUpdatedByProvider>
</EnvDetails>
<EmulateSensing>
False
</EmulateSensing>
<SensingInterval>
60
</SensingInterval>
</ExternalEnv>
</Create>
</Query>
<Query>
<Find format="known, version">
<ExternalEnv>
<DCBaseURL ne=""/>
</ExternalEnv>
</Find>
<SetResponseData>
<Key>
Message.Value.Find.Result.ExternalEnv.EmulateSensing.Value
</Key>
<Value>
true
</Value>
</SetResponseData>
<Update>
<From>
Result
</From>
<Include>
$Response.Message.Value.Find
</Include>
</Update>
</Query>
<Query>
<Find format="known, version">
<ExternalEnv>
<DCBaseURL ne=""/>
</ExternalEnv>
</Find>
<SetResponseData>
<Key>
Message.Value.Find.Result.ExternalEnv.EmulateSensing.Value
</Key>
<Value>
False
</Value>
</SetResponseData>
<Update>
<From>
Result
</From>
<Include>
$Response.Message.Value.Find
</Include>
</Update>
</Query>
<Query Storage="TqlSubscription">
<Save>
<TqlSubscription Label="Changes To External Env" sid="1">
<Topic>
*Atomiton.ExternalEnvService.ExternalEnv*
</Topic>
</TqlSubscription>
</Save>
</Query>