5. USB camera
- jrenbb
- Alok Batra
- Baseer Khan
Key Concepts
Installing V4I4j
Hardware Setup
Hardware setup for USB Camera is as simple as plugging in the Camera to a USB port on the micro controller (Raspberry Pi or other).
Write a ThingFacet, Workflow and Device Logic (Message Transformation)
Write a ThingFacet
- To abstract the interactions with the USB Camera Setup in the previous section, we create a USBCameraFacet ThingFacet.
- Based on /wiki/spaces/TQLDocs/pages/1179871 result we can chose UCAM Protocol Handler for this purpose.
As described in writing ThingFacet section there are two kinds of attributes that must be added to a ThingFacet.
- Attributes that store Protocol Specific Parameters
- Attributes that are required to Control a Device (Actuator) OR Receive Data from a Device (Sensor).
For the purpose of this tutorial we have protocol specific parameters structured in the following composite types:
- ImageSize
<Def Name="ImageSize"> <Number Name="Height" Default="0"/> <Number Name="Width" Default="0"/> </Def>
- CameraConfig
<Def Name="CameraConfig"> <Number Name="Channel" Default="0"/> <Number Name="Standard" Default="0"/> <Number Name="Brightness" Default="0"/> <Number Name="Contrast" Default="0"/> <Number Name="Gamma" Default="286"/> <Number Name="Saturation" Default="65"/> <Number Name="Sharpness" Default="6"/> <Number Name="Hue" Default="0"/> <Number Name="Gain" Default="0"/> <Number Name="Quality" Default="80"/> </Def>
Types define the reusable structures for models. Types can be inherited and combined. Here we create a USBCameraParams type that combines/inherits the previous two types.
- USBCameraParams
<Def Name="USBCameraParams"> <ImageSize Name="Size"/> <CameraConfig Name="Configuration"/> <String Name="SaveToDisk" Default="true"/> <String Name="ImageFormat" Default="jpeg"/> <String Name="ImageName" Default="usb_camera.jpeg"/> </Def>
The only Actuation that can be performed with USB Camera is to take a picture. We will be controlling taking of picture using an actionable attribute:
- Image
<String Name="Image"/>
Putting attributes into a USBCameraFacet
<ThingFacet Name="USBCameraFacet">
<String Name="Image"/>
<USBCameraParams Name="USBCameraParameters"/>
</ThingFacet>
Write a Action with Workflow
The next step in ThingFacet is writing a Action which contains a workflow responsible for making external protocol specific call and store the output in the appropriate ThingFacet attribute. An important decision while writing Action is correct protocol selection. See the /wiki/spaces/TQLDocs/pages/1179871 in the Working with Things section. For this serial sensor we need the ucam:// Protocol Handler which is part of the TQLEngine.
1. Write an Action name as SerialReadAction
<ThingFacet Name="USBCameraFacet">
...
<Action Name="USBCameraImageAction" Documentation="Get the image from USB Camera">
</Action>
</ThingFacet>
2. Create a Workflow within the Action, with one single continuous running Task and waiting for the Event with its ActionArgument.
<Action Name="USBCameraImageAction">
<Workflow Limit="1" Live="1" Timeout="-1">
<Task name="Main" while="true">
<Event name="Argument" as="ActionArgument"/>
...
</Task>
</Workflow>
</Action>
Here we used three modifiers for this workflow. Limit = "1" means there can be at most one instance of this workflow waiting. Live = "1" means there can be at most one instance of this workflow running. Timeout ="-1" means this workflow will never be timed out. We used a modifier while = "true" with the Task to make the workflow running in a continuous loop, because it needs to run repeatedly, not just once. For more details, refer to workflow modifiers and the lifecycle of a workflow.
The task will be activated by the event handler Event called ActionArgument. "ActionArgument" is the event generated whenever the attribute(s) associated with this Action is modified (See Associate Action with a ThingFacet Attribute). ActionArgument carries all the current values of the ThingFacet attributes, which can be used in the task if needed.
3. Invoke UCAM call: Method of UCAM is POST. We use Invoke modifier Post for this purpose. The parameters required UCAM are passed as part of <Message> and <Value>. More details of the ucam:// handler are here in the Working with Things Section.
<Invoke name="GetImage" waitFor="ActionArgument" post="ucam://">
<Message>
<Value>
<USBCameraParameters>
<ImageFormat>
[%:Event.Argument.USBCameraParameters.ImageFormat.Value:%]
</ImageFormat>
<SaveToDisk>
[%:Event.Argument.USBCameraParameters.SaveToDisk.Value:%]
</SaveToDisk>
<ImageName>
[%:Event.Argument.USBCameraParameters.ImageName.Value:%]
</ImageName>
<Size>
<Height>
[%:Event.Argument.USBCameraParameters.Size.Height.Value:%]
</Height>
<Width>
[%:Event.Argument.USBCameraParameters.Size.Width.Value:%]
</Width>
</Size>
<Configuration>
<Standard>
[%:Event.Argument.USBCameraParameters.Configuration.Standard.Value:%]
</Standard>
<Quality>
[%:Event.Argument.USBCameraParameters.Configuration.Quality.Value:%]
</Quality>
<Channel>
[%:Event.Argument.USBCameraParameters.Configuration.Channel.Value:%]
</Channel>
<Brightness>
[%:Event.Argument.USBCameraParameters.Configuration.Brightness.Value:%]
</Brightness>
<Contrast>
[%:Event.Argument.USBCameraParameters.Configuration.Contrast.Value:%]
</Contrast>
<Gamma>
[%:Event.Argument.USBCameraParameters.Configuration.Gamma.Value:%]
</Gamma>
<Gain>
[%:Event.Argument.USBCameraParameters.Configuration.Gain.Value:%]
</Gain>
<Sharpness>
[%:Event.Argument.USBCameraParameters.Configuration.Sharpness.Value:%]
</Sharpness>
<Saturation>
[%:Event.Argument.USBCameraParameters.Configuration.Saturation.Value:%]
</Saturation>
<Hue>
[%:Event.Argument.USBCameraParameters.Configuration.Hue.Value:%]
</Hue>
</Configuration>
</USBCameraParameters>
</Value>
</Message>
</Invoke>
Here Invoke gets the attribute values that the Event ActionArgument (or Argument) carries. It uses them as the parameter values for the UCAM by substitution. For example:
<Hue>[%:Event.Argument.USBCameraParameters.Configuration.Hue.Value:%]</Hue>
means replacing the value of Message.Value.Configuraiton.Hue with the value of Event.Argument.USBCameraParameters.Configuration.Hue.Value
Value substitution (rather than assignment) is very common in TQL, given that it is a declarative language. The notation [%: is part of the /wiki/spaces/TEACH/pages/21170773. More details on /wiki/spaces/TEACH/pages/21170773 can be found in the Developer's Guide.
4. Process the message received from the sensor
<Log Message="******Invoked USBCamera******"/>
<Output name="Result" as="ActionResult">
<Value>
<Image>
[%:[%:@Output:%]Invoke.GetImage.Message.Value:%]
</Image>
</Value>
</Output>
Here we used an XPath statement to pass the message. Alternatively, you can use Javascript. An example of using JavaScript for message processing within an Action can be found in another tutorial here.
We used a TP notation "@" in @Outupt. It means the value should be substituted after the process is completed and not before. More details on /wiki/spaces/TEACH/pages/21170773 can be found in the Developer's Guide.
Associate Action with a ThingFacet Attribute
We will now have to associate a ThingFacet Attribute (Image) with the Action using the KnownBy modifier. When Image is "KnownBy" USBCameraImageAction, any Image value changes will activate the USBCameraImageAction.
<ThingFacet Name="USBCameraFacet">
<String Name="Image" KnownBy="USBCameraImageAction"/>
<USBCameraParams Name="USBCameraParameters"/>
</ThingFacet>
Combining ThingFacet with a ThingModel
Finally, in order to use ThingFacet we have to combine it with a ThingModel. We define ThingModel to contain only a unique system identifier.
<ThingModel name="USBCameraModel" combines="USBCameraFacet"> <Sid name="camId"/> </ThingModel>
When USBCameraModel is combined with USBCameraFacet, the USBCameraModel inherits all the attributes of USBCameraFacet. USBCameraModel is be instantiated (Create TQL Query). Note that ThingFacet's can not be instantiated. The ThingFacet USBCameraFacet hence serves as a reusable component.
More details on the use of "combines" can be found here. Information on Sid cab be found here.
Deploy and test via queries and subscriptions
Export, import and deploy
Once the models are built in a TQLStudio project, you can export the project. URL of the Zip file containing the content of your project will be sent over to your email account. Once you have downloaded the engine – launch the TQLEngine User Interface in the browser and select Import Project. You can copy the URL from Export Project Email and hit import button. Go to ThingSpaces and Deploy the project.
Instantiate Camera and Take Test Picture
Initialize the Camera and take a First Test Picture:
<Query>
<DeleteAll>
<USBCameraModel>
<camId ne=""/>
</USBCameraModel>
</DeleteAll>
<Save>
<USBCameraModel>
<Image>
$Null()
</Image>
<USBCameraParameters>
<Size>
<Height>
0
</Height>
<Width>
0
</Width>
</Size>
<Configuration>
<Channel>
0
</Channel>
<Standard>
0
</Standard>
<Brightness>
0
</Brightness>
<Contrast>
0
</Contrast>
<Gamma>
286
</Gamma>
<Saturation>
65
</Saturation>
<Sharpness>
6
</Sharpness>
<Hue>
0
</Hue>
<Gain>
0
</Gain>
<Quality>
</Quality>
</Configuration>
<SaveToDisk>
true
</SaveToDisk>
<ImageFormat>
jpeg
</ImageFormat>
<ImageName>
ucam_testpic.jpeg
</ImageName>
</USBCameraParameters>
</USBCameraModel>
</Save>
</Query>
Take a new Picture
Each time we need to take a picture we can set the Image attribute to NULL (Different from Known). We can also modify other parameters if desired. For the sake of simplicity we will be changing only the file name where the image gets stored.
<Query>
<Find format="version">
<USBCameraModel>
<CamId ne=""/>
</USBCameraModel>
</Find>
<SetResponseData>
<key>
Message.Value.Find.Result.USBCameraModel.Image.Value
</key>
<value>
$Null()
</value>
</SetResponseData>
<SetResponseData>
<key>
Message.Value.Find.Result.USBCameraModel.USBCameraParameters.ImageName.Value
</key>
<value>
ucam_baseer.jpeg
</value>
</SetResponseData>
<Update>
<from>
Result
</from>
<Include>
$Response.Message.Value.Find
</Include>
</Update>
</Query>
You will see the result from the RESULT window:
Serving Static Files :
There are two ways in which you can serve static files in TQLEngine.
a) Attaching an Application from the ThingSpace of TQLSudio
For Example:
Assume that want to serve image stored in current folder by UCAM Facet.
- App URL - imageview (any name can be given)
- BasePath - . (Current folder)
- Resource and TailPath is empty.
The URL Generated will be:
http://<host/ip of your engine>:<port of your engine>/fid-<App URI Name give>/<Resource if any>/
In the case of UCAM the files can be served as:
http://10.0.1.52:8080/fid-imageview/ucam_baseer.jpg
Assuming Engine is at 10.0.1.52, 8080 (IP / Port combo).
b) Using NewFacetInstance as part of your Package File (**.mpq.xml)
<<NewFacetInstance fid="tsm" name="file" type="SffHttpStaticFileFacet">
<OnActivate>
<Process>
<BasePath>[:AppBase:]</BasePath>
<TailPath>index.html</TailPath>
<Resource>res</Resource>
</Process>
</OnActivate>
</NewFacetInstance>
Source Code
Import into TQLStudio
| ProjectName | Import Link |
|---|---|
| USB Camera Model | USBCamera |
Complete Source Code