Protocol handlers

Owned by jrenbb
Last updated: Jan 12, 2017 by Baseer Khan

What is a protocol handler in TQL

The run-time environment of Atomic Domain Languages (TQL, Workflow etc) is A-Stack. A-Stack is an asynchronous event-driven framework with network communication at its core for the rapid development of high-performance and high-scale IoT Applications. All communications between devices require that the devices agree on the format of the data. The set of rules defining a format is called a protocol. A-Stack protocol handler is one of the key extension point that handles communication with external devices. Protocol Handlers does the following: 

  • handle whether synchronous or asynchronous
  • encoding and decoding of data
  • detect and recover from transmission errors

A-Stack uses Non-blocking I/O (NIO) mechanism at its core to provide access to low-level I/O operations of modern Operating Systems. A-Stack greatly simplifies and streamlines network programming such as TCP and UDP socket server development. A-Stack implements a lot of basic protocols such as HTTP, WebSocket. Adding new protocol support in easier without compromising on performance, stability, and flexibility.

Understanding some of the core A-Stack concepts is useful in realizing what makes writing a new Protocol Handler much easier -

Buffer

At the lowest level A-Stack implements Zero-Copy feature. Zero-Copy avoids context switches between Kernel and Application space. Often copying is not necessary. If data is not modified a slice can be passed forward without copying to a different buffer. Buffer is a random and sequential accessible sequence of zero or more bytes (octets). Buffer provides an abstract view for one or more primitive byte arrays (byte[]). In the context of A-Stack buffer are normally referred to as ChannelBuffer because everything with A-Stack is a channel (see below).

Channel

A basic abstraction of a network through which the bytes in a buffer are transported is called a channel. A channel represents an open connection to an entity such as a hardware device, a file, a network socket, or a program component that is capable of performing one or more distinct I/O operations, for example reading or writing. A channel is capable of:

  • Read - reading data from buffer
  • Write - writing data to buffer
  • Connect - open a connection to the device
  • Bind - bind the connection
  • Disconnect - close a connection to the device
Event

Almost everything that happens within A-Stack occurs asynchronously. There are core driving code patterns that generate events (like Connect, Disconnect and Write) and bits of code that handle those events once they've been executed. All these events in the A-Stack are instances of ChannelEvent and the code that handles these events are called as ChannelHandlers. In order to implement communication protocols we need handlers that can perform encoding and decoding of messages.

  •     Encoder: Converts a non ChannelBuffer object (Raw data stream or payload) into a ChannelBuffer, suitable for transmission to somewhere else. It might not encode directly to a ChannelBuffer, rather, it might do a partial conversion, implicitly relying on another handler[s] to do the conversion.  An example of a encoder is say PhidgetDataEncoder which converts regular data from Phidget Sensors / Actuators into ChannelBuffers containing the meaningful representation of the read data.
  •     Decoder: The reverse of an encoder where a ChannelBuffer's contents are converted into something more useful. The counterpart of the PhidgetDataEncoder mentioned above, is the PhidgetDataDecoder and it does exactly this.


Pipeline

Putting it the above concepts together we have a construct called Pipeline (or more specifically, ChannelPipeline). A pipeline is a stack of handlers that can manipulate or transform the values that are passed to them. Then, when they're done, they pass the value on to the next handlers. In order to achieve the proper sequence of modifications of the payload, the pipeline keeps strict ordering of the handlers in the pipeline. Another aspect of pipelines is that they are not immutable*, so it is possible to add and remove handlers at runtime. This feature comes in handy while implementing certain types of device communication.


 
Deployment View

 

In order to make A-Stack extensible it provides a capability to load extra bundles after engine core start and initial configuration as those extension bundles may come from different places defined by configuration. It is also necessary to be able to instantiate those bundles when needed i.e. on-demand. The bundlers are automatically configured to be loaded from  "sff.auto.launch" folder in the current folder where the core A-Stack is running.

 


Protocol Handler Usage

There are two ways in which a Protocol Handler can be used

  • Within workflow Invoke using a protocol prefix.
  • Using FacetScript to open an explicit connection to the Protocol
Invoking a Protocol Handler

In order to invoke a protocol handler you need the protocol prefix that is provided by the handler as well the Request / Response payload format.

For example, let's make a request to a Serial Communication Protocol Handler provided by A-Stack. The protocol prefix provided by the handler is - perif. Method of invoke is "Get". The argument consists of InterfacePort, Baudrate, Interface, UniqueId etc. 

Request to Perif Serial Handler
Invoking Serial (Perif) Handler
<Invoke name="InvokeSerialRead" waitFor="Argument" get="perif://">
    <Message>
        <Value>
            <InterfacePort>"[%:Event.Argument.interfacePort.Value:%]"
            </InterfacePort>
            <Baudrate>"[%:Event.Argument.Baudrate.Value:%]"</Baudrate>
            <Interface>"[%:Event.Argument.Interface.Value:%]"</Interface>
            <UniqueId>"[%:Event.Argument.UniqueId.Value:%]"</UniqueId>
            <Operation>"[%:Event.Argument.Operation.Value:%]"</Operation>
            <format>"[%:Event.Argument.Format.Value:%]"</format>
            <Payload>"[%:Event.Argument.Payload.Value:%]"</Payload>
            <Peripheral>"[%:Event.Argument.Peripheral.Value:%]"</Peripheral>
        </Value>
    </Message>
</Invoke>

Response from Perif Serial Handler can be retrieved using Message / Value container of request Invoke i.e. InvokeSerialRead

Response Serial (Perif) Handler
<TempValue>

   [%:Invoke.InvokeSerialRead.Message.Value:%]
</TempValue>


Explicit Connection using FacetScript

Protocol Handlers can also be opened manually at the time of the deployment using CreatePipeline command of Facetscript at the time creating a new Facet Instance. In order to open an explicit connection we need the Protocol Handler's Extension Argument Name. This name will be provided in the specific documentation section of the custom protocol Handler. For example,  PeripheralServerExtensionArgs is the name of the extension argument for Perif Handler.

Explicit Connection to Perif Handler
<NewFacetInstance fid="wstest" name="wstest" type="SffMsgFacet">
  <OnOpen ModifyPipeline="WsServerExtensionArgs"/>
</NewFacetInstance>


<NewFacetInstance fid="tempsensor" name="xbee" type="SffTcpFacet">
  <OnActivate>
    <CreatePipeline arguments="PeripheralServerExtensionArgs" Peripheral="serial" InterfacePort="/dev/cu.usbserial-AL01C1HO" Interface="serial" Format="ascii" Baudrate="'9600'" Operation="recieve" uniqueId="76522" Payload="$Null()" TempValue="$Null()"/>
  </OnActivate>
  <OnRequest>
    <DoResponse target="wstest" process="$Request"/>
  </OnRequest>
</NewFacetInstance>

In this example, we open a connection to Perif using CreatePipeline and then once the data is received we transit it to a WebSocket Server Facet Instance.