What is a protocol handler in TQL
The run-time environment of Atomic Domain Languages (TQL, Workflow etc) is TQLEngine. TQLEngine is an asynchronous event-driven framework with network communication at its core for the rapid development of high-performance and high-scale IoT Applications. All communications between devices require that the devices agree on the format of the data. The set of rules defining a format is called a protocol. TQLEngine protocol handler is one of the key extension point that handles communication with external devices. Protocol Handlers does the following:
- handle whether synchronous or asynchronous
- encoding and decoding of data
- detect and recover from transmission errors
TQLEngine uses Non-blocking I/O (NIO) mechanism at its core to provide access to low-level I/O operations of modern Operating Systems. TQLEngine greatly simplifies and streamlines network programming such as TCP and UDP socket server development. TQLEngine implements a lot of basic protocols such as HTTP, WebSocket. Adding new protocol support in easier without compromising on performance, stability, and flexibility.
Understanding some of the core TQLEngine concepts is useful in realizing what makes writing a new Protocol Handler much easier -
Buffer
At the lowest level TQLEngine implements Zero-Copy feature. Zero-Copy avoids context switches between Kernel and Application space. Often copying is not necessary. If data is not modified a slice can be passed forward without copying to a different buffer. Buffer is a random and sequential accessible sequence of zero or more bytes (octets). Buffer provides an abstract view for one or more primitive byte arrays (byte[]). In the context of TQLEngine buffer are normally referred to as ChannelBuffer because everything with TQLEngine is a channel (see below).
Channel
A basic abstraction of a network through which the bytes in a buffer are transported is called a channel. A channel represents an open connection to an entity such as a hardware device, a file, a network socket, or a program component that is capable of performing one or more distinct I/O operations, for example reading or writing. A channel is capable of:
- Read - reading data from buffer
- Write - writing data to buffer
- Connect - open a connection to the device
- Bind - bind the connection
- Disconnect - close a connection to the device
Event
Almost everything that happens within TQLEngine occurs asynchronously. There are core driving code patterns that generate events (like Connect, Disconnect and Write) and bits of code that handle those events once they've been executed. All these events in the TQLEngine are instances of ChannelEvent and the code that handles these events are called as ChannelHandlers. In order to implement communication protocols we need handlers that can perform encoding and decoding of messages.
- Encoder: Converts a non ChannelBuffer object (Raw data stream or payload) into a ChannelBuffer, suitable for transmission to somewhere else. It might not encode directly to a ChannelBuffer, rather, it might do a partial conversion, implicitly relying on another handler[s] to do the conversion. An example of a encoder is say PhidgetDataEncoder which converts regular data from Phidget Sensors / Actuators into ChannelBuffers containing the meaningful representation of the read data.
- Decoder: The reverse of an encoder where a ChannelBuffer's contents are converted into something more useful. The counterpart of the PhidgetDataEncoder mentioned above, is the PhidgetDataDecoder and it does exactly this.
Pipeline
Putting it the above concepts together we have a construct called Pipeline (or more specifically, ChannelPipeline). A pipeline is a stack of handlers that can manipulate or transform the values that are passed to them. Then, when they're done, they pass the value on to the next handlers. In order to achieve the proper sequence of modifications of the payload, the pipeline keeps strict ordering of the handlers in the pipeline. Another aspect of pipelines is that they are not immutable*, so it is possible to add and remove handlers at runtime. This feature comes in handy while implementing certain types of device communication.
Deployment View
In order to make TQLEngine extensible it provides a capability to load extra bundles after engine core start and initial configuration as those extension bundles may come from different places defined by configuration. It is also necessary to be able to instantiate those bundles when needed i.e. on-demand. The bundlers are automatically configured to be loaded from "sff.auto.launch" folder in the current folder where the core TQLEngine is running.
