12 Essential IoT Skills for a Software Engineer

— by Jane Ren and Alok Batra

This is you – a science or engineering major back in college, fell in love with software. Have developed cool stuff from consumer-oriented to enterprise-based; have led teams, architected systems, and changed jobs to stay away from ignorant boss (es). But nowadays the topic of IoT keeps you thinking.

More importantly, you decided it’s now the turn for engineers to lead IoT. The business folks have had their time (almost 2 years bathing in its hype) and they have not figured out a real thing.

The real deal is that when things become a major actor in business applications, the boundary between software and the real world blurs. Software will now be redefining the whole operating principles of our businesses, while facing new programming challenges and security threats.

It is therefore time now to think how to make this new operating paradigm happen, rather than keep debating what value it might generate in $ amounts.

The following 12 concrete skills distinguish an IoT software engineer apart from an average engineer working on conventional business applications (Internet of People, or IoP applications).

These skills are “new” at 3 different levels

Level 1. Augmenting: skills you already have but need to be enhanced for IoT

Level 2. New: new skills you need to acquire for IoT

Level 3. Rewiring: skills that need to be “unlearned” in order for new IoT thinking to be adopted (this is where your existing skill might actually be a baggage inhibiting the right IoT design).

1. New Skill – IoT Modeling: the design and implementation of software models/entities for IoT applications.

In IoP you are familiar with the design and implementation of objects (or their equivalent) to represent entities. These objects are the useful building blocks for application logic (such as inventory), but they are insulated from the external (real) world.

In IoT you need a systematic way to model things. These real things can be directly connected to the application. So you need to build software constructs to:

(1) Represent things in the real world, including both sensors, devices and non-electronic things (a street, a building);

(2) Encapsulate the interactions between software and things;

(3) Represent and manage the relationships and interactions between these real world things. For example, the moisture of the soil will be increasing if the irrigation nozzle is turned on.

How do I learn?/which language?/which tools?

Although any object-oriented languages, such as Java and C++, can get you started on such modeling, they don’t have the framework to easily encapsulate external thing interactions into the entities. Instead, most developers end up “outsourcing” such interactions from Java to Python, and then stitch the two together. 

In TQL, Thing Definition Language (TDL) allows you to define ThingModels with both attributes and actions, where attributes represent the state, and actions encapsulate the software-to-thing interactions. 

2. Rewired Skill – System Architecture: the distribution of application and application logic across an IoT system.

In IoP, software engineers are mostly concerned with the logical/functional architecture. System architecture can be a separate decision made nearing deployment time, driven by database, network and performance considerations. Even if an enterprise application may run over distributed databases and have distributed clients, its application logic is always centralized.

In IoT, the physical constraints (such as location and physical wires/pins) are no longer external to the application logic. A software engineer developing without understanding of the system architecture will produce an application that is impossible to run in the real world. IoT systems often consist of multiple edges, gateways and clouds. So you need to know:

(1) What are the physical constraints/boundaries of my application’s runtime?

(2) How should I distribute the logic of my application across the system based on its relationship with devices and networks?

(3) How to develop those different parts of the application logic and keep them consistent and in sync in runtime?

This is not a small challenge. For decades the industry has not been able to write distributed applications well. Your comfort with centralized application logic needs to be tossed away before the new thinking can be learned.

How do I learn?/which language?/which tools?

It is certainly possible to write distributed applications in existing popular programming languages such as Java. However you will carry all the burden of developing the logic for communication and synchronization between each component of your application. That is a hefty task.

The new IoT programming language TQL is designed for distributed programming. Program components are intrinsically independent of each other, only connected through data flows over pipelines. TQL pushes a developer to think in distributed ways.

3. Enhanced Skill – Message Transformation: the inline processing of messages going between the IoT application and things.

You deal with message transformations in IoP applications, often where 3rd party application integration is involved. The transformation is usually a concentrated and isolated activity (isolated from program logic), for example, in offline ETL processes.

In IoT, applications are running in tandem with multiple things hooked into it. Transformation of messages from and to the things is inline, high frequency, and usually more complex. It becomes a part of your programming logic. You will need to know how to do:

(1) Parse

(2) Transform, and

(3) Format

of messages from and to things.

How do I learn?/which language?/which tools?

JavaScript is a good language for message transformation. It has processors for C++ and Java. Xpath – a template driven language – is also a good candidate and its processor is available in C, C++, Java, and JavaScript.

4. Rewired Skill – State Management: maintaining states that represent external things.

The word “state” means different things in IoT versus IoP applications. In IoP, entities/objects within the software have states. The application has the responsibility of maintaining the internal state. It is however, not concerned with whether this internal state accurately reflects any “external” state in the real world (e.g. has this employee actually been on boarded in reality?) That responsibility, if it exists at all, lies with people who use the application.

In IoT, much is about using software to represent external states of the real world things (Is the light on? Is the door open?). You can call this a “reflective state”. The “reflective state” however, can never be 100% identical with the state in the real world. Even if you refresh every second, your states are out of sync with reality in between the 1-second wait time. To do state management in IoT, you need to be able to:

(1) Translate software interactions with things into reflective states;

(2) Create the logic on what to do when inconsistency is encountered between the reflective state and the real state in the real world. 

This may require counter-intuitive approaches. For all the years since you knew software, applications have insisted on expecting consistent states and trying to keep them consistent, often using the database as the single source of truth.

In IoT, if you always assume consistency between internal and external states, your application will run into surprises and fail. If your application cannot tolerate the multiple versions of “truth” and forces them to converge, you will lose valuable information. 

How do I learn?/which language?/which tools?

Conventional programming languages do not explicitly address this issue. As a developer this is a logic you will need to include in your program.

In TQL’s ThingModel, the language concept is built in to allow different versions of reality to be kept (e.g. you asked the light to turn off, but it is still on). It provides an easy conceptual structure to think about states in the IoT specific way.

5. Rewired Skill – Parallel and Asynchronous Programming: program in parallel logic with asynchronous request handling.

Most conventional IoP application has sequential logic and synchronous request handling. As a developer, your program encodes the sequence of task execution at runtime. 

IoT applications however interact with things, often many at a time. Think about many machines on a factory floor. The sequence of their interactions with your application is usually not up to you to decide. Some of these machines may be asynchronous themselves, and may interact at high frequencies with your application. You need to know how to:

(1) Create program logic that are parallel to each other and yet fulfill the business requirements;

(2) Handle requests asynchronously to avoid blocking of resources between each parallel interaction.

How do I learn?/which language?/which tools?

Java does provide multi-threading for you to create parallel programs. However, synchronization becomes a significant programming overhead when the number of threads increases. 

TQL provides a parallel programming paradigm from its foundation. The execution of different TQL program snippets is driven by data flows at runtime. This makes it counter-intuitive to code sometimes, because you no longer fully control the sequence of how your program runs. However, IoT applications written in this way become vastly flexible and dynamic to deal with the unpredictable world of things in each specific context.

6. New Skill – Failure Management: create application logic that responds to failure conditions in things. 

In conventional applications, “failures” are primarily infrastructure issues – database down, network down, peer application not accessible, etc. Failure management is handled separately, often by the IT department, outside the application logic and outside the responsibility of the developer.

In IoT, many of the “failures” you expect to see are from things. It can be devices not accessible, machines down, or things sending malfunction signals. Because such failures are extremely variable in nature and mostly machine and domain-specific, it cannot be handled as a domain-independent issue across the board. If you ping a machine and do not hear back in 5 seconds, it may be a perfectly OK situation in agriculture but disastrous in medical domains. How to manage such failures becomes business logic that a developer has to implement. You will need to take care of:

(1) Things not responding (not sending updates as expected, or not responding to commands);

(2) Malfunction signals from things;

(3) Resilience and recovery from offline/not available states, such as holding pending messages until things come back online.

How do I learn?/which language?/which tools?

Rules and workflows are tools that can help you implement such device failure management logic. The challenge will be to integrate them with your object-oriented application. This is why in TQL workflows are designed to be naturally compatible with the modeling paradigm.

7. New Skill – Time Series Data: manage time series data coming from things.

When people interact with applications, it is more important what they are doing now, and what they did in one episode of interaction (fill out a whole form).

When things interact with applications, we sometimes care about the time sequence in which many interactions happen, resulting in the same variable carrying different values against time. In IoT you need to be able to:

(1) Store

(2) Retrieve, and

(3) Process

time series data.

How do I learn?/which language?/which tools?

 NoSQL databases can be used to manage time series data. More specialized technologies include Historian and OpenTSDB, for example.

8. New Skill – Simulation: creating simulated things and systems to test IoT applications.

Conventional applications run against data and interact with people. Because data can be easily created and test users can be recruited, what you run and test in your development and staging environment can predictably tell how your application will behave in production environment. Not so for IoT.

In IoT, a few things make meaningful testing hard:

(1) The number – it is very hard to replicate the situation of hundreds or thousands of sensors or devices in your lab. But your application has to deal with them in production time;

(2) It is the interactions that matter, not just data. It is interactions with things, not just interactions with people (which are easier to create).

(3) The system and physical dynamics makes a difference. In production environment, turning on the heater will affect the temperature, which in turn creates an effect on the machine behavior – this (physical) logic you do not implement in your application.

In order to test IoT applications for them to be ready to deploy at scale, you need to:

(1) Simulate the behaviors of sensors/devices;

(2) Simulate a scaled system with multiple sensors/devices and rich set of data;

(3) Simulate the interactions between different components of a system;

(4) Simulate the changes of the external/physical environment.

How do I learn?/which language?/which tools?

There is a lack of simulation platforms specifically designed for IoT applications. The TQL platform is the only one we are aware of that lets you easily simulate devices, data, and more importantly, systems. To qualify for a good simulation platform, a technology needs to be event-driven (because things communicate in events), parallel in nature (to simulate many things at a time), and have the language tools to allow easy creation of simulated models.

9. New Skill – Serial Protocol: integrate with devices that communicate on serial protocols.

Most of the protocol knowledge you have from IoP applications is on HTTP, XMPP, or SMTP.

There are many different protocols used in IoT, especially in the industrial/enterprise side of IoT. If one has to pick one most important, it has to be the serial protocol. There are different variations of serial protocols, but the same interaction patterns (such as handshakes) are followed. Many industrial machines use serial protocols. In addition, most of the low-level sensor protocols today (e.g. GPIO, I2C, SPI, ADC, DAC) are converted to serial protocols through the micro-controller units (MCUs). Unless you desire to code at the micro-controller level (which we don’t think most software engineers should do), you should be good most of the time with serial protocol knowledge. 

How do I learn?/which language?/which tools?

There are many in-depth books on serial protocol, including this one.

However, as a software engineer a high level knowledge on serial is sufficient. More importantly, you need to find the right tools to work with serial devices.

In terms of use, Python has good support for serial protocol (Python Serial Package). If you are coding in Java you can use RxTx (which is written in C). But RxTx has certain license restriction coming with GPL, making it harder to embed it into your software. RxTx is not actively maintained currently and hence there is some risk in using it.

In TQL, a generic serial handler comes out of the box. This means whatever device you need to integrate with a serial protocol, all you need to do is to call the protocol handler and provide the parameters. More information can be found here.

10. Enhanced Skill – Security/Access Control: build secure IoT applications.

Security is not a new skill to you. You need to provide access control to any IoP applications. This primarily involves managing which user has access to which system, or which function.

In IoT, since it is things, not just users who are accessing the application, access control becomes much more multi-faceted and complex. To name a few areas:

(1) You need to control not only which user can access which function, but also the specific thing instances on which the function can be executed. For example, all users can access the function to turn the streetlights on and off. But the store manager can only control lights next to his storefront, while the city manager can access this function across all lights in the city.

(2) You need to authenticate and control not just how application can access which devices, but also which devices can access the application. In IoP, this control is single directional (only from people to application, not from application to people). With things, it has to be bi-directional.

How do I learn?/which language?/which tools?

Writing good secure IoT applications means security can no longer be left as an afterthought. In the IoP world, you complete every other function and then add role-based access control on top of your application. In IoT, you need a framework that allows access control between each and every component within the system, as a part of the programming logic itself.

11. New Skill – Micro-controller Units (MCUs): understand and integrate with MCUs in IoT applications.

You cannot do IoT without touching MCUs. Many industrial machines come with controllers built in, so you don’t need MCUs for machines. However, most sensors require an MCU to handle analog to digital conversion, power management and communications.

The programs running on MCUs are mostly written in C or C++, which are good tools for writing close-to-device logic and light-weight enough to run on MCUs.

Given the proliferation of MCU provider’s today, a software engineer does not need to master the C/C++ programming skills on MCUs. What you need is to:

(1) Be familiar with the leading MCU providers (e.g. Arduino, Arm, Atmel, Freescale);

(2) Know how to configure MCUs using vendor provided tools/IDEs;

(3) Know how to communicate with MCUs to receive and send data (primarily serial protocol based).

How do I learn?/which language?/which tools?

A good start is Arduino MCU and Arduino IDE. Arduino.io provides good support through its community. 

12. New Skill – Physics Requirements: taking into account the dynamics of the physical entities and environment in IoT applications.

We all talk about “business requirements” as something software engineers need to understand and keep in mind when they do development. Without the understanding of business requirements, applications work technically but not practically.

In IoT there is a new set of requirements – the “laws” of physics. If there is a security camera within 3 feet of the entrance, but there is a wall in between, you need to know that the right data will not come in. If you send command to open the valve in a pipe, you should know the flow rate of the liquid may be different at different temperature levels. These are things you cannot control, but have to take into account. On the other hand, they may give you endless fun (or frustration) in your IoT development.

How do I learn?/which language?/which tools?

There is no set of books for the wide variety of IoT domains. The best resources for you are the domain experts. They can be hardware engineer in the field, the farmers on the farm, or the operators on the factory floor. Take opportunities to have site visits. IoT application is not for engineers who only sit behind their desks doing virtual thinking. You may find physics requirements more fun than people-oriented business requirements ;)

We are at the end of these 12 skills. It seems a lot. And that is the expectation put on an IoT software engineer. Out of these 12, #1-#8 are needed at "proficient" level, whereas #9-#12 are required at a more general "knowledgeable" level. 

You can be assured that very few people out there have the full range of such skills and knowledge. Getting yourself hands on with some real IoT projects, combined with this guideline is the best way to stand out on the competency level as fast as possible.

Since we cited TQL here frequently – TQL is an IoT application platform provided by Atomiton, which specifically addresses these 12 areas with solid principles, frameworks and tools. Atomiton also offers a TQL IoT course, aiming at bringing software engineers up to speed quickly on this set of skills using TQL. You can find out more from here.

The 12 essential IoT skills for a software engineer. Summarized below:

1. New Skill – IoT Modeling: the design and implementation of software models/entities for IoT applications. (- Proficient)

2. Rewired Skill – System Architecture: the distribution of application and application logic across an IoT system. (- Proficient)

3. Enhanced Skill – Message Transformation: the inline processing of messages going between the IoT application and things. (- Proficient)

4. Rewired Skill – State Management: maintaining states that represent external things. (- Proficient)

5. Rewired Skill – Parallel and Asynchronous Programming: program in parallel logic with asynchronous request handling. (- Proficient)

 6. New Skill – Failure Management: create application logic that responds to failure conditions in things. (- Proficient)

7. New Skill – Time Series Data: manage time series data coming from things. (- Proficient)

8. New Skill – Simulation: creating simulated things and systems to test IoT applications. (- Proficient)

9. New Skill – Serial Protocol: integrate with devices that communicate on serial protocols. (- Knowledgeable)

10. Enhanced Skill – Security/Access Control: build secure IoT applications. (- Knowledgeable)

11. New Skill – Micro-controller Units (MCUs): understand and integrate with MCUs in IoT applications. (- Knowledgeable)

12. New Skill – Physics Requirements: taking into account the dynamics of the physical entities and environment in IoT applications. (- Knowledgeable)