AI, Edge Computing Architecture Drive Embedded IoT Development

Key takeaways from this article: 

Cloud systems have entered the realm of AI and machine learning, changing the nature of embedded IoT development, which already required an ample mix of skill sets.

As AI work moves to the edge in many Internet of Things deployments, this trend could accelerate, setting the stage for greater development in platform diversity. 

Still, all roads continue to go through embedded development. Today, programming embedded devices at the edge of vast global systems remains an art form. Power and memory constraints persist, and latency demands are stringent — measured in milliseconds.

[IoT World, North America’s largest IoT event, is going virtual August 11-13 with a three-day virtual experience putting IoT, AI, 5G and edge into action across industry verticals. Register today]

To overcome such hurdles, embedded IoT developers employ simulators, emulators, test beds, software development kits and cloud platforms from both mainline cloud providers or specialists. And their embedded developer ranks still include individuals adept with a soldering iron. 

The C language remains a mainstay on embedded microcontrollers, microprocessors, and systems on chip, modules on chips and board-level systems they power. But in the cloud, where compute and storage are almost limitless, the high-level language of Python has encountered success in machine learning development.

When Worlds Collide

Cloud and embedded development styles diverge today. But that could change, according to Chris Shore, director of product marketing at Arm, the global semiconductor IP leader. Shore has a tenure of more than 30 years in embedded development and was one of the first to port Linux to Arm.

“There are two worlds colliding. It involves changes in expertise and changes to working practices,” Shore said. “If you run an analytical machine learning job using microservices on the cloud, you don’t have to care about how much energy it uses, or how much memory you need. But if you put such analytics on an embedded widget, you do have to worry.”

Communications trips between the edge and the cloud are a developer concern as well. Data analyzed at the point of collection can be acted on more quickly; a system need not wait for data to make a round trip to the cloud and back. So AI processing on IoT device modules has garnered attention, he said.

Several semiconductor firms are moving quickly to link AI and machine learning design to embedded systems. Earlier this year Arm, for instance, launched an Ethos-U55 neural processing unit for machine learning processing on the edge.

IoT Rapidly Evolving

Much of what falls under IoT development is familiar to the ranks of embedded developers; device measurements need to be taken, levels need to be judged — these and similar system events kick off other processes, and so on. 

And connecting these embedded systems to networks is familiar, too. That is why more than a few veteran embedded developers were unsettled by the publicity generated by the Internet of Things. Count among these Jack Gansalle, independent embedded systems engineer, author and editor of The Embedded Muse newsletter.

“Devices have been connected to the network since the day I started. When IoT emerged, we had been already doing it for 20 years,” Gansalle said.

Yet, the field of embedded IoT is evolving rapidly, and few engineers know the nuances needed for globally networked distributed sensor data processing and analytics. Building from scratch is not an option.

As a result, “the engineers buy connectivity in the form of both software and hardware,” he said. Importantly, the embedded developers focused on operations now find themselves working more closely with IT teams. These teams include cloud developers versed in machine learning and other advanced analytics, Gansalle noted. 

Further, he said, real-time operating systems that are bread-and-butter elements of embedded development are adding cloud capabilities.

Cloud platform providers stress the importance of embedded OSes for IoT. Consider, for instance, Amazon Web Services’ increasing activity with Amazon FreeRTOS. For AWS, easing the task of embedded system development is a crucial step to moving its cloud services out onto the Internet of Things.

For its part, Microsoft recently announced Azure RTOS embedded IoT development kits to simplify development. Azure RTOS has grown out of Microsoft’s 2019 purchase of Express Logic. The new kits are supported on development hardware from Microchip Technology, NXP, Qualcomm, Renesas and STMicroelectronics. They form an important interconnect between cloud and embedded computing.

Platforms Show Promise

As AI and machine learning have become part of the embedded IoT discussion, field-programmable gate arrays for the cloud and the edge have entered the mix. 

Embedded developers can configure and reconfigure FPGAs, which are highly flexible to support a variety of machine learning models, including convolutional neural networks. 

The span of development skills to program these chips for embedded systems can be broad, so tooling must be as well. While dedicated embedded system developers need software development kits, data scientists need machine learning development frameworks, according to Chetan Khona, director for vision, healthcare and science services at FPGA-maker Xilinx. 

He said embedded systems that once worked in the field unchanged for 10 years — he cites the photocopier as a classic example — now may be expected to update as regularly as everything else in the digital enterprise. 

That has, in Khona’s estimation, created a strong move to development platforms based on standards to handle the different layers of electronics, control, connectivity, security and AI. The goal of the platforms is to eventually unite the work of developers working at different levels of embedded design.

“We find there is no single person making the key development decisions today. There are different personas involved,” Khona said. “There are hardware developers, FPGA developers, system architects, application developers and data scientists. You need to have an offering for each of those different personalities.”

Khona said Xilinx has worked to bring Python language developers — often key members of the data science team — to FPGA development via PYNQ, an open-source project the company created to allow use of Python language and libraries. 

 Opportunities and Constraints With Embedded IoT Development

The drive toward cloud-oriented embedded IoT development platforms is reshaping industry offerings. Take, for example, electronic component distributor Avnet. 

In 2018 the company purchased Softweb Solutions, a provider of Azure cloud connectivity and data analytics services, following that up in 2019 with the purchase of Witekio, maker of a platform for embedded IoT development. 

The goal was to spur embedded IoT development, according to Yannick Chammings, founder and president of Witekio, which he now runs as an Avnet company. 

Embedded IoT development today is something of a “Wild West,” Chammings said, one in need of greater integration of tools . 

Today, he sees the diverse stakeholders beginning to come together, pursuing designs that are more highly connected than in the past. But it is a balancing act. 

“The embedded community is used to working in a world of constraints — on the other end you have an IoT world that is about new possibilities — new kinds of capabilities you can build if you bring your data to the cloud,” Chammings said.

Managers have to prepare for both opportunities and constraints to succeed in IoT development today, he said. They have to realize they can fail on either side of the equation, he cautioned, recommending early-stage innovation workshops that bring system architects together with other team members to sort through options.

Here Come Containers

Besides AI and machine learning, primary trends influencing the evolution of IoT development today include agile methods and open source software, according to Chuck Byers, chief technology officer for the OpenFog Consortium within the Industrial Internet Consortium. 

Agile methods distill complex programs into manageable chunks of code, he indicated, while open source software speeds design — providing APIs and libraries associated with generally defined protocol stacks.

There are also embedded reference architectures, such as those for Fog and Edge computing that Byers helped forge while at Cisco, and as part of the OpenFog Consortium. 

Byers sees architectural trends on the cloud influencing IoT device development. Chief among these are microservices and container-based technologies, which combine pieces of code with sets of resources that can run in the cloud, at the edge, in smart sensors or what have you.

“Cloud is moving rapidly to container-based workloads. These allow developers to port code from the cloud to security cameras, drones — different nodes on the edge,” he said. These workloads can be spun-up on the fly, he continued, and within as little as 1 second be deployed anywhere on the internet. 

Such technology could well represent a next step forward in embedded IoT development.

Programming Like It’s 1995? 

Over time, people will see a move to platforms that reduce the overall complexity of IoT development, according to Bill Curtis, IoT analyst, Moor Insights and Strategy, and founder of Tread Group, an organization pursuing standards for low-power Internet Protocol-based (IP-based) computing.

“Today, we are still developing software for IoT like it’s 1995. We have tiny memory footprints, very constrained devices, and people are still writing low-level C and assembler code,” he said. “Moreover, you won’t get security without a real platform.”

These platforms, which span development from IoT devices at the edge to cloud data centers will become standard platforms that, Curtis said, allow development managers to “move their cloud programming model into the world of IoT.

“That means the same developers working on the cloud can work on IoT on a daily basis without a change in tool,” he continued. 

Calculating the rate at which such change will occur is the hard part. On that point, industry veteran Curtis adds a philosophical note. Dramatic change to embedded IoT development may not come too soon, but it may not come too late either.

“Folks always tend to overestimate technology changes over two or three years, but they underestimate what happens in 10 years,” he said, paraphrasing Microsoft founder Bill Gates.