How Edge IoT Is Reshaping Industry

By Mary Shacklett

Fleet tracking, asset tracking, autonomous vehicles, manufacturing automation and warehousing are all areas in which artificial intelligence-embedded chip technologies can offload network data-carrying loads. They can do this while providing frontline, real-time information.

Many of these on-the-go processes need lots of data to be activated. At the same time, they need this data in real time, and in transit, to take place. These kinds of processes don’t benefit as much from cloud computing as other data-intensive processes, such as training data through machine learning. Instead these processes benefit most from edge computing, which brings compute, networking and other resources directly to the devices and data that need them.

By activating artificial intelligence (AI0 processing loads at the level of a system-on-a-chip (SOC), IT can expand its options for distributing and offloading data-processing loads to different layers of enterprise architecture (e.g., cloud, a central data center, or the edge itself). This improves data management and processing. It also conserves bandwidth, and it expedites data and results.

SOC embedded micro-controllers use narrower memory and power consumption than that required by traditional  GPUs (graphical processing units), FPGAs (field-programmable gate arrays) or other types of integrated circuits (ICs).

“We’ll see AI at the edge becoming commonplace in the next five years,” said Steve Conway, Hyperion research senior adviser, HPC Market Dynamics.

“ARM Atom, GPU and other embedded processors are already common in edge devices such as cell phones, sensors, automobiles, diagnostic medical imaging systems, gaming systems and many other devices. These established embedded processors will likely become the mainstream for supporting AI methods as these methods gain ground,” he said.

The Industry Impact of Edge IoT

In 2011 the term “Manufacturing 4.0” first appeared. It originated from the German government’s push to computerize manufacturing, and it introduced a future vision of digitalization, automation and artificial intelligence for factory production. In the scheme,  edge technology could facilitate decisions at the locus of a problem or situation, where AI-embedded SOCs play major roles.

Today, this real-time edge decision making is real. Manufacturing processes are powered by AI-enabled decisions at the edge. In the future, an AI-enabled edge chip could send an actionable alert to purchasing about a shortage of raw materials, or alert sales about the possibility of a product shortfall if a deficient component is found.

Edge AI chip automation is also transforming logistics.

A  truck convoy can cross-communicate with low-latency edge communications deployed to conserve fuel and optimize routes. Going forward,  it will be possible for only one of these trucks to have a human driver, with the remainder of the convoy running on SOC-driven automation.

This could solve a major trucking industry issue:  the shortage of qualified drivers. “This is one of the reasons you see so much technology coming into the trucking industry,” said Shelley Simpson, executive vice president, chief commercial officer, and president of highway services at J.B. Hunt Transport Services,

Perishable goods can also be monitored by intelligent sensors within each truck’s cargo compartment for temperature and humidity.

A truck carrying produce to Atlanta, for example, was rerouted to the more proximate Washington, D.C., market. The reroute was ordered after a sensor inside of the truck’s cargo compartment alerted the driver and the logistics company to the danger of produce spoilage from overheating. The company’s ability to proact in real time to the information averted spoilage and saved money. In the food industry, it’s major.  The Food and Agriculture Group of the United Nations estimates that $1 trillion of food is lost or wasted each year.

AI-enabled chip technology is also changing how airborne and ground-based vehicles perform.

Logistical challenges confront military personnel when they observing and/or enter a dangerous area. In the past, a risky surveillance job  might have required humans to inspect an area firsthand, subjecting personnel to danger and loss of life.

Now with edge AI processing, a fleet of unmanned drones performs reconnaissance and inter-communicates in real time. If a drone in a squadron is downed, the fleet detects the issue and adjusts its formation to continue the mission. “Demanding workloads that require the processing of multiple sensory inputs including video and audio may start to push the envelope unless supported by specialized chips,” said Saurabh Mishra, Senior Manager for Product Management in SAS’ IoT and Edge Division. “Autonomous drones, robotic arms, and industrial automation are all good examples of how these chips may be used.”

Geopolitics and innovation

Nevertheless, companies worry because of geopolitical forces at work in the chip and semiconductor industries.

In 2019, Huawei was placed on the U.S. restricted list. NVIDIA then acquired Arm, Ltd. in a $40 billion deal that had Google, Microsoft, Qualcomm, Apple, Intel, Samsung, Huawei and Amazon concerned about a critical supplier.

In 2019, Intel acquired AI-chip startup Habana Labs for $2 billion, and AMD acquired Xiliinx for $35 billion.

“The trend over the past 50 years has been to keep unrelated national security concerns siloed from the economic analysis driving antitrust decisions. However, where potential anticompetitive conduct is also detrimental to national security, we should not be surprised if the USG takes a more aggressive approach to enforcement,” wrote Cullen O’Keefe, research affiliate at the Centre for the Governance of AI, University of Oxford.

IT must consider these lawsuits and antitrust actions when it justifies and attempts to “future proof” AI investments.

“Today, AI is widely seen as key for future economic leadership, and there are strong initiatives in China, Japan and Europe to cast off reliance on the U.S. and develop indigenous processors,” Conway said. “IT departments can’t do much to affect these geopolitical battles, but they can plan to ensure that the supplies of processors they need are secure, especially by negotiating long-term supplier contracts with penalty clauses and maintaining adequate inventory levels.”

IT’s to-Do List

The move to smaller form factor IoT will force an IT focus in three key areas:

IT architecture. IT architecture must be realigned to fit the business use cases that companies want to solve with chip-level AI. Minimally, this architectural revision is likely to yield three tiers of IT technology, processing and data architecture: the data center, the cloud and the edge.

“The starting point, of course, is to map out and optimize the end-to-end process and use that information to assign appropriate resources at each point along the way,” Conway said, who referenced the work of PayPal.

“Half a dozen years ago, PayPal had a serious problem with fraud in credit card transactions,” said Conway. “It was taking up to two weeks to identify fraud, and by that time the fraud had often hit customers’ cards.  The company installed a high-performance computer that could spot and prevent fraud as it happened, within 150 milliseconds, saving PayPal more than $700 million in the first year or so.

The application at PayPal and other firms relies on embedded processors in the card readers, along with the Internet for the round-trip authorization process, and server systems with non-embedded processors for the heavy lifting, on-premises or in clouds.”

IT skills. Only 47% of survey respondents in a 2019 Microsoft IoT Signals Report believed the market had individuals with the necessary IoT job skills https://news.microsoft.com/2019/07/30/microsoft-announces-iot-signals-research-report-on-state-of-iot-adoption/.

“The availability of skilled resources to manage the deployment of AI models on chips will remain a challenge,” said Saurabh Mishra, Senior Manager for Product Management in SAS’ IoT and Edge Division. “Companies should also recognize that

edge AI chips are not silver bullets. They work in the context of a larger system. It is critical to think of the complete pipeline when deploying AI-embedded chips since a weak link upstream or downstream can negate their targeted boost.”

Commercial IoT software and hardware stacks can help address pipeline integration challenges—but processing must still be defined at each tier by IT. This includes model-building and programming SOCs.

Investment management. Consolidations, antitrust and intellectual property lawsuits will continue to play out in the AI/chip space, as it has in other areas of IT.

The good news is that corporate IT departments are no strangers to this.

Selecting  a broadly accepted IOT stack solution with a large user base is one form of future-proofing, as well as ensuring that the IoT  you use conforms to common security standards and APIs. A second strategy is negotiating with IoT vendors for liability and investment protection that you define in your contracts.

Finally, AI-enabled chips must  deliver business results.

“The impact of edge IoT on IT architecture will come down to the use cases that IT is asked to implement, where AI offers the ability to pre-process information in real time and only transfer relevant and useful data,” said Murali Gopalakrishna, head of product management for autonomous machines and general manager for Robotics at NVIDIA.

“An automated AI inspection process in a factory will use real-time information to make split-second decisions at the edge while transferring relevant data to back-end systems for post-processing, analytics and new model development out of band to the IoT edge based decisions.”

Applications can detect occupants wearing masks or count the number of people entering and exiting a space by creating heat maps to ensure occupancy limits are not exceeded. And with additoinal sensors, cameras and automation happening in IoT and at the edge, AI will become more relevant for IT managers and infrastructure architecture, Gopalakrishna said.