At Microsoft Ignite: How IoT and Robotics Are Driving Industry 4.0

When people hear the term “robotics” they conjure images of robots that are hostile to humans. This is mainly driven by their interest in myriad dystopian science fiction movies, such as the “Terminator” series, that hit the big – and small – screens.

But, of course, this isn’t what robotics is about, especially in industrial and business scenarios, said Rene Schulte, director of global innovation at Valorem Reply, in his Microsoft Ignite session: “The Internet of Robotic Things: How IoT, MR and Robotics are Driving Industry 4.0.”

Robotics has been around for some time when it comes to entertainment, but many are often surprised to learn how advanced robotics has become in business, said Marek Matuszewski, manager Valorem Reply sister company Cluster Reply GmbH & CO. KG, in the Ignite session.

“We’re actually in the middle of a robotic boom market that can be attributed to two factors: the price of hardware is dropping and we are experiencing a new era of mobile robots, autonomous mobile robots [AMRs],” Matuszewski said.

Autonomous Things Prove Value

Although enterprises have already been using automated guided vehicles, they’ve realized that they need dedicated infrastructure, such as cables, physical markings, mostly defined roads, which typically amount to extra upfront costs, he said.

That’s why AMRs are becoming more popular, They move freely, use their own navigation, have collision avoidance systems and ultimately reduce upfront infrastructure costs when compared to robotics of the past, according to Matuszewski.

“A very simple example of an AMR you can find in many households worldwide is what you have in your house – the modern robotic vacuum cleaner with LiDAR [light detection and ranging] sensors,” he said.

Mobile robots are actually Internet of Things (IoT) edge devices on the move, Matuszewski said.

“They can go to places where no humans can go without harming themselves, which is pretty exciting,” he said. “The same concept applies in industrial robotic settings, with the added benefit that these mobile robots can go where humans can’t or maybe shouldn’t go without harming themselves.”

This evolution to mobile robots outfitted with IoT sensors, such as thermal vision cameras, gas sensors, thermometers, etc., and enhanced with artificial intelligence (AI) for object or animal detection are having the largest impact in hazardous or remote industrial locations, Matuszewski said.

There are a lot of use cases for inspection purposes in industries, such as oil and gas and wind turbine monitoring, using drones and AI models, he said.

“Our Reply data scientists work with our customer E.ON, one of the Germany’s biggest energy companies to train machine learning models that will support and improve the monitoring of wind turbines and in the end, decrease their downtimes,” Matuszewski said.

Reply has worked in the IoT space for some time, initially working in the connected vehicle space, said Justin Jackson, president at Valorem Reply, in the Ignite session

“We’ve done some work with CNH Industrial, working on being able to capture sensor data off the vehicles and being able to do fleet management to be able to do scheduled maintenance, things like that,” Jackson said.

Moving to Smart Data Decisions with Robotics

Recently, Valorem Reply has been operating a lot more in the IoT robotics space – not just being able to capture sensor data but do some command and control of those machines as well, he said.

“We’ve got a big robotics customer that works in the automotive space, where they actually are taking data for maintenance schedules, things like that, and actually marrying that up with the IoT data to be able to do things like predictive maintenance off AI models – things along those lines,” Jackson said.

The exciting part is seeing the linear progression from simply being able to take straight vehicle data or more traditional industrial IoT data and start mapping that over into predictive analytics and seeing how that help companies make smarter business decisions off the assembly line, he said.

Valorem Reply is part of Boston Dynamics’ early adopter program for its SPOT robot system  Currently, Reply works with customers that use SPOT in different ways. One such company is ECE, which manages more than 200 shopping centers consisting or about 20,000 shops in Europe, Jackson said. SPOT performs multiple tasks in the shopping centers throughout the week.

“Firstly, SPOT is doing predictive maintenance, monitoring for damage detection on the parking lots,” he said. “Secondly, SPOT also monitors the property for dangerous incidents like water leaks, fire or signs of a break-in. Thirdly, SPOT enables digital twin technology because it basically surveys the shopping center and scans the whole building during the week to provide an updated digital building information model to the facilities managers.”

Previously, this was a very time-consuming and expensive process and as such could only be done annually, Jackson said.

Jackson said that Valorem has been working in the digital twin space for quite some time. He explained that a digital twin consisted of taking a physical object that exists in the physical world and being able to create a 3D replica in the virtual space, e.g., virtual reality or mixed reality.

“We saw very early on that within IoT where we were tracking sensor data and simply displaying it on gauges, [there was a] much more compelling and rich way that we can visualize that information,” Jackson said. “By taking those digital twins, that 3D replica, and taking that sensor data from the physical world and being able to overlay on top of those models, actually presented a really compelling way for someone to be able to see what was going on in that space.”

One of the challenges with digital twins is that it’s often time consuming for customers adopting digital twin technology to create these 3D models, he said.

“When we first started working on this approach, you had to rely on the customer already having a 3D model, maybe a CAD drawing or something like that of the object,” Jackson said.

The problem was that often the initial drawing didn’t quite match up to the finished product, he said.

“So it became an ongoing maintenance task to try and update those models based on what was in effect in the real world,” Jackson said. “Today, the work we can do with photogrammetry, with LiDAR sensors, being able to go in and actually scan a physical object and create a 3D model after the fact off that object is incredibly powerful and enables us to really streamline the process for creating these digital twins.”

Jackson said with the introduction of SPOT and with some of the work being done in the drone space, he’s excited to see how quickly the digital twin concept can be accelerated for businesses.

In the Ignite session, Matuszewski explained how mobile robots, such as SPOT could be used with Microsoft’s HoloLens or other virtual reality or augmented reality devices.

“Those devices are really good examples [of]  how you can really utilize the telepresence and teleoperation to amplify human abilities,” he said. “Having HoloLens on your head, you can basically steer the robot . . . or interact with special labels set by the robot or its operator.”

Valorem Reply has developed a solution that uses a wearable device, such as HoloLens, to enable maintenance workers to fix machines on the shop floor, for example.

“You could go in and in a hands-free environment, be able to approach a unit that might need fixing,” Jackson said. “Imagine maybe like pump control unit with a lot of complex parts, and you can go in and  . . .  be given a step-by-step walkthrough via the wearable device to be able to understand what needs to be done. In addition, companies can also do classroom training on how to do that maintenance on a [virtual reality] device.”