Minority Report Meets the Factory Floor

Given its strong foundation in 3D CAD modeling software, it should come as no surprise that PTC is betting its future on ways to help customers create and monetize rich, interactive user experiences that leverage those 3D assets. And its mashing those assets up with IoT, virtual reality, and augmented reality technologies.

“Most people think of the IoT as a way to connect, monitor, optimize, and automate remote products—these are powerful capabilities,” said Jim Heppelmann, President and CEO of PTC. He presented the keynote at Liveworx, PTC’s business IoT event, in Boston.

“But we at PTC think that the notion of digital and physical convergence needs to incorporate the way in which humans experience products, that’s why we’ve spent so much time investing in virtual reality and augmented reality,” he emphasized. “IoT + AR/VR, now that’s a really big idea. “

Virtual reality (VR) is a digital representation of an environment that allows for user interaction, through a simulation of the user’s physical presence. Augmented reality (AR) lets users to visualize a virtual product in the real world by overlaying (augmenting) a real-world environment with CAD models, graphics, sound, data, and other sensory input. Mixed reality, on the other hand, is a convergence of the real and virtual environment, where the two can co-exist.

PTC got a jumpstart in the space through the acquisition of the augmented reality company, Vuforia. The company recently announced a pilot program for Vuforia Studio Enterprise. Integrated with PTC’s Thingworx IoT Platform, it’s designed for users to create AR experiences from 3D data and virtual dashboards for connected products.

Three concepts involving industrial equipment were showcased at the event, demonstrating how IoT plus mixed reality can add value and positively impact the user experience when it comes to asset management, maintenance, sales and marketing, and training.

Caterpillar: Transforming How We Interact with Heavy Equipment 

Caterpillar teams are bringing the physical world and digital world together, with the help of libraries of thousands of CAD models, virtual components, and systems across its entire product line.

Through virtual reality, Cat is optimizing the experience for operators and service technicians and asset management. At startup, the user sees a virtual representation of the machine based on sensor data, which displays voltages, battery utilization, and other operating characteristics. A red light indicates a fault condition.

The experience is further enhanced through augmented reality, whereby the user view on a mobile device a digital representation of the product overlaid on the actual product in the real world, as shown in the image above. For example, should the fault indicate that a battery needs to be changed, a technician can overlay a graphic showing an animated view of how the housing containing the battery is to be disassembled—down to the tiniest screw, if desired.

Cat also sees the potential for sales and marketing. “Physical inventory can be expensive, but with augmented reality a customer can see a life-size virtual representation and when coupled with animations can even open the doors and look inside,” said Teri Lewis. “That’s completely impossible with a brochure or photo.”

A video released last year from Caterpillar demonstrates how several of these functions work:

Flowserve: Rebooting Pump Maintenance

Flowserve leveraged machine learning, augmented reality, and virtual reality to give users a better experience in operating, maintaining, and managing its centrifugal pumps, shown in the image above. One of the biggest challenges was maintenance: Each maintenance issue required an average of three visits to fix the problem.

The pump is instrumented with pressure, temperature, and vibration sensors. Data is passed through an NI DAQ board and converted to digital, where it is processed on an edge device.

When an anomaly is detected, say in the system pressure, one challenge is that the root cause could be one of many things. By running a real time CFD (computational flow dynamics) analysis, technicians can identify the exact problem without even the need to see the machine.

If the problem is a machine misalignment, a common cause of failure, a virtual representation of the machine placed on top of the device can literally expose the misalignment. Taking things one step further, the integration of solid models and animation shows the exact steps for physically disassembling the pump before the technician even pulls out a screwdriver.

JLG: The Augmented Reality Comes to Boom Lifts

Engineers at lift-equipment maker JLG participated in an 8-week pilot program with PTC to add VR and AR capabilities to its 70-ft boom lift. One of the advantages was that the machine is heavily instrumented with sensors, meaning it was relatively straightforward to tap directly into the data to create a virtual reality experience.

Based on the sensor data, programmers developed a virtual representation of the machine, allowing a user to display machine status in real-time on an iPad or another mobile device. Similar to the previous applications, integrating CAD models into the application allows for a visual representation of exact maintenance procedures.

JPL engineers discovered another potential use for augmented reality: By viewing the virtual machine on a handheld device aligned with the real machine (shown in the image above), a user can be trained to operate the boom with a better understanding of its location in the real world. Because the virtual machine is based on the output of a pressure transducer, it could also potentially identify an error should the model not match what the machine is doing in real time.

Feature image courtesy of Wikimedia.