Can AR and VR Change the Energy and Utilities Industries?

By Valerie Silverthorne

The energy and utilities sectors aren’t known to be early adopters of cutting-edge technologies, but that might be about to change.

Augmented reality and virtual reality solutions have the potential to lower costs, improve safety and broaden the types of services provided in these large and often staid industries. The changes won’t happen quickly, and significant challenges — from hardware design to connectivity to software updates – remain. But experts and practitioners say the promise behind the hype is real in this space, particularly for augmented reality.

Using a head mounted device, augmented reality gives an individual in the real world the ability to get extra (augmented) information that can streamline a task or allow it to be performed remotely. Virtual reality also uses an HMD, but completely immerses someone in a virtual world where there is no connection to reality. Today many see AR as a tool that can be used at work while VR is better suited to training. To add to the confusion, some companies, most notably Microsoft, use a different term – “mixed reality” – to describe its HoloLens AR HMD, something it describes as a “more advanced” version of AR. The central difference between AR and MR is that the latter can blend 3D objects into an environment rather than merely overlay images atop the visual field. For instance, a user gazing at a MR object behind a table would see it positioned behind that device. In AR, the object would be superimposed in front of that physical object.

Whatever it’s called, the promise is intriguing. “In terms of enterprise adoption AR is focused on both smart devices and wearable devices that enable a hands-free experience,” explained Piers Harding-Rolls, director and lead AR/VR analyst with IHS Markit in London. “For utilities people they can be instructed or given overlay diagrams so they can literally read from a screen while their hands are free. They’re connected even in a remote environment. They can interface with headquarters and get the information right in front of their eyes rather than looking it up on the laptop or in the back of the van.”

Although Gartner Group’s Hype Cycle for Artificial Intelligence 2018 placed augmented reality in the “adolescent” stage, the report said the long-term benefits of the technology were high, particularly in vertical markets like energy and utilities. But it’s early days, of course – just between 1 percent and 5 percent of the target audience is actively using AR today. When it comes to virtual reality, the adoption rates are similar but Gartner wasn’t quite as bullish, calling out the long-term benefit as moderate.

Shell Oil has been experimenting with AR and VR technology for the last three to four years, said Justin Stroud, portfolio development manager with Shell TechWorks. And he would agree with Gartner that AR is a much more obvious fit for the energy space. “For AR I believe the usability is a little more obvious as we look in oil and gas sectors,” he said. “In energy in general we want to improve the operational efficiency of boots on the ground sites, facilitate and execute as much as we can remotely and automate as much as we can. AR can make us more efficient faster and alternately lead to cost savings. And obviously of great interest to us is the other value point, safety.” 

For Shell and other companies in the energy and utilities sectors, the idea of remote repairs or a technician with easy access to all the information necessary is what is driving the interest in AR. With the right augmented reality solution, “we can have on demand information in a digestible heads up display provided to an operator when a certain event is triggered,” Stroud said. A dead gauge is always problematic, but if it’s located in a remote or hard to reach location it’s even trickier, Stroud said. But AR can not only provide location information to a technician, it can also suggest the best route to reach it, he said, and then provide the necessary schematics and overlays to troubleshoot the repair. “There’s certainly an attraction to something that offers over-the-shoulder coaching,” he said.

The case for investing in VR is less clear-cut, at least at Shell. Stroud, who called VR “a bit of an enigma,” said the company is experimenting with the technology largely for training. “I think we’re seeing some real uptake using VR as a training environment,” he said. For out of the way or deep water environments, VR training can bring recent graduates up to speed before they are sent in to the field, he said. The company also has a Center of Excellence with a small team tracking how VR is used around the world, and then bringing those ideas back for consideration. Shell’s looked at VR for crane instruction and other scenarios “where we could provide level one instruction without actually having to be on site.” But the company’s most unique use of VR so far is internally. “We’re using it as kind of a culture change tool,” Stroud said. Environmental and other factors will force sweeping restructuring in the oil and gas sectors, he said, which will be hard for the large percentage of long-term employees. So the company is having “immersive” staff meetings where employees can experience what new facilities and new roles actually look like. “Seeing is believing and if you put a headset on and move around I think you get a better, more immersive understanding of what we’re driving towards. I think that’s a very powerful tool for change.”

But the changes will happen gradually. Harding-Rolls estimates the AR/VR tipping point is still three to five years away in most industries, including energy. And he and Stroud see a number of hurdles that must be overcome before these technologies move out of the proof-of-concept stage. The infrastructure barriers – access to Wi-Fi, a power source/battery, or security – are significant, Stroud warned. “It’s easy to overlook but if you’re out in West Texas you don’t have access to Wi-Fi,” he said. “You’re lucky to have cellphone service. If you want to truly operationalize this (technology) there are a lot of co-dependencies.”

Form also has to follow function. An HMD built-in to a hard hat can’t weigh 20 pounds because no one is going to walk around all day wearing that, Stroud says. And Gartner’s Hype Cycle report backs that up, citing cases of product offerings not matching actual customer needs. “AR is currently struggling with mismatched expectations (vendors promising solutions beyond current capabilities) and poor implementations (for example, solutions delivered without immersive development knowledge or workflow integration, or not mapped to business value or need),” the report said. Also, energy and utility companies will need robust AR headsets that meet the protective standards for the so-called Class 1 Division 1 risk factors. Stroud said Shell is working closely with a number of manufacturers to refine designs and improve safety. 

And if all of that was not enough, companies also need to think through just how to maintain, manage and update the mass of the information that will flow through the headsets, Harding-Rolls said. “They’re producing content and keeping it up to date and creating 3D content to ensure the manuals being followed contain the right information,” he said. “All of that needs to be kept up-to-date and to have proper overlays and all the latest information. This is not like something you can just switch on. There has to be a lot of thought and investment going in to it.”

Even with a thoughtful approach, it may not be immediately clear what the benefits are, Stroud said. “How do I know going into this I picked the right horse? My answer would be that I don’t think you do,” he said flatly. “Minimize your exposure on investments. Make  small iterative deployments with different vendors and users and eventually set functional requirements. The more you play with it, the less the uncertainty.”