Smart Speakers, Automation and the Future of Smart Buildings

In the not-too-distant future, a hotel guest might control their TV by talking to an in-room smart speaker, or schedule a reservation for two in a cruise-ship restaurant with a voice command. Perhaps they’ll use voice command to check out of the hotel room. Or maybe a member of the housecleaning staff will use a voice command to report an emergency. Farther into the future, the sci-fi-esque possibility of conversational interfaces becoming a smart building avatar could become a reality. Perhaps guests will receive a personalized greeting as they walk into a building, and once there, will hear a disembodied voice asking for their help to save energy during their stay there. Or maybe the building automation system will simply decide to, say, raise or lower the blinds on its own based on the time of day and user preference.

In any event, the potential applications of conversational interfaces in hospitality are seemingly limitless. But in the long run, the technology, in conjunction with the steady rise in IoT capabilities and advances in building automation, could lead to significant changes in user experience in the hospitality sector. To succeed, the makers of such interfaces will need to win over the user’s trust, or merely convince users to trade privacy for convenience.

In the relatively short term, voice interfaces could prove to be an important smart building technology in the hospitality sector while also accommodating restaurant reservation requests and other customer service queries, said Sandeep Kalra, senior vice president and general manager at HARMAN Connected Services, which recently announced that its own bespoke smart speaker technology would be featured in the MSC Bellissima cruise ship. Refined in tests involving 400 people, the smart speaker, known as “ZOE” can provide answers to some 800 questions in seven languages. ZOE will eventually be retrofitted into existing MSC cruise ships and will come standard in new ones such as the MSC Grandiosa and MSC Virtuosa cruise ships, scheduled to debut in 2020.

[Attend Internet of Things World to see how buildings and homes are being transformed by IoT. Book your conference pass and save $350, get a free expo pass or see the smart buildings and smart home speakers at the event.]

Prominent hotel chains such as Marriott and Best Western Hotels & Resorts are also experimenting with Amazon’s Alexa conversational interface in a select number of hotels. And during CES this year, Google demoed its smart speaker’s language translation capabilities in Las Vegas hotels. Hyatt Hotels is working with Google to pilot the technology at its Hyatt Regency San Francisco Airport location.

Farther down the road, conversational interfaces could be used to help create a sort of relationship between a smart building and its inhabitants. A building with an IoT-enabled HVAC and lighting system, for instance, could use such voice-based UI to help building owners meet sustainability goals. Researchers at USC deployed a smart building avatar known as “Ellie,” which led to energy consumption savings. The avatar would ask questions such as: “If I open the blinds for you to have natural light, would you please dim or turn off the artificial lights?” Most users complied. But rather than serve as a sort of personification of the building, the researchers found that the persona functioned better when it served as an extension of the building manager.

Last year, Gartner’s Hype Cycle for Artificial Intelligence pegged conversational interfaces as nearing the so-called peak of inflated expectations, estimating that the technology was five to 10 years away from mainstream adoption. The same research found that 4 percent of enterprise companies had a conversational interface in production while 38 percent were either in the experimentation or planning stages. 

One wild card is that a segment of the population doesn’t trust such interfaces — 41 percent of the population, according to an October 2018 Business Insider Intelligence survey. That research followed a May 2018 news story in which an Amazon Echo device erroneously recorded a Portland, Oregon, couple’s conversation and then shared it with one of the husband’s employees in Seattle. Amazon released a statement saying the bizarre chain of events followed Alexa mistaking a conversation for a string of commands and added that it was working to make such snafus less likely to occur. In December, an Amazon Alexa user received some 1,700 audio recordings as a result of a similar error. “This was an unfortunate case of human error and an isolated incident,” Amazon explained in a statement to The Washington Post.

On the domestic front, however, enthusiasm for smart speaker technology has been voracious. At present there are an estimated 65 million smart speaker owners, making the segment easily one of the most quickly growing consumer technologies since the introduction of the smartphone.

In hotel rooms, however, smart speakers are still uncommon. And Alexa can be an unwelcome roommate, as a Travel Weekly post quipped. The majority of Best Western hotel guests involved in an Amazon Alexa pilot chose not to use the virtual assistant. Many guests involved in the trial disconnected the speaker, “presumably because they didn’t want Alexa listening to them in the room,” said CEO David Kong according to Travel Weekly. But Tom Long, general manager of the Westin Buffalo, apparently has had a better experience with the technology, which has enabled the hotel to keep track of common requests while also integrating with an autonomous robot butler known as “Chip,” which delivers amenities and room service to guests. “Concerns about privacy have been minimal, [but] in the event of a concern, we recommend the guest turn the device off or unplug it,” he said.

In the case of ZOE, Kalra said that HARMAN is not collecting any personal data from the ZOE speakers. “We are anonymizing the data,” he explained. And opposed to Amazon’s Alexa or Google’s conversational UIs, ZOE is a proprietary conversational interface.

Ultimately, conversational interfaces will gain in popularity — especially for simple queries. “In my home, for example, I use smart speakers for basic things such as setting alarms,” Kalra said. And HARMAN is working on using voice as a multipurpose user interface and data-entry tool.

The fact that ZOE is bespoke enables it to serve as a context-sensitive interface for its surrounding environment. It integrates with onboard restaurants as well as the Samsung TVs in guest rooms. The functionality is much more robust than “text-based chatbots that some hotels are using,” Kalra said. “This is going to be the way of the future.”

“The possibilities with voice assistants are limitless. A number of businesses and consumers rely on voice AI not just to perform workflows of tasks, but also to predict or prescribe the best next steps in that workflow,” Kalra said. HARMAN Connected Services is working in partnership with organizations to integrate AI services with other enterprise platforms to improve the efficiency of interactions between users, devices, and facilities and buildings. “As we look to the future, we expect AI assistants will have an immense organizational impact on sustainability efforts, such as energy efficiency and waste reduction, to meet their long-term goals.”

Building such a future will require considerable coordination as well as keeping a close watch on cybersecurity threats, which can be difficult to detect in complex connected environments. According to cyber defense expert Peter Tran, the more sophisticated smart IoT ecosystems get, the more difficult it will become to monitor for and detect malicious activities and indicators of compromise. Tran says the use of artificial intelligence and machine learning will be critical to keep up with the potential vulnerabilities hidden within “smart learning” environments at the scale and speed necessary given its sheer growth.

In the longer run, as organizations deploying voice interfaces in smart building contexts come up with strategies for allaying privacy concerns, the next usability question could be the degree of automation for users in the environment. A USC study found adaptive building automation to be more popular than either full automation or no automation. In their research, their learning algorithm could predict participants’ preference with an accuracy level of higher than 85 percent in 8.5 days. But for such adaptive automation to work effectively, it must observe user activity, which could spur privacy concerns. For the companies developing and deploying such systems, the way forward is to follow a strategy proposed for space missions that sounds like a strategy commonly used when assigning work to a new employee: give the system small projects, let it prove it can accomplish the objective effectively and expand the scope of those projects over time. While the slogan that it is better to ask for forgiveness than permission may be a common trope in business contexts, when it comes to futuristic buildings that can think, speak and act for themselves, it’s the other way around.