Smart Home Companies Are Walking a Privacy Tightrope

Is privacy over? 

That may not exactly be a new question, but the proficiency with which tech giants can profile you continues to increase steadily. 

The proliferation of smart speakers and other IoT gadgets along with our tendency to carry smartphones with us nearly everywhere we go gives those firms an ever-more-detailed window into your life. And if an individual tech firm has a fragmented view, they can fill in the gaps by partnering with other companies. For instance, Facebook has data-sharing arrangements with dozens of companies, according to an article from The New York Times.

The potential privacy ramifications are vast, given the amount of information even a single company can gather.

For instance, Google can intuit where you shop and what kind of food you like not just by tracking your web browsing, but by tracking your location via your smartphone. If you have a Google smart speaker, the company also knows what your voice sounds like. And if you have a smart speaker and have lax privacy settings, the company can use text from voice interactions to personalize the ads they serve you.   

If you are a regular Amazon customer, the company likely has a clear idea of the types of products you are interested in. Given their dominance in the smart home, they have a clearer potential not just to serve you an ad, but to convince you to buy a product. “Forget about serving up an ad click for, say, $1 or $2. They could try to convince you to buy, for instance, a blender for $89.95. That’s real money,” said Chris Kocher, co-founder of Grey Heron. 

On a related note, an Amazon team auditing consumers’ commands to the Alexa assistant can access users’ location data, Bloomberg revealed in April. The company responded to Bloomberg in a statement that employees’ “access to internal tools is highly controlled,” adding “we have a zero tolerance policy for abuse of our systems.”

A similar report from The Guardian related to Siri reported that Apple contractors hear confidential details. A whistleblower told the paper: “There have been countless instances of recordings featuring private discussions between doctors and patients, business deals, seemingly criminal dealings, sexual encounters and so on. These recordings are accompanied by user data showing location, contact details and app data.”

Apple responded at the time: “A small portion of Siri requests are analyzed to improve Siri and dictation. User requests are not associated with the user’s Apple ID.”

Apple recently released a statement vowing to overhaul its data collection practices from Siri. 

[Smart Home Summit is the conference that focuses on interoperability, cybersecurity and the future of the smart home. Save $200 on your conference pass with VIP code “IOTWORLDTODAY”.]

Owners of a Ring doorbell could be granting access of that device’s video feed to developers in Ukraine as well as local law enforcement, thanks to an expanding program that, among other things, gives more than 400 law-enforcement agencies the ability to request video footage in the case of an active investigation. 

“Many of the services and the mobile phone apps we use every day are provided to us at no charge,” observed Asaf Ashkenazi, chief strategy officer at Verimatrix. Such applications, ranging from Google Maps to the Weather Channel app, are often free to users because of the valuable user information they provide. 

The providers of such applications have, in some cases, received complaints that they manipulate users into sharing information. For instance, the city of Los Angeles alleged, in a lawsuit filed earlier last year, that, the Weather Channel “deceptively collected, shared and profited from the location information of millions of American consumers.” 

There have been several recent examples of tech companies paying fines or settlements in the range of hundreds of millions — or even billions — of dollars, as The Guardian observed. 

That’s not to say that all free apps are equal in terms of data collection. But Ashkenazi said the old American axiom about there being no free lunch applies to apps that are available for free on smartphones or other devices. “If you are not paying for it, you are probably the product,” he said. “The owners of these apps and services make a profit,” and such companies achieve some ROI from user data. 

Apple, which prides itself in its privacy protections, helps disseminate apps that capture user data. “Companies such as Google and Facebook get access to iPhone users by offering their apps — Messenger, Gmail, Google Maps, and so on — for download from the Apple App Store,” wrote The Atlantic. “Most cost consumers nothing because they exist to trade software services, such as email or mapping, for data. That business model helped stimulate the data-privacy dystopia we now occupy.” 

Ultimately, it is difficult to ascertain precisely how tech companies are using and sharing our data. Such firms could deduce behaviors from seemingly banal data streams, such as a voice command to a smart speaker.

“With AI, vendors can intuit a lot of things. More data leads to higher-quality insights and higher confidence levels in predicting intention and behavior. That, in turn, can directly affect advertising clickthrough rates as well as e-commerce conversion ratios and sales,” Kocher said. Machine learning algorithms can sometimes detect patterns about us that we may not even be aware of ourselves.

They can also make simple deductions. If a user is setting a timer at 5 p.m. while at home, there’s a good chance it is related to food. If the same user asks the smart speaker about cooking corn immediately before setting the timer, his or her intent becomes even more transparent. Similarly, if you are in a car and set a timer to a voice assistant — either integrated into the vehicle or via a smartphone, — the tech firm in question could determine you are likely parking at a metered spot, especially if it can leverage geo-coordinates derived from your smartphone at the same time. 

From there, a tech company armed with that knowledge could serve an ad — or perhaps offer a discount relevant for a nearby merchant. 

[IoT Security Summit is the conference where you learn to secure the full IoT stack, from cloud to the edge. Visit the website for more information.]

The increasing number of data points tech firms have at their disposal enables fairly precise targeting. Companies that sell meal kits could be interested in targeted individuals who Google or Amazon conclude frequently cook. Similarly, an owner of, say, a vegetarian restaurant in San Francisco could purchase advertising for individuals who are known vegetarians within a two-block radius. 

Companies that can connect a number of behavioral and contextual puzzle pieces in online activity can cobble together “some pretty amazing profiles,” Kocher said. “That allows them to sell more targeted advertising at higher rates.”

There is also the possibility that smart speakers from the same companies serving consumers will pop up with increasing regularity in commercial settings. Already, for instance, a handful of hotels have experimented with using smart speakers inside of hotel rooms as well as in hotel lobbies to facilitate check-in and check-out. 

If smart speakers are used in a hotel line, they may hear sensitive personal information. “I’m more concerned that someone might shout details such as a credit card number or passport information,” said Candid Wueest, principal threat researcher at Symantec. Anyone with access to the relevant recordings — or even an individual who hides a connected microphone nearby — could pick up a significant amount of information. “And for a hotel [testing smart speakers to help with check-in and check-out], if you have a few hundred people per day coming in, that could already be enough for an attacker to be profitable,” Wueest said. 

Another consideration is Alexa for Business, which Amazon is promoting for use in conference rooms and other work environments. An Alexa-enabled smart speaker near a stockroom could help staff reorder office supplies, for instance, while one in a conference room could help participants make calls, record meeting notes, transcribe action items, keep track of calendar-based items and coordinate meeting summaries. While Alexa for Business remains in an early phase of development, there are considerable privacy and security ramifications involved in such applications.

The fact that smart speakers have expanding capabilities in the home is another concern. Amazon, for instance, has developed Alexa Guard technology, that enables its smart speakers to detect the sound of a broken window — distinguishing it from the sound of other objects, like a shattered drinking glass. The smart speaker can also detect the sound of smoke alarms. The more variables a smart speaker listens for, the more actively it must tune into the environment. “It’s no longer just waiting for your keywords. It has to analyze the whole time,” Wueest said. 

If consumers come to rely on smart speakers to double as security devices, it increases the pressure for the manufacturers of those devices to have a backup plan in the case of a cloud outage. That fact, in turn, could lead smart speaker manufacturers to build more electronics and processing capability into future devices, which will enable them to operate offline while also collecting more information from their environment.

While the increasing capabilities of such devices can enable more-targeted advertising, it could also provide a means for blackmail, Wueest said. Already, several companies with such devices have admitted their staff listens to some of the recordings from smart speakers. “It might be possible for someone to type in an email address and say, ‘Oh, yes. This is the device I want to listen to,’” Wueest said.

Even if a tech company providing free smartphone apps or smart home-based services has reasonable guidelines in place for securing data, consumers should still be concerned, Ashkenazi said. “Your private data is likely stored in a cloud database,” he said. “With all of the recent database breaches, how can you be sure that the next attacked database won’t hold years of collected data [capturing] your behaviors, preferences and location history?”