IoT World Austin 2022: Who Is Responsible for Holding AI Accountable?

Speakers at IoT World and AI Summit in Austin were torn as to whom the responsibility should fall on to hold AI models accountable.

Paul Hecht, senior product marketing manager at Telus, said the “whole ecosystem” was responsible for holding AI accountable – including consumers.

“Consumers are responsible,” he said, “we get to vote with our wallets.”

“It’s incumbent upon organizations that are releasing a product to be able to show transparency to its customers. Customers are going to be more loyal to organizations that are transparent that know that their data is safe and that the methods they’re using are responsible.”

Hecht’s view on consumer responsibility for holding commercial deployments accountable was not shared by Andy Terrel, VP of ML at Korbit AI, who said the commercial world “is hiding.”

Terrel referenced the algorithmic system used by Starbucks to determine employee hours. The automated system would make schedules – but workers have expressed dismay that the software scheduled irregular shifts and would often reduce weekday hours in favor of longer weekend workdays.

“If I walk into Starbucks, I don’t expect it,” Terrel said. “And so the way corporations work has hidden a lot of these details.

“It’s coming out now a lot more about the treatment of folks and things like that, but it’s incumbent upon us as technology leaders who know how these systems work to … speak up.”

Consumers aside, Terrel and Hercht agreed, however, that organizations should teach all employees about responsible AI use and ways to avoid potential biases.

“Ultimately, anybody who’s touching an AI system that doesn’t see it following principles and that there are certain biases in how it works, I think they’re not being fair and responsible,” said Terrel.

Hercht agreed, adding, “I’ve heard it, fortunately, echoed throughout the conference the last couple of days is it can’t be the sole responsibility of a group of data scientists in the lab. It has to be top to bottom and an organization more than merely paying lip service.”

And fellow panelist Brian Yang, senior director of product strategy at Verint, echoed the pair’s points, saying, “there’s a degree of responsibility that everyone holds.”

“At the end of the day, it’s unfair to pin everything on one area.,” he said. “The key is always about transparency, and knowing where it is, how it’s being used and bringing that general awareness to the public.”

AI Bill of Rights

Responsible and ethical approaches to AI have become a growing industry trend.

Governments around the world are attempting to weigh in on the matter. The EU’s proposed AI legislation would categorize every system based on levels of the potential harm they would inflict. 

Meanwhile, the Biden administration published the AI Bill of Rights – not a piece of legislation, but a set of guidelines, to avoid impact on civil rights when developing AI.

AI Business asked the panel whether the Bill of Rights non-binding nature went far enough.

Terrel said the AI Bill of Rights was effectively the first step in how laws get made and that regulation is on the way.

“There should be a Bill of Rights,” Terrel said, “like what data you’re going to see about me when you’re making a legal decision or deciding my education rights.

“And so now, is it as broad as some people are asking for like, where, you can’t ever use anything I put on the internet? I don’t know about that. There are boundaries. But absolutely, things are coming down.”

Hercht acknowledged that other jurisdictions are further ahead – referring to the AI Accountability Act in Canada as well as AI laws in Estonia.

He also pointed to the EU’s GDPR as an example of drawing the line between regulation and consumer behavior and poor behavior.

“When was the last time you opened a webpage, and it didn’t ask you to manage your cookies? That’s a direct result of regulation,” he said.

This article first appeared in IoT World Today’s sister publication AI Business.