ScaleUp AI 2022: Risks of ‘Wild West’ AI in Health Care

AI is permeating all aspects of the health care system, whether it is in diagnostics, treatment, drug discovery, precision medicine, imaging and more.

“It’s really the sky’s the limit at this point, which is both a blessing and a curse because it’s the Wild West,” said Dr. Danny Tobey, a partner at law firm DLA Piper, during a panel at the ScaleUp: AI conference in New York. “There are no robust standards yet on what differentiates good and responsible from not as good and irresponsible AI.”

What’s more is consumers cannot tell the difference, Tobey said. “Everything looks kind of the same when you can’t get under the hood” of why AI recommends, say, a certain path of treatment.

“The thorny issue over the next three to five years is helping people make informed decisions about adoption,” he added.

AI can start off with certain outcomes and then pivot. “AI is increasingly able to change itself with new information in ways that a traditional pacemaker or insulin pump would not have five to 10 years ago,” Tobey added.

And the stakes of using AI in medical situations are high since it deals with people’s health.

Patient Health Must Come First

Patients need to be put first when designing AI for health care deployments. By doing so, Tobey said developers are ultimately making the correct design decision over time.

“We’re on the precipice where the doctor isn’t the most learned individual anymore,” Tobey added. “We are going to have a lot of interesting conversations not just about what is reasonable but who is reasonable.”

Agreeing that patients must be top of mind was Suchi Saria, founder and CEO of Bayesian Health, which has built an AI-based clinical decision support platform.

Saria warned that health care and medicine are unlike any other field and that technology developers cannot “say whatever and get away with it.”

“In the last seven or eight years, there have been considerable amounts of literature published and the industry is moving extremely fast,” she warned.

She told attendees that her team benefited from a combination of technology and culture when approaching design. “You have to have patience and vigor,” she added.

This article first appeared in IoT World Today’s sister publication AI Business.