MIT Creates Virtual Avatar to Train Brain Surgeons

MIT has teamed up with AR/VR startup EDUCSIM to create a virtual avatar for surgeons to practice and teach neurosurgery.

In a recent test, pediatric neurosurgeon Benjamin Warf used the technology to perform simulated surgery on a model of a baby’s brain. Warf himself was in MIT’s Immersion Lab, operating a virtual avatar more than 3,000 miles away in Brazil where neurosurgery resident doctor Matheus Vasconcelos observed.

Using a pair of virtual reality (VR) goggles, Vasconcelos was able to watch Warf’s avatar demonstrate the brain surgery, before taking over himself.

“It was my first training using this model, and it had excellent performance,” said Vasconcelos. “As a resident, I now feel more confident and comfortable applying the technique in a real patient under the guidance of a professor.”

The project responds to current limitations in training and teaching options for surgeons by leveraging VR and simulations to provide more in-depth and rapid learning opportunities.

Giselle Coelho, EDUCSIM’s scientific director, created the digital avatar, as well as the model of the baby’s brain on which to operate. She said the physical model was created with all the structures of the brain and can even bleed, “simulating all the steps of a surgery, from incision to skin closure.”

Related:MIT Automated Co-Pilot Designed for Safer Flights

EDUCSIM used MIT’s Immersion Lab to craft Warf’s avatar, leveraging high-fidelity motion-capture technology, volumetric video capture, and a range of VR/AR technologies to capture the intricacies of Warf’s movements. 

Warf was also observed while wearing gloves and clothing with embedded sensors, to gain a holistic understanding of the surgery.

Warf’s avatar can also respond to questions, using an AI algorithm programmed with responses from a lengthy Q&A with Warf.

While testing is still underway, MIT said potential for the project is huge, with virtual training seeing increased popularity in the wake of the pandemic.

“One day we could have avatars, like this one from Giselle, in remote places showing people how to do things and answering questions for them, without the cost of travel, without the time cost and so forth, I think it could be really powerful,” Warf said.