A team at UTD will develop an application allowing students to practice interviewing medical patients through augmented reality. The concept won the U.S. Ignite challenge, a competition “to develop the next generation internet applications that will power smart cities of the future.”
With a $200,000 grant and a collaboration with UT Southwestern Medical Center, a UTD team will create an Emotive Virtual Patient (EVP) as part of the Emotive Virtual Reality Patient System.
The team creating the EVP is researching the concept of using augmented reality to facilitate medical interviewing practice. They narrowed down the options to Microsoft’s HoloLens. With the HoloLens, students will be able to see a patient and fine-tune their medical interviewing skills.
“(HoloLens) … is a totally inclusive device. So in other words, there’s no computer or anything necessary for this,” said Marjorie Zielke, ATEC professor and director of the center for modeling and simulation at UTD.
One of the reasons the application will be used as opposed to people is because students do not always have actors available to them, for instance, when they are at home.
Winning the U.S. Ignite challenge provided another $5,000, which the team will use to create a “spectator view,” where a third party can observe both the student and what the student sees through the HoloLens. Another $5,000 will be awarded once the application is complete.
“That’s what really distinguishes this from what other people are doing, is using the spectator view so that a remote expert can provide real-time feedback rather than just watching something, but can actually have an interchange, and that’s what makes this app unique,” said Gary Hardee, associate director for the center for modeling and simulation.
The patient is viewed by the student through augmented reality. Augmented reality allows an image to be projected into a room, while virtual reality takes a person into a projected room.
“What we’re trying to do is to create virtual patients that can do things that real standardized patients can’t do, represent conditions that are unique and rare. And all of our virtual characters are designed to solve issues that are challenging to do in the real world and education,” Hardee said.
Zielke and her team proposed the EVP to U.S. Ignite because it would also allow the team a “slice” of their high-speed network. The third party would watch the student and “patient” through video, using the network. Zielke said in the medical world, students needed to learn of mistakes immediately from the third party, so the EVP would benefit from the high-speed network.
The application only had to meet one of the three criteria U.S. Ignite requested – Software Design Networking, low latency or high bandwidth – and the EVP satisfied them all.
“It’s kind of something that would be a(n) evolutionary — revolutionary, if you want to call it — type of application,” said Mike Skelton, director of the mayor’s office of international business for the Richardson economic development partnership and judge for the U.S. Ignite Challenge. “Augmented reality is state of the art and we need applications like (the team’s) to stimulate the infrastructure.”
Learning to use the HoloLens in one of the challenges Zielke expects to face, as it is a modern technology. Another is being able to use natural language, or the way humans talk to each other. Zielke said because the language of the EVP is constrained to medical language, the natural language will not be as extensive.
“You’re literally trying to recreate humanity and we don’t pretend like that’s not challenging,” she said.
The goal is to become a world-renowned center for virtual human research and to one day possibly create an actual hologram.
“Because we’re a research lab, we deliberately look for things that nobody else has done … we’re not just any development shop that just does projects,” Zielke said. “We’re literally looking for things that are new and nobody knows how to do.”