5 Ways Virtual and Augmented Reality Are Changing Medicine
Especially after Facebook’s famous acquisition of Oculus VR, you probably associate virtual reality exclusively with video games. But the technology that creates immersive, three-dimensional virtual environments has huge potential not only for the creation of captivating entertainment experiences, but also for changing how fields like medicine have traditionally worked. From head-mounted displays to five-walled environments, and from computer-based solutions to Google Glass, here are five different experiments that have demonstrated how virtual and augmented reality are already changing the way that diseases and chronic conditions are diagnosed and treated.
1. Virtual reality can help diagnose cognitive decline
A new research project at the University of California’s Qualcomm Institute called the VE-HuNT System (Virtual Environment Human Navigation Task) looks to use virtual reality to aid the diagnosis and treatment of memory failure and diseases like dementia. The system is a research project of Eduardo Macagno, a biological sciences professor at the University of California San Diego.
When diagnosing cognitive decline, physicians often assess how patients navigate a neighborhood or a building, since that ability is one of the first to suffer. But it’s difficult for researchers to detect when an adult becomes lost or disoriented in real time, so Macagno’s virtual reality-based tool places test subjects in an interactive “room,” which they navigate with a steering wheel and a gas pedal. Test subjects will be asked to perform an increasingly difficult range of navigational tests, like finding a colored tile on the floor with and without navigational cues. Macagno explains: “The idea is to give an older person a series of tests and see where they fail. We record how long it takes them, which paths they take.” An algorithm will track the test subject’s visual attention to assess visuospatial abilities.
The VE-HuNT research will take place in either Qualcomm Institute’s NexCAVE or StarCAVE virtual reality environments.(Earlier news releases from the university explain that the StarCAVE is a five-walled virtual reality room that projects 3-D animations on 360-degree screens surrounding the viewer and on the floor, while the NexCAVE uses 9 HDTV screens to create high-definition virtual reality that’s very close to human vision, without the need for the projects that the StarCAVE relies on.) The VE-HuNT system itself is built of consumer components, and is reportedly compact enough to be placed in any neurology or clinical facility, where it would provide a less expensive alternative for risk assessment than the MRI or PET scans that are currently used. Macagno expects the tool to be used by clinical and biomedical researchers, healthcare providers, designers, and potentially healthcare architects.
Macagno notes that the use of a head-mounted virtual reality display like the Oculus Rift has also been proposed for the research, which will involve 20 to 30 adults in states of cognitive decline, plus a corresponding number of control subjects, in its first trial. But he says that the feeling of being disembodied or not being able to see oneself leads to vertigo and other balance problems in older adults, and head-mounted displays also compromise the wearer’s peripheral vision, which is integral to navigation ability, particularly in seniors.
2. Virtually reality improves social cognition in young adults with high-functioning autism
Several studies have used virtual reality as a therapy method for both children and adults with autism spectrum disorders. One study, by Michelle R. Kandalaft, Nyaz Didehbani, Daniel C. Krawczyk, Tandra T. Allen, and Sandra B. Chapman, all researchers at the University of Texas Dallas, focused on using virtual reality social cognition training to help eight young adults with high-functioning autism enhance their social skills, social cognition, and social functioning.
The research focused on young adults because young adults with autism typically find the transition to adulthood difficult, as social impairments interfere with the process of building relationships, functioning in a job, and participating in a community. Each of these aspects of social functioning take on an increasing importance in the transition to adulthood, and traditional therapy methods have focused on selected social skills to improve broader social cognition and functioning.
The research at the University of Texas used virtual reality technology developed using Second Life, a 3-D virtual world software, to enable study participants to practice and gain feedback in realistic social scenarios. The virtual reality environment included an office building, a pool hall, a fast food restaurant, a technology store, an apartment, a coffee house, an outlet store, a school, a campground, and a central park, and each participant and coach had an avatar modeled after him or her. The avatars were controlled with a standard keyboard and mouse, and one clinician used MorphVox voice manipulation software to sound like whichever avatar she was using in the virtual reality environment, such as the interviewer in a job interview scenario.
By practicing situations like social introductions, conversations with friends, managing a conflict with co-workers, or interacting with someone of a different background, participants improved in three areas of social skills and cognition: verbal and non-verbal recognition, theory of mind, and conversational skills. The research shows that a virtual reality environment is an interactive and visually stimulating approach to treatment, and is a promising method of improving social skills and functioning.
3. Exposure therapy in virtual environments relieves post-traumatic stress
Research at the University of Southern California’s Institute for Creative Technologies uses virtual reality exposure therapy to relieve post-traumatic stress. The therapy system, called Bravemind, is based on the concept of exposure therapy, in which a patient confronts and processes his or her trauma memories by retelling the experience to a therapist. Traditional methods of exposure therapy have been endorsed as an effective treatment for post-traumatic stress. The ICT’s Bravemind system takes exposure therapy a step further by leveraging virtual reality to enable patients to experience a scenario again instead of simply relying on their memory as they recount an experience.
The Bravemind system is currently used at more than 60 sites, including VA hospitals, military bases, and universities. Researchers have created virtual reality worlds that simulate Afghan and Iraqi cities, desert road environments, and scenarios that are relevant to combat medics. Additionally, scenarios that address military sexual trauma are in development.
The virtual environment is experienced via a head-mounted virtual reality display, and all of the simulations include directional 3D audio, vibrations, and even smells . A trained clinician controls the environment and is in contact with the patient via audio. Researchers are adapting the Bravemind system as a tool for stress resilience training, and also for assessment of post-traumatic stress.
Bravemind project leader Albert “Skip” Rizzo has also developed virtual reality systems for physical rehabilitation of patients who have experienced stroke and traumatic brain injury, and for prosthetic use training. He’s currently designing scenarios for patients with autism spectrum disorders to address social and vocational interaction.
4. Augmented reality therapy treats phantom limb pain
Researcher Max Ortiz Catalan, a doctoral student in biomedical engineering at Chalmers University of Technology in Sweden, and colleagues Nichlas Sander, Morten B. Kristoffersen, Bo Hakansson, and Rickard Branemark developed an augmented reality therapy to relieve an amputee’s phantom limb pain. Phantom limb pain is traditionally treated with a variety of different methods, among them mirror therapy. Mirror therapy uses an actual mirror to train the body to reconfigure its mental representation or where each limb of the body is. The body often misrepresents where a limb is when severe injury or the loss of a limb occurs, and experts think these misrepresentations cause pain in “phantom limbs” that are no longer there. The brain remembers the pain in a limb before it was removed, and continues sending the same nerve signals because it thinks that the limb is still there. In mirror therapy, the patients moves both limbs — the one that is still intact and the phantom limb — in coordination to correct the body’s misrepresentations of the limb and pain.
In the research at Chalmers University, an experimental treatment was used in a case study with a patient who experienced chronic phantom limb pain for forty-eight years, and didn’t respond to other medical and non-medical methods of therapy. In the study, the patient controlled a virtual limb with signals from his limb stump. The virtual limb responded directly to myoelectric activity at the stump, and the illusion of a restored limb was enhanced through augmented reality. The predicted movements were then used in virtual and augmented reality environments, including a racing game.
The patient initially reported that the posture of the phantom arm was a closed fist, and the therapy helped that posture gradually relax, and resemble the posture of the virtual limb. Additionally, the patient became able to move the phantom limb, and the phantom hand was restored to the anatomically correct distance. The patient also experienced a reduction in pain, and even experienced periods without pain. The augmented reality therapy, combined with the patient’s ability to control the phantom limb with electrical signals from the stump, is thought to make the therapy more effective than traditional mirror therapy or virtual reality mirror therapy, which requires moving two limbs simultaneously (a method that excludes bilateral amputees) and is more restricted and less realistic than the augmented reality therapy.
5. Augmented reality improves surgeons’ accuracy
As VentureBeat reported in March, Stanford physician Homero Rivas has experimented with a Google Glass app called MedicAR, developed by Droiders, a European team of mobile developers who specialize in “Glassware,” or apps for Google Glass. The partnership between Rivas, who is Stanford’s director of innovative surgery, and Droiders came out of the interest in wearable tech that Rivas shares with Julian Beltran, Droiders’ chief executive.
Using the augmented reality capabilities of Google Glass, surgeons can see a procedure illustrated step by step, and superimposed on the skin of the patient. That enables them to start a procedure with a better idea of where a mass is, or where an incision should go, so that they minimize the trauma of the procedure. With the MedicAR app, surgeons wearing Google Glass can look to the right for video playback of the augmented reality procedure overlaid on the patient. They can see MRI imaging directly over the patient, to determine exactly where the procedure is necessary.
Rivas foresees a variety of medical uses for Google Glass and the apps that developers build for the wearable device. Surgeons will be able to perform procedures while consulting with others long distances away, and streaming operations online will enable the medical community to better benefit from the insight afforded by expert surgeons. Additionally, medical students will be able to use augmented reality to improve their surgical technique. They could even use the technology instead of cadavers or real (live) patients.