Virtual reality (VR) uses advanced display and immersive audio technologies to create an interactive, three-dimensional image or environment. Augmented reality (AR), meanwhile, uses digital technology to overlay video and audio onto the physical world to provide information and embellish our experiences.
At the University of Rochester, we’re crossing disciplines to collaborate on VR/AR innovations that will revolutionize how we learn, discover, heal, and create as we work to make the world ever better.
An accidental discovery by Rochester researchers in 2003 touched off a wave of research into the area of neuroplasticity in adults, or how the brain’s neural connections change throughout a person’s lifespan.
Fifteen years ago, Shawn Green was a graduate student of Daphne Bavelier, then an associate professor of brain and cognitive sciences at the University. As the two created visual tests together, Green demonstrated exceptional proficiency at taking these tests himself. The two researchers hypothesized that it might be due to his extensive experience playing first-person, action-based video games. From there, Green and Bavelier demonstrated that, indeed, action-based video games enhance the brain’s ability to process visual information.
In years since, video gaming technology has gotten more sophisticated, regularly incorporating or featuring virtual reality (VR). The Oculus Rift headset, for example, connects directly to your PC to create an immersive VR gaming experience.
If we know that action-based video games enhance visual attention, might VR games do the same (and perhaps to a greater degree) because of the increased level of immersion?
That’s the question a current group of Rochester researchers—Duje Tadin, associate professor of brain and cognitive sciences; Jeffrey Bazarian, professor of emergency medicine; and Feng (Vankee) Lin, assistant professor in the School of Nursing—hope to answer.
“Much of the existing research on brain plasticity in adults has focused on healthy adults,” says Tadin, who studies the neural mechanisms of visual perception. However, Tadin’s own recent research has shown that brain-training video games can improve the peripheral vision of children with poor eyesight. In other words, he says, “The people who might benefit the most from this work are those who have decrements in their attention or cognitive abilities.”
“All three of us are studying the brain, but in different disciplines. And I think the biggest scientific advancements come not from within a discipline, but from branching out.”
But accessing such populations can prove challenging on a college campus, where undergraduate students (the typical study participants) abound.
A meeting of interdisciplinary minds
Enter Lin and Bazarian.
Tadin was one of Lin’s postdoc mentors, and Lin is now a leading researcher on cognitive aging in older adults. Her lab studies the use of computerized cognitive training to help slow cognitive decline. Meanwhile, Tadin met Bazarian during a February 2017 meeting of the Health Lab, a multidisciplinary campus collaboration that aims to bring computing, analytics, and technology to bear on modern-day challenges in medicine.
“When I went to this meeting, I didn’t realize how many people on the River Campus were doing work with potential medical applications,” recalls Bazarian, who studies concussions and other traumatic brain injuries with an eye toward preventing, diagnosing, and treating such conditions. “And Duje probably had no idea there were researchers like me with patients who could help test his ideas.”
“All three of us are studying the brain, but in different disciplines,” Tadin says. “And I think the biggest scientific advancements come not from within a discipline, but from branching out.”
The researchers presented their idea at the AR/VR Grand Challenge in April 2017, a University-wide retreat connecting researchers across the disciplines and various campuses. Combining their scientific expertise and resources helped them secure grant funding from Arts, Sciences & Engineering, the Medical Center, and the School of Nursing to study the effects of VR brain training on three groups:
- Healthy individuals, whose results would provide a baseline measure
- People with mild traumatic brain injury (TBI) resulting from concussions
- Older adults with mild cognitive impairment (MCI), a group at high risk for developing Alzheimer’s disease, but still with the possibility of recovering normal function
Wearing an Oculus headset and using the NeuroTrainer software, the study participants must keep track of specified objects—basketballs on a court, for example—in their visual field. As the difficulty level increases, there is more visual sensory input presented across a broader area, along with the introduction of surprising or unexpected elements.
“We’re basically taking the key components of video games, using them in an immersive VR setting, and then collecting the data with eye-tracking technology,” says Tadin.
Providing care using VR technology
Although Tadin was familiar with computerized cognitive training for older adults through his work with Lin, he admits he was unaware of the problems that people with concussions experience when it comes to their eye movements.
“We used to think that with a concussion you just get a headache,” Bazarian explains. “But it turns out, there are measurable deficits in how concussed patients’ eyes track and move in all directions across the visual field.” Those movements indicate an injury to the brain. Typically, these deficits have been observed by physical therapists while working with patients in a clinical setting. In such cases, the patient is instructed to go home and repeat the brain training exercises.
But it can be difficult for many patients to follow through with the typical treatment plan. “These are often adolescent kids who want something fun and engaging. If we can give them that through virtual reality, then their potential adherence to the regimen would increase and hasten their recovery,” Bazarian says.
“Not to mention, concussion specialists are few and far between. If this project is successful, patients wouldn’t have to come to the doctor’s office for treatment.”
Ideally, by the time the researchers complete their study roughly a year from now, the next generation of VR equipment will be more user-friendly and affordable. “Right now, you need special hardware that’s tethered to a computer, one that requires a high-performance video card,” Tadin says. Meanwhile, the mobile or cardboard box-style headsets often have problems with flickering.
But the future is near—and the Rochester researchers are poised to bring therapeutic interventions harnessing VR technology to their patients. “Oculus has a stand-alone model, Oculus Go, coming out in April,” notes Tadin. “If we have our intervention worked out by the time the next generation of hardware comes out, we’ll be ahead of the curve.”