Virtual Reality Helps Train Low-Vision Patients for Real-World Situations

People who have low vision — those who have difficulty seeing even when wearing corrective lens, according to American Foundation for the Blind — may soon be able to learn basic skills such as crossing the street or learning their way around by simulating the experience through a new application of virtual reality, which is being designed by NJIT Associate Professor Jacob Chakareski and his students.

VR software, displayed to patients through goggles so the screens are directly in front of their eyes, could be prescribed by medical professionals called certified orientation and mobility specialists (COMS) in controlled practice environments. The specialist arranges the lessons, giving input and guidance in real-time to a goggles-wearing patient through a wireless Internet connection, which means the specialist and patient needn't be in the same place.

That's potentially a boon for patients with travel constraints, whether financial, logistical, medical, or due to the COVID-19 pandemic, and also for those who live far from medical vision facilities, which tend to be located in metropolitan areas.

But for it to work, first the software developers need to overcome technical obstacles. For example, the system requires ultra-fast, high-bandwidth connections so that trainers and trainees have the identical view in real-time. Experts call this field of software virtual reality-based intelligent orientation and mobility specialists, or VR-IOMS, which Chakareski said will take a big leap forward thanks to the bandwidth and speed of emerging fifth-generation (5G) cellular networks and millimeter-wave computer networking technology. There are also still geographic challenges, in that rural areas will probably be the last to get 5G networks from the major American cellular providers like AT&T, T-Mobile and Verizon. Chakareski wants the assistance technology to be ready when the networks become available.

"We work on the computer system challenges of how to do this," Chakareski said. "I have always been interested in marrying technology and healthcare. I have participated in multiple other projects that have had a direct impact on people’s health."

Chakareski said that even though his team's new Laboratory for Virtual and Augmented Reality Communications and Networked Systems is still under construction in the Ying Wu College of Computing, they are already pushing forward to develop custom algorithms, methods and prototypes. Today's off-the-shelf technology would be too big, heavy and slow, he explained. He intends to start clinical trials this year at Alabama Institute for Deaf and Blind (AIDB), supported by a five-year grant from National Institutes of Health which reached $500,000 in 2020 and in 2019. His research partner is Associate Professor Lei Liu from University of Alabama Birmingham's School of Optometry.

Experience gained here can be used to improve other fields of rehabilitation.

"A randomized training trial will be conducted to compare VR-IOMS and human COMS training. Low-vision subjects will be randomized to learn the three O&M tasks from the VR-IOMS, from human COMS and no training. All subjects will be taken to real streets by COMS before and after training to evaluate their O&M task performance. Pre- and post-training performance changes of the three groups will be compared to quantify relative training effectiveness," Chakareski wrote in the grant summary.

"If these VR-IOMS are proven effective, more will be developed and validated in future research. Clinical deployment of these training agents will restore independent travel to more low-vision patients. The VR-IOMSs, with their explicitly defined training strategies, objective assessments of learner behavior and quantitative outcome measures, can serve as platforms to advance O&M research," he stated. "Experience gained here can be used to improve other fields of rehabilitation."

Amber James, who works as a COMS at AIDB, said her 2,500 peers nationwide are not enough, so help in the form of new technology is welcome. She understands that technology will augment, not replace, human trainers.

"A program like this could teach [the visually impaired] the basics of orientation and mobility, and the use of their cane, or the use of their low vision, to get around safely and efficiently," James said.

James said she believes in the technology now, but was initially skeptical because teaching low-vision people is a hands-on, exploratory activity.

"When I think of teaching someone about how to time a street crossing, they need to be there to experience it all. It's not just the traffic moving by, it's all of the extra things going on that we can't plan for," she noted, such as ambient noise, harsh weather conditions, or a street light being out. "But, after hearing everything and actually seeing some of the programs in the VR instruction and training that they used in the past, it's pretty interesting to me. They do consider some of the unknowns."

The software and infrastructure isn't quite ready to function as an independent virtual guide dog in the wild. But, James concluded, "I think it's going to be a wonderful addition to low-vision rehabilitation and the number of consumers that we can reach."