Virtual reality can help prepare cancer patients for urgent situations. Image created with Adobe Firefly AI.

Virtual reality expert Erin Truesdell, in NJIT’s Ying Wu College of Computing, is designing a new kind of user interaction technology that could help cancer patients and caregivers prepare for potential complications due to fevers.

Truesdell works on the technical challenges inherent in multiple people trying to simultaneously interact through virtual reality. She previously worked with VR gaming and pediatric cancer specialists at Drexel University and Yale University School of Medicine, so she signed on again to help them study how simulations can help patients understand what will happen in the event of a medical emergency.

She explained that a common emergency for blood cancer patients aged 13-39 is febrile neutropenia, when people have a low white blood cell count and a fever above 100.4 degrees. The weakened immune system leads to sepsis, where the body overproduces infection-fighting germs. Organ damage or death can result.

Doctors, scientists and software developers at the trio of universities, along with a virtual reality company in New York called Glimpse Group, are building a virtual reality environment that shows patients how to identify, react to, and clinically understand what will happen should febrile neutropenia occur. They’re funded by a National Institutes of Health grant totalling almost $400,000.

“When you are diagnosed with cancer, or your child or spouse is diagnosed with cancer, on that diagnosis day you are handed a ton of paperwork written in complicated medical jargon that tells you about all of the things that can kill you — not great for retaining information,” Truesdell noted. “For something like febrile neutropenia, one of the bad things is that the longer it takes you to figure out what to do in that situation, the more risky and the higher your risk of mortality is. Our goal is to leverage virtual reality technology and basically use a VR dress rehearsal to prepare patients.”

VR has been shown, because of the increased immersion, to help with knowledge retention ... Our goal is to apply this to a patient education setting

"VR has been shown, because of the increased immersion, to help with knowledge retention. And there are several studies that have been done specifically using VR to help people specifically practice procedural skills and retain that knowledge. Our goal is to apply this to a patient education setting, and that's some of the novelty of this research.”

The other part is technical. Virtual reality generally works for individual users, and it can work decently if multiple users are in the same physical space. The latter required intensive computing power until recently, Truesdell said. Her roles in the project are to find more efficient ways for multiuser systems to interact, and also to consider how users can avoid physically bumping into each other.

“Fortunately, the era of needing a big PC to run every headset is in the past, so we're more able to put multiple headsets in the same room. We have two classes of problems: technical and design,” Truesdell explained.

“Technical problems include the challenge of actually programming the thing — these headsets don't come with off-the-shelf solutions to showing other headsets and headset users in the same space, since most people aren't using multiple VR headsets at home. We'll have to develop a system that lets each headset track the others in space, which may be achieved through camera-based computer vision, either on the headset or external, manual calibration, or additions to inbuilt room-scanning systems.”

“Design-wise, two of our big questions are how to position objects in virtual space so the people who need to interact with them can, and how to handle audio when each headset has its own set of speakers. We're experimenting with different grabbing methods now to determine if something like a ‘tractor beam’ can help players grab the virtual objects they need, and we're laying out the space in the intervention such that anyone within the group space can reach any relevant objects, such as a thermometer or phone. In terms of sound, we'll likely tackle this by keeping audio output to a minimum to allow users to talk with each other in real life, tailoring specific audio cues to individual users to avoid overlap.”

Truesdell, for yet another segment of the project, plans to hire students from the computing college or NJIT’s Hillier College of Architecture and Design, who will build digital 3D models of common items in a healthcare environment as well as household items. “I plan to recruit students from across campus, because we have lots of great students, and I'm looking forward to being able to bring them in and involve them,” she said.

She said that if the project succeeds, then the same approach can be applied to other healthcare intervention scenarios.