CONQUER Collaborative, Monell Chemical Senses Center, Room 100, located at 3508 Market Street.
BIOMED Master's Thesis Defense
Title:
The NEVRland Platform: An Immersive Virtual Reality (VR) Experimentation Platform for Research in Neuroergonomics
Speaker:
Callan Moira Powell, Master's Candidate
School of Biomedical Engineering, Science and Health Systems
Drexel University
Advisor:
Hasan Ayaz, PhD
Professor
School of Biomedical Engineering, Science and Health Systems
Drexel University
Details:
Neuroergonomics research utilizes continuous metrics of the brain, body, behavior, and environment to develop a wholistic understanding of the brain at work in the complex, naturalistic context of everyday life. Though the development of increasingly mobile neurophysiological monitoring devices has enabled real-world experiments, researchers must accept the tradeoff of experimental control that occurs when bringing investigations out of the controlled lab environment. Psychological researchers have historically mitigated this tradeoff by employing immersive virtual technologies that create the illusion of naturalistic interactions and scenarios that are of interest from within lab settings. Literature in psychology suggests that the more immersive the technology used in an experiment is, the more accessible naturalistic experimental paradigms are to the researcher without a tradeoff in experimental control. Head-mounted display virtual reality (HMD-VR) systems have become nearly synonymous with immersion, reliably evoking presence and embodiment.
Immersive VR-HMD paradigms are accordingly being adopted in neuroergonomics research to take advantage of the illusions of naturalistic interpretations and interactions they enable. Although research-specific platforms have been developed, they vary in terms of level of immersion, visual realism, coverage of cognitive domains, system transparency, or accessibility. The alternative to these platforms is to design a custom VR environment suitable for neuroergonomics research in-house using engines for creating video games, which still requires a significant amount of technical expertise specific to programming and VR design.
As a solution to the technological barrier posed by VR development in the context of neuroergonomics research, this thesis proposes NEVRland: a reliable, plug-and-play platform designed to naturalistically simulate any computer-based visual cognitive task through a photorealistic immersive virtual environment (NE short for neuroergonomics, VR for virtual reality). One goal of the NEVRland platform is to make immersive VR paradigms in neuroergonomics research more accessible, which is achieved by enabling the use of the pre-existing, robustly validated tasks already familiar to researchers from within the NEVRland environment. The default environment of the NEVRland platform emulates a generic behavioral experimentation room to support research where the visual task presentation medium is manipulated as an independent variable, as well as to serve as a control setting where needed.
In the default NEVRland environment, the unaltered task presentation generated in the original task software is mirrored on a virtual desktop monitor at a computer station. The positions of the keyboard and mouse at the virtual station mimic the common arrangement of a physical mouse and keyboard to support visual-haptic alignment and unbroken immersion. This immersion in NEVRland is also supported without task interference by the inclusion of hand-tracking. NEVRland was developed in Unreal Engine 5.3 using the natively integrated OpenXR runtime for cross-platform HMD compatibility. Local session logging was enabled through C++ scripting within the Unreal Engine blueprint architecture.
To evaluate NEVRland, a well-established paradigm for technological assessment of visual stimulus presentation event and duration timing was performed using a home computer and a Meta Quest 2 HMD. This paradigm was used to assess the NEVRland task presentation pipeline as a whole and in separate components. Evaluation was segmented into an assessment of the impact of running NEVRland on the same computer as the original task software as well as assessing the time delay introduced by using a screen-capture based task presentation method before evaluating the overall temporal distortion of a task presented in NEVRland as compared to its intended, isolated presentation. Validation test results suggest that the default NEVRland environment can run in parallel to the original task software without significantly impacting task presentation performance. Future work includes the evaluation of the NEVRland platform across a spectrum of cognitive paradigms while simultaneously collecting comprehensive neural and physiological data.