Virtual reality guided machinary tutorials: from concept to prototype

Abstract

This project endeavors to enhance the immersive experience in Virtual Reality (VR) environments by incorporating an eye-tracking module into the Oculus Quest Pro headset. The primary focus lies in extracting crucial eye-tracking data, specifically blink rate, saccades, and fixation, to gain valuable insights into user behavior within the virtual space. Initially, the project explored the feasibility of utilizing OpenXR libraries for eye-tracking data extraction. However, due to inadequate documentation and limited online resources, a decision was made to construct the entire module from the ground up, employing the Oculus Integration SDK within the Unity development environment. The development process encountered challenges, particularly in replicating the functionality achieved through OpenXR. In seeking solutions, a revisit to the initial approach led to an exploration of extracting data through the XR Interaction Toolkit, a Unity-developed framework that interfaces with OpenXR libraries. Assistance from the Meta forum further clarified the extraction process, revealing that the XR Interaction Toolkit could be employed to obtain eye-tracking data, a pivotal step towards achieving the project's objectives. However, challenges persisted as Meta's privacy policies restricted access to specific eye-tracking metrics such as blink rate, fixation, and saccades and provided only the gaze data. Consequently, the project shifted its focus to extracting desired values from the available gaze data. This necessitated an in-depth exploration of Python libraries and research papers to develop an algorithm capable of computing fixation, blink rate, and saccade values.. While progress has been made in implementing the fixation algorithm, challenges remain in decoding blink rate and saccade values due to the absence of certain parameters in the provided gaze data. Efforts are ongoing to address these challenges and complete the comprehensive eye-tracking module for the Oculus Quest Pro, ultimately contributing to a more immersive and intuitive virtual reality experience.