Augmented reality game design development in unity

Abstract

This thesis shows the development of a novel framework which is designed to make dynamic interaction between virtual objects and real-world objects in an augmented reality (AR) envi ronment. This framework is designed to work with Microsoft Hololens 2, as it utilizes Spatial Awareness to recognize real-world objects, this is all demonstrated in an AR game. The objec tive is to achieve a seamless integration of our component to virtual objects in Unity so that virtual objects not only coexist but also interact with real-life objects in real time. An additional focus of this research was to create a system capable of recognizing sequences of gestures to perform and execute pre-defined functions. Though the implementation of said task is ongoing, the method involves the use of Convolutional Neural Networks (CNNs) for identifying gestures, and coupling this with Long Short-Term Memory(LSTM) networks will help in processing the sequences of the gestures. The project uses Unity and the Mixed Reality Toolkit (MRTK) to develop the AR game, component as well as train our model. This will provide a way to advance AR applications in fields such as gaming, education, and industrial training.