Extended reality testing of autonomous vehicles

Abstract

As autonomous vehicles advance toward mainstream adoption, efficient testing methods have become paramount to expand their presence in crowded and unpredictable environments like those seen in Indian cities. Autonomous systems must perform better than human drivers to find a practical use in human society. This requires a reliable testing framework for verifying the deployability of autonomous systems in the real world. Contemporary testing paradigms for autonomous vehicles have three primary drawbacks: 1. Physical testing poses safety concerns for individuals in the surrounding environment, 2. Testing on simulator-based scenarios cannot ensure safety for real world deployment as many latent factors are missed out from the real world. 3. Critical scenarios are rarely encountered in real-world testing, and identifying such scenarios within long streams of data is hard. This thesis project, titled Extended Reality Testing of Autonomous Vehicles, aims to address this challenge by proposing and implementing a novel approach. The method first flags unique video segments where agents act unusually using active learning techniques, and later, we extract information from these video segments and inpaint them into the real-world data that an autonomous vehicle perceives, using generative diffusion models.