Design of a novel interpretability analysis method to handle out-of-distribution samples

Abstract

With the growing introduction of machine learning in critical domains such as healthcare, finance, law and other public services, the need for increased trust and understanding of these models is becoming paramount for their increased adoption. However, due to the nature of these models, which contain complex architectures and possibly millions of parameters, it has become exceedingly difficult even for practitioners of ML to understand and predict their behaviour, making each of these models to be ”black-boxes”. Explainable AI is a research domain which aims to explain and understand these black-box models, in order to assess them for their fairness and biases, thus allowing for greater trust in these models. This trust is critical in order to facilitate the increased and willing adoption of these ML models in the mentioned fields. This work wishes to analyze some modern Explainable AI methods and their strengths and shortcomings, and also explore ways to rectify their shortcomings in order to develop more robust methods. Finally, we wish to extend the methods explored in the domain of Out-of-distribution (OOD) sample detection, allowing for greater generalizability and shedding the ”closed-world” assumption that many modern ML algorithms work on.