Evaluating the quality of ontologies

Abstract

Ontologies serve as fundamental tools for organizing and representing knowledge in various do mains. Evaluating ontology quality is crucial for ensuring their e!ectiveness and usability. This paper presents a comprehensive review of ontology evaluation framework, encompassing both traditional approaches and emerging techniques. Method-based evaluation, focusing on criteria such as syntax, structure, and semantics, o!ers insights into ontology design and construction. Additionally, level-based evaluation divides assessment into distinct layers, enhancing the un derstanding of ontology characteristics. Our tool will combine the major frameworks used by knowledge experts to evaluate their ontologies like OQuaRE, OntoQA, Orme et. al., etc., which are selected after several literature reviews about their use cases. Our tool is complemented with an additional layer of LLMs suggesting knowledge experts on how to improve their ontol ogy based on the calculated metrics and o!ering in-depth insights from each calculated metric. The LLMs in our tool are connected via a retrieval augmentation pipeline. Our tool is built to overcome the limitations of pre-exisitng tools which are either not working, not available, do not take all major frameworks into account or do not provide insights regarding calculated metrics for the knowledge expert to improve their ontology. Our paper concludes with insights into future research directions and the importance of adopting a holistic approach to ontology evaluation.