Knowledge graph creation using LLM

Abstract

Knowledge Graphs are essential for structuring data into machine-interpretable formats, enabling ac- curate information retrieval, improved decision-making, and understanding of complex relationships. While KGs have seen significant progress in many languages, Hindi-specific KGs remain underdeveloped, limiting technological advancements and knowledge accessibility in the language. Large language models, with their extensive training on diverse datasets and deep contextual understanding, offer the potential for constructing Hindi KGs, surpassing traditional models like encoders in identifying and mapping complex relationships effectively. This study evaluates two LLMs — Bard, and GPT for generating RDF triples from Hindi NCERT Bi- ology texts. Other hindi llm’s like Llama Results reveal significant variation in accuracy, with Bard achieving a maximum hits@1 accuracy of 32.2%, and Llama performing poorly at 10.5%. After improvising the prompt, for GPT 4o model, it gave an average of 64.5% accuracy for three different Hindi chapters of Biology. The limitations of improvising the performance demonstrated better contextual understanding but faced challenges in extracting relationships from complex, nested sentence structures. Prompt engineering improved GPT’s performance to some extent, yet limitations in handling intricate information and the scarcity of annotated data in Hindi remain barriers to generating reliable KGs.