Model for short subjective questions

Abstract

Exams are conducted to test whether the learner has gained knowledge about the concepts in the subject. Recently, during the pandemic-induced lockdown, there has been a rise in concerns regarding the lack of integrity in exams conducted in the online mode that consists of objective questions. To prevent the learners from guessing or exchanging solutions, the mode of tests administered must have sufficient subjective questions that can gauge whether the learner has understood the concept by mandating a detailed answer. Hence, in this B.Tech Project, we propose a methodology for automatically converting the objective questions to subjective questions. Though the state-of-the-art systems for question generation exist, they do not address the problem of converting objective questions to subjective ones. This project presents a novel unsupervised approach formulated using rule-based techniques, clustering, pattern recognition, syntactic features, and open-source knowledge bases such as Google's "People also ask" (PAA)section. The approach presented is very runtime e_cient as it doesn't rely on training and does not require any labeled data. The converted questions are further augmented with questions from a transformer-based model _ne-tuned for question generation. We then select the top-questions using a ranking mechanism. We have conducted extensive testing on a proprietary objective question bank from an e-learning platform and it is observed that the presented approach outperforms the existing data-driven approaches by 36.45% as measured by Recall@kand Precision@k.Further, a dataset has been curated for the short subjective questions which contains the variety of possible correct and wrong answers. This can electively become a base point for the task ofautomatic evaluation of the candidate answers to the short subjective questions. A system forthe same has also been proposed in this B.Tech Project and is tested for high level electiveness through some ablation studies. However, this model hasn't yet been tested end to end.