Fact verification in CQA platforms

Abstract

Healthcare Community Question Answering (CQA) platforms provide valuable information on medical topics but are often rife with misinformation, posing significant risks to users’ health. Inspired by the need for factual accuracy in healthcare CQA, our study focuses on fact verification using the PUMA dataset. This semester, we have primarily concentrated on evaluating the factual correctness of claims in CQA responses rather than tackling the summarization task. Our objective is to determine whether the generated summaries or individual claims are factually correct by verifying them against reliable evidence. Additionally, we aim to lay the groundwork for future research in creating refined summaries that either retain only factually correct claims or normalize incorrect ones through evidence-based corrections. Over the past three months, we have focused on understanding the PUMA dataset, extracting claims from its complex structure, assessing claim check-worthiness, and verifying claims against reliable evidence. We also performed data annotation using various approaches wherever necessary to enhance the dataset’s utility for fact verification tasks. This research aims to contribute to developing robust tools that ensure the accuracy and reliability of healthcare information on CQA platforms, ultimately enhancing user trust and safety.