LLM-based research assistant

Abstract

This project explores the capabilities of Large Language Models (LLMs) as research assistants by developing an advanced research paper reviewer that surpasses current leading tools such as AI-Scientist developed by SakanaAI. Initially, using foundational APIs from Google Gemini and OpenAI’s GPT models, we created a baseline reviewer that generated generic feedback in adherence to target conference guidelines. To enhance its effectiveness, we implemented a sophisticated pipeline incorporating agentic patterns and reflection-based models, enabling iterative refinement of reviews for increased specificity and accuracy. Using ExtractorAPI, the system retrieves conference-specific review guidelines, allowing the LLM to dynamically split research papers into independent sections for detailed evaluation. In subsequent iterations, we transitioned to a LangGraph-based architecture, adopting a multi agent approach with a supervisor node and integrated tools like internet search and Semantic Scholar to emulate a professional reviewer’s comprehensive capabilities. This enhanced workflow not only automates the review process but also delivers high-quality, conference-aligned feedback more efficiently than existing solutions. Our results demonstrate the potential of LLMs to significantly streamline the research review process, offering scalable and consistent support to researchers.