Abstract:
The objective of this project is to explore the use of Large Language Models (LLMs) in au tomating aspects of software project management, starting with the objective of creating a ”Su per Project Manager” capable of handling tasks such as organizing workflows, assigning tasks, and generating code in an agile framework. Through iterative exploration and refinement, the project has evolved significantly in its scope and focus. Initial efforts aimed to replicate soft ware processes by integrating Jira workflows and GitHub repositories. However, challenges in generating consistent initial codebases prompted a shift towards replicating software evolution using GitHub commits as the primary input. The problem was further refined to focus on tran sitioning codebases between versions (e.g., v5.0 to v6.0) based solely on commit descriptions. Despite testing various methods such as zero-shot learning, Retrieval-Augmented Generation (RAG), multi-agent systems, and fine-tuning, results remained unsatisfactory. To address this, the scope was narrowed to generating code and test scripts written in Python. This led to another key realization: successful code generation requires first identifying changes and struc turing them into a coherent design plan. The project now centers on generating design plans for commit-based changes using summary generation of the code base and evaluating it against human-generated plans to measure alignment. We have also created a new dataset for this design plan evaluation using a open source project repository and generated result for our new refined problem statement.
