A Machine Learning and Large Language Model-Integrated Approach to Research Project Evaluation

Abstract

Research project evaluation upon completion is one of the important tasks for research management in government funding agencies and research institutions. Due to the increased number of funded projects, it is hard to find qualified reviewers in the same research disciplines. This paper proposes a machine learning and large language model integrated approach to provide decision support for research project evaluation. Machine learning algorithms are proposed to compute the weights of key performance indicators (KPIs) and scores of KPIs based on the evaluation results of completed projects, large language models are used to summarize research contributions or findings on project reports. Then domain experts are invited to consolidate the weights and scores for the KPIs and assess the novelty and impact of research contribution or findings. Experiments have been conducted in practical settings and the results have shown that the proposed method can greatly improve research management efficiency and provide more consistent evaluation results on funded research projects.