Teaching Plan Generation and Evaluation With GPT-4: Unleashing the Potential of LLM in Instructional Design

Abstract

This study explores and analyzes the specific performance of large language models (LLMs) in instructional design, aiming to unveil their potential strengths and possible weaknesses. Recently, the influence of LLMs has gradually increased in multiple fields, yet exploratory research on their application in education remains relatively scarce. In response to this situation, our research, grounded in pedagogical content knowledge theory, initially formulated an instructional design framework based on mathematical problem chains and corresponding prompt instructions. Subsequently, a comprehensive tool for assessing LLM's instructional design capabilities was developed. Utilizing Generative Pretrained Transformer 4, a high school mathematics teaching plan dataset was generated. Finally, the performance of LLMs in instructional design was evaluated. The evaluation results revealed that the teaching plans generated by LLMs excel in setting instructional objectives, identifying teaching priorities, organizing problem chains and teaching activities, articulating subject content, and selecting methods and strategies. Particularly commendable performance was noted in the modules of statistics and functions. However, there is room for improvement in aspects related to mathematical culture and interdisciplinary assessment, as well as in the geometry and algebra modules. Lastly, this study proposes initiatives, such as LLM prompt-based teacher training and the integration of mathematics-focused LLMs. These suggestions aim to advance personalized instructional design and professional development of teachers, offering educators new insights into the in-depth application of LLMs.