A comparative study of AI-generated and human-crafted learning objectives in computing education

Abstract

Background

In computing education, educators are constantly faced with the challenge of developing new curricula, including learning objectives (LOs), while ensuring that existing courses remain relevant. Large language models (LLMs) were shown to successfully generate a wide spectrum of natural language artefacts in computing education.

Objectives

The objective of this study is to evaluate if it is feasible for a state-of-the-art LLM to support curricular design by proposing lists of high-quality LOs.

Methods

We propose a simple LLM-powered framework for the automatic generation of LOs. Two human evaluators compare the automatically generated LOs to the human-crafted ones in terms of their alignment with course goals, meeting the SMART criteria, mutual overlap, and appropriateness of ordering.

Results

We found that automatically generated LOs are comparable to LOs authored by instructors in many respects, including being measurable and relevant while exhibiting some limitations (e.g., sometimes not being specific or achievable). LOs were also comparable in their alignment with the high-level course goals. Finally, auto-generated LOs were often deemed to be better organised (order, non-overlap) than the human-authored ones.

Conclusions

Our findings suggest that LLM could support educators in designing their courses by providing reasonable suggestions for LOs.