Enhancing data analysis and programming skills through structured prompt training: The impact of generative AI in engineering education

Abstract

The advent of Generative Artificial Intelligence (GenAI) and large language models like LLama, Palm2, GPT, Gemini, and Claude has revolutionized education by generating human-like text and contextually relevant responses. Our research investigates the impact of structured prompt training on students' learning in data analysis and programming. We experimented with 157 first-year engineering students divided into three groups: a control group (internet access, no GenAI), an experimental group 1 (internet and GenAI without prompt training), and an experimental group 2 (internet and GenAI with prompt training). The prompt training session included techniques like few-shot prompting, chain prompting, and the CLEAR framework. We assessed participants' performance in data analysis tasks using Python, with pre-tests and post-tests measuring their skills in programming across three Bloom's taxonomy levels (understanding, application, and analysis). ANOVA on post-test scores showed significant differences among the groups, with G3 (with prompt training) outperforming G2 (without prompt training) and the control group across all three levels, evidenced by higher mean scores (G3: 6.60, G2: 4.94, Control: 4.28), similar pattern observed in task completion also. These results underscore the effectiveness of structured prompt training in enhancing students' data analysis and programming skills. Our study highlights the potential of GenAI and structured prompt training to transform educational practices and suggests future research directions, including integrating prompt engineering within human-AI collaboration.