Short-term learning effect of ChatGPT on pharmacy students' learning

Abstract

Introduction

Students in pharmacy are positive towards integrating artificial intelligence and ChatGPT into their practice. The aim of this study was to investigate the direct short-term learning effect of using Chat GPT by pharmacy students.

Methods

This was an experimental randomized study. Students were allocated into two groups; the intervention group (n = 15) used all study tools and ChatGPT, while the control group (n = 16) used all study tools, except ChatGPT. Differences between groups was measured by how well they performed on a knowledge test before and after a short study period.

Results

No significant difference was found between the intervention and control groups in level of competence in the pretest score (p = 0.28). There was also no significant effect of using ChatGPT, with a mean adjusted difference of 0.5 points on a 12-point scale. However there was a trend towards a higher proportion of ChatGPT participants having a large (at least four point) increase in score (4 out of 15) vs control group (1 out of 16).

Conclusion

There is a potential for positive effects of ChatGPT on learning outcomes in pharmacy students, however the current study was underpowered to measure a statistically significant effect of ChatGPT on short term learning.