Harnessing generative AI in chemical engineering education: Implementation and evaluation of the large language model ChatGPT v3.5

Abstract

With the recent and rapid growth of the adoption of generative artificial intelligence (GAI), including the use of large language models (LLMs) there has been growing concern amongst higher education institutions regarding assessment, potential plagiarism, and ultimately a negative impact on student learning outcomes. However, GAI is likely to be a useful tool in future professional environments, including in many chemical engineering-related roles. It is, therefore, essential that students are equipped with the knowledge and skills to use GAI responsibly, ethically, and safely. This research adopts the IDEE (Identify desired outcomes, Determine level of automation, Ensure ethics, Evaluate effectiveness) framework to develop a chemical engineering lab session which is augmented by the use of LLMs. As part of the pre-lab work, Year 1 students were tasked with using ChatGPT v3.5 to derive a model which predicted the drainage profile of water from a tank. They then tested the validity of this model experimentally in a lab session and analysed the data obtained as part of the post-lab work. Pre- and post-lab surveys were conducted which revealed that students had limited prior experience with GAI but there was a general belief that it could be useful for future work. The post-lab survey showed that the vast majority of people believed that this exercise had helped them learn how to use LLMs, how to use it ethically, how to critique the output, and what some of its limitations were. Reflexive thematic analysis was applied to the qualitative data obtained in the same surveys. This revealed eight distinct themes, one of which showed that there was a strong awareness of the need for criticising the LLM output, of the potential pitfalls associated with its use, and concerns over the quality of the output. As such, this work provides not just a case study for the integration of LLMs, and GAI more broadly, into chemical engineering curricula, but also valuable insight into student perceptions regarding the use of this nascent technology more generally.