DETECTing the anomalies: Exploring implications of qualitative research in identifying AI-generated text for AI-assisted composition instruction

Abstract

This study inspires new pedagogical practices with evolving technological innovations. One example of such innovation is the emergence of artificial intelligence (AI) in education. The potential impact of generative AI, such as ChatGPT, on composition education has caused concerns among educators due to its human-like writing capabilities. However, there is no escape from ChatGPT-generated text, which is influenced by prompt engineering. Such engineering can lead to underlying issues in content, prompting a pedagogical opportunity for understanding human-written and AI-generated texts. Since, to this date, there is no single reliable source for identifying AI-generated text, this study introduces a pedagogical approach, DETECT, with two major goals: (1) explore the nuances that differentiate human expression from the algorithmic patterns and tendencies of generative AI writing and (2) inspire ways to integrate generative AI in composition instruction in a post-plagiarism era. Using exploratory practice research, this article examines DETECT in composition instruction of 32 students during Fall 2023 and Spring 2024. The findings showed that using DETECT improved students’ confidence in analyzing human-written and AI-generated texts, which enhanced their recognition and appreciation of their own writing voice. The study concludes with pedagogical implications for the possibilities of generative AI in writing instruction.