A Comparison of Human-Written Versus AI-Generated Text in Discussions at Educational Settings: Investigating Features for ChatGPT, Gemini and BingAI

Abstract

Generative artificial intelligence (GenAI) models, such as ChatGPT, Gemini, and BingAI, have become integral to educational sciences, bringing about significant transformations in the education system and the processes of knowledge production. These advancements have facilitated new methods of teaching, learning, and information dissemination. However, the widespread adoption of these technologies raises serious concerns about academic ethics, content authenticity, and the potential for misuse in academic settings. This study aims to evaluate the linguistic features and differences between AI-generated and human-generated articles in educational contexts. By analysing various linguistic attributes such as singular word usage, sentence lengths, and the presence of repetitive or stereotypical phrases, the study identifies key distinctions between the two types of content. The findings indicate that human-generated articles exhibit higher average singular word usage and longer sentence lengths compared to AI-generated articles, suggesting a more complex and nuanced language structure in human writing. Furthermore, the study employs ensemble learning models, including Random Forest, Gradient Boosting, AdaBoost, Bagging, and Extra Trees, to classify and distinguish between AI-generated and human-generated texts. Among these, the Extra Trees model achieved the highest classification accuracy of 93%, highlighting its effectiveness in identifying AI-generated content. Additionally, experiments using the BERTurk model, a transformer-based language model, demonstrated a classification accuracy of 95%, particularly in distinguishing human-generated articles from those produced by Gemini. The results of this study have significant implications for the future of education, as they underscore the critical need for robust tools and methodologies to differentiate between human and AI-generated content.