Graduate instructors navigating the AI frontier: The role of ChatGPT in higher education

Abstract

This research study explores the use of artificial intelligence (AI) in undergraduate assessments, specifically focusing on the ability of graduate teaching assistants (GTAs) to identify AI-generated assessments and the performance of ChatGPT, an AI model, in producing high-quality work. The study examines four guiding research questions and hypotheses related to the accuracy of GTA identification, the achievement of AI-generated work compared to student marks, the impact of GTA characteristics on identification accuracy, and the variation in identification and assessment across different subject areas. The study incorporates ten AI-generated assessments across seven classes taught by five GTAs. The findings reveal that ChatGPT consistently excelled the average student in all classes receiving 10 scores of A or higher out of 11 and receiving the top mark in 8 of the ten classes. GTAs accurately identified 50 % of the AI-generated assessments, with results suggesting a potential connection between class size and GTA accuracy in identifying AI-generated work. GTAs with prior experience and familiarity with ChatGPT demonstrated higher accuracy in identifying AI-generated assessments. However, further research is needed to explore this comprehensively. This study also reviews the effectiveness of TurnItin's new AI detector, highlighting an accuracy of 92 % across the ten assessments. The study highlights the adaptability of ChatGPT across different subject areas and assessment types, producing assessments that align with diverse educational contexts.

In conclusion, this research study contributes to understanding the effectiveness and adaptability of AI in undergraduate assessments. It underscores the need to further explore and develop AI technologies in education.