Shan Li, Xiaoshan Huang, Tingting Wang, Juan Zheng, Susanne P. Lajoie
{"title":"Using text mining and machine learning to predict reasoning activities from think-aloud transcripts in computer assisted learning","authors":"Shan Li, Xiaoshan Huang, Tingting Wang, Juan Zheng, Susanne P. Lajoie","doi":"10.1007/s12528-024-09404-6","DOIUrl":null,"url":null,"abstract":"<p>Coding think-aloud transcripts is time-consuming and labor-intensive. In this study, we examined the feasibility of predicting students’ reasoning activities based on their think-aloud transcripts by leveraging the affordances of text mining and machine learning techniques. We collected the think-aloud data of 34 medical students as they diagnosed virtual patients in an intelligent tutoring system. The think-aloud data were transcribed and segmented into 2,792 meaningful units. We used a text mining tool to analyze the linguistic features of think-aloud segments. Meanwhile, we manually coded the think-aloud segments using a medical reasoning coding scheme. We then trained eight types of supervised machine learning algorithms to predict reasoning activities based on the linguistic features of students’ think-aloud transcripts. We further investigated if the performance of prediction models differed between high and low performers. The results suggested that students’ reasoning activities could be predicted relatively accurately by the linguistic features of their think-aloud transcripts. Moreover, training the predictive models using the data instances of either high or low performers did not lower the models’ performance. This study has significant methodological and practical implications regarding the automatic analysis of think-aloud protocols and real-time assessment of students’ reasoning activities.</p>","PeriodicalId":15404,"journal":{"name":"Journal of Computing in Higher Education","volume":"41 1","pages":""},"PeriodicalIF":4.5000,"publicationDate":"2024-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Computing in Higher Education","FirstCategoryId":"95","ListUrlMain":"https://doi.org/10.1007/s12528-024-09404-6","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
引用次数: 0
Abstract
Coding think-aloud transcripts is time-consuming and labor-intensive. In this study, we examined the feasibility of predicting students’ reasoning activities based on their think-aloud transcripts by leveraging the affordances of text mining and machine learning techniques. We collected the think-aloud data of 34 medical students as they diagnosed virtual patients in an intelligent tutoring system. The think-aloud data were transcribed and segmented into 2,792 meaningful units. We used a text mining tool to analyze the linguistic features of think-aloud segments. Meanwhile, we manually coded the think-aloud segments using a medical reasoning coding scheme. We then trained eight types of supervised machine learning algorithms to predict reasoning activities based on the linguistic features of students’ think-aloud transcripts. We further investigated if the performance of prediction models differed between high and low performers. The results suggested that students’ reasoning activities could be predicted relatively accurately by the linguistic features of their think-aloud transcripts. Moreover, training the predictive models using the data instances of either high or low performers did not lower the models’ performance. This study has significant methodological and practical implications regarding the automatic analysis of think-aloud protocols and real-time assessment of students’ reasoning activities.
期刊介绍:
Journal of Computing in Higher Education (JCHE) contributes to our understanding of the design, development, and implementation of instructional processes and technologies in higher education. JCHE publishes original research, literature reviews, implementation and evaluation studies, and theoretical, conceptual, and policy papers that provide perspectives on instructional technology’s role in improving access, affordability, and outcomes of postsecondary education. Priority is given to well-documented original papers that demonstrate a strong grounding in learning theory and/or rigorous educational research design.