Pub Date : 2025-01-27DOI: 10.1016/j.compedu.2025.105252
Carita Kiili, Kristian Kiili, Eija Räikkönen, Julie Coiro
This study tested the efficacy of explicit video-based instruction (4 x 45-min lessons) in promoting sixth graders' online credibility evaluation skills. It also investigated whether storified instruction enhanced learning more than non-storified instruction. Storified and non-storified interventions contained the same learning content, and the same components of explicit instruction informed their design. The only difference in the two formats was that storified instruction was framed as a detective school, and non-storified instruction did not include any storified elements. In storified instruction, students joined the detective school, taught by two fictitious detectives who appeared as human instructors in the videos. Storified elements were also integrated into other learning materials to provide students with a holistic learning experience. The study employed a quasi-experimental research design with three conditions: storified instruction (n = 115), non-storified instruction (n = 135), and control (n = 127) conditions. Learning of online credibility evaluation skills was assessed with a credibility evaluation task that measured students' skills to confirm more credible online texts, question less credible online texts, and justify their overall evaluations. In the post-test, both intervention conditions outperformed the control condition in questioning the less credible online texts and justifying their evaluations. Notably, storified and non-storified instructions similarly improved students’ online credibility evaluation skills. Examination of the intervention effects for each individual revealed that basic reading skills were not associated with the level of change in questioning less credible online texts and justifying credibility evaluations. This suggests that video-based explicit instruction has the potential to support different kinds of readers.
{"title":"Explicit video-based instruction enhanced students’ online credibility evaluation skills: Did storifying instruction matter?","authors":"Carita Kiili, Kristian Kiili, Eija Räikkönen, Julie Coiro","doi":"10.1016/j.compedu.2025.105252","DOIUrl":"https://doi.org/10.1016/j.compedu.2025.105252","url":null,"abstract":"This study tested the efficacy of explicit video-based instruction (4 x 45-min lessons) in promoting sixth graders' online credibility evaluation skills. It also investigated whether storified instruction enhanced learning more than non-storified instruction. Storified and non-storified interventions contained the same learning content, and the same components of explicit instruction informed their design. The only difference in the two formats was that storified instruction was framed as a detective school, and non-storified instruction did not include any storified elements. In storified instruction, students joined the detective school, taught by two fictitious detectives who appeared as human instructors in the videos. Storified elements were also integrated into other learning materials to provide students with a holistic learning experience. The study employed a quasi-experimental research design with three conditions: storified instruction (<ce:italic>n</ce:italic> = 115), non-storified instruction (<ce:italic>n</ce:italic> = 135), and control (<ce:italic>n</ce:italic> = 127) conditions. Learning of online credibility evaluation skills was assessed with a credibility evaluation task that measured students' skills to confirm more credible online texts, question less credible online texts, and justify their overall evaluations. In the post-test, both intervention conditions outperformed the control condition in questioning the less credible online texts and justifying their evaluations. Notably, storified and non-storified instructions similarly improved students’ online credibility evaluation skills. Examination of the intervention effects for each individual revealed that basic reading skills were not associated with the level of change in questioning less credible online texts and justifying credibility evaluations. This suggests that video-based explicit instruction has the potential to support different kinds of readers.","PeriodicalId":10568,"journal":{"name":"Computers & Education","volume":"60 1","pages":""},"PeriodicalIF":12.0,"publicationDate":"2025-01-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143072445","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Large language models built upon artificial intelligence (AI) hold great promises to innovate automatic short answer scoring (ASAS) - significantly alleviating educators’ workload in assessing student answers. However, ASAS systems on such basis have seen limited adoption in authentic teaching environments due to the models’ inability to explain the predictions they generate. To address this, we recruited 32 human graders to comparatively analyse the decision-making processes of human graders and AI-driven graders. Specifically, we exploited two types of data to holistically unveil the decision-making processes of human graders throughout grading, namely manual annotation of important words and gaze data of the human graders. The decision-making processes of AI-driven graders were revealed by important words extracted though eXplainable Artificial Intelligence technique and grading confidence reflected by the prediction probability distributions. We measured the alignment in their decision-making regarding their (i) estimated scoring difficulty, (ii) important text segments and (iii) crucial grammatical categories to enhance the transparency and trustworthiness of AI-driven graders. Subsequently, we conducted randomised control studies, presenting machine-extracted insights like important words and estimated scoring difficulty to scrutinise how they affected human grading. Our findings contribute new knowledge regarding the consistency between human and machine scoring and validates machine-extracted insights, such as important words and scoring difficulty, to be valuable in facilitating human grading, encouraging the adoption of ASAS systems and urging the potential collaboration between machine and human grading in pedagogical practices. However, we emphasised the significance of grasping question context and intricacy before leveraging such machine-extracted insights.
{"title":"Can AI support human grading? Examining machine attention and confidence in short answer scoring","authors":"Yuheng Li, Mladen Raković, Namrata Srivastava, Xinyu Li, Quanlong Guan, Dragan Gašević, Guanliang Chen","doi":"10.1016/j.compedu.2025.105244","DOIUrl":"https://doi.org/10.1016/j.compedu.2025.105244","url":null,"abstract":"Large language models built upon artificial intelligence (AI) hold great promises to innovate automatic short answer scoring (ASAS) - significantly alleviating educators’ workload in assessing student answers. However, ASAS systems on such basis have seen limited adoption in authentic teaching environments due to the models’ inability to explain the predictions they generate. To address this, we recruited 32 human graders to comparatively analyse the decision-making processes of human graders and AI-driven graders. Specifically, we exploited two types of data to holistically unveil the decision-making processes of human graders throughout grading, namely manual annotation of important words and gaze data of the human graders. The decision-making processes of AI-driven graders were revealed by important words extracted though eXplainable Artificial Intelligence technique and grading confidence reflected by the prediction probability distributions. We measured the alignment in their decision-making regarding their (i) estimated scoring difficulty, (ii) important text segments and (iii) crucial grammatical categories to enhance the transparency and trustworthiness of AI-driven graders. Subsequently, we conducted randomised control studies, presenting machine-extracted insights like important words and estimated scoring difficulty to scrutinise how they affected human grading. Our findings contribute new knowledge regarding the consistency between human and machine scoring and validates machine-extracted insights, such as important words and scoring difficulty, to be valuable in facilitating human grading, encouraging the adoption of ASAS systems and urging the potential collaboration between machine and human grading in pedagogical practices. However, we emphasised the significance of grasping question context and intricacy before leveraging such machine-extracted insights.","PeriodicalId":10568,"journal":{"name":"Computers & Education","volume":"34 1","pages":""},"PeriodicalIF":12.0,"publicationDate":"2025-01-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143027347","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-14DOI: 10.1016/j.compedu.2025.105249
Siu Cheung Kong, Ting Jun Lin, Yee Man Kayla Siu
Programming education has become essential in primary schools, yet students' perceptions vary significantly across different achievement levels. While teacher support is crucial for learning, little is known about how it influences students' attitudes and flow experience across different achievement groups in programming education. This study examined the interrelationships among perceived teacher support, students' attitudes, and flow experience in programming learning across different achievement levels. Data were collected through surveys from 549 students across 43 Hong Kong primary schools. Students were categorised into three groups based on their programming achievement levels: above-average, average, and below-average. A multi-group mediation analysis was conducted to examine the relationships between variables. The above-average group showed significantly more positive perceptions across all variables than other groups. Students’ attitudes mediated the relationship between perceived teacher support and flow experience differently across achievement groups. The findings highlight the importance of differentiated teacher support strategies across achievement tiers and suggest that fostering positive attitudes towards programming is crucial for all achievement levels, particularly through enhancing the perceived usefulness of programming.
{"title":"The role of perceived teacher support in students’ attitudes towards and flow experience in programming learning: A multi-group analysis of primary students","authors":"Siu Cheung Kong, Ting Jun Lin, Yee Man Kayla Siu","doi":"10.1016/j.compedu.2025.105249","DOIUrl":"https://doi.org/10.1016/j.compedu.2025.105249","url":null,"abstract":"Programming education has become essential in primary schools, yet students' perceptions vary significantly across different achievement levels. While teacher support is crucial for learning, little is known about how it influences students' attitudes and flow experience across different achievement groups in programming education. This study examined the interrelationships among perceived teacher support, students' attitudes, and flow experience in programming learning across different achievement levels. Data were collected through surveys from 549 students across 43 Hong Kong primary schools. Students were categorised into three groups based on their programming achievement levels: above-average, average, and below-average. A multi-group mediation analysis was conducted to examine the relationships between variables. The above-average group showed significantly more positive perceptions across all variables than other groups. Students’ attitudes mediated the relationship between perceived teacher support and flow experience differently across achievement groups. The findings highlight the importance of differentiated teacher support strategies across achievement tiers and suggest that fostering positive attitudes towards programming is crucial for all achievement levels, particularly through enhancing the perceived usefulness of programming.","PeriodicalId":10568,"journal":{"name":"Computers & Education","volume":"30 1","pages":""},"PeriodicalIF":12.0,"publicationDate":"2025-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142987973","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-13DOI: 10.1016/j.compedu.2025.105248
Fangzhou Jin, Lanfang Sun, Yunqiu Pan, Chin-Hsi Lin
This research employed metaphor analysis to explore 277 postgraduate students' perceptions of the role of generative artificial intelligence (GenAI) in academic writing. All participants were international students, from a total of 14 countries and regions, studying in the United Kingdom. Data collection was carried out in two phases. The first was a survey comprising demographic and metaphor-related questions, and the second involved metaphor checking, in which participants provided screenshots of their interactions with GenAI. The data, which were analyzed both qualitatively and quantitatively, yielded 53 unique metaphors for the concept of GenAI in academic writing. We divided these into four conceptual categories in what we term the 4T Pyramid Model: Technical Support (representative metaphor: high-heeled shoes), Text Development (compass), Transformative Potential (Spider-Man), and Threat (drug). The respondents' academic disciplines influenced their perceptions of GenAI, but overall, the results suggest that most viewed it as transformative, i.e., more than just a writing tool. This study's innovative methodology integrating metaphor analysis with real user interactions offers a framework, aligned with Bloom's Taxonomy, that reveals the multi-level benefits and potential risks of GenAI. It also provides actionable insights for AI literacy education, including strategies for effective prompt design.
{"title":"High heels, compass, spider-man, or drug? Metaphor analysis of generative artificial intelligence in academic writing","authors":"Fangzhou Jin, Lanfang Sun, Yunqiu Pan, Chin-Hsi Lin","doi":"10.1016/j.compedu.2025.105248","DOIUrl":"https://doi.org/10.1016/j.compedu.2025.105248","url":null,"abstract":"This research employed metaphor analysis to explore 277 postgraduate students' perceptions of the role of generative artificial intelligence (GenAI) in academic writing. All participants were international students, from a total of 14 countries and regions, studying in the United Kingdom. Data collection was carried out in two phases. The first was a survey comprising demographic and metaphor-related questions, and the second involved metaphor checking, in which participants provided screenshots of their interactions with GenAI. The data, which were analyzed both qualitatively and quantitatively, yielded 53 unique metaphors for the concept of GenAI in academic writing. We divided these into four conceptual categories in what we term the 4T Pyramid Model: Technical Support (representative metaphor: high-heeled shoes), Text Development (compass), Transformative Potential (Spider-Man), and Threat (drug). The respondents' academic disciplines influenced their perceptions of GenAI, but overall, the results suggest that most viewed it as transformative, i.e., more than just a writing tool. This study's innovative methodology integrating metaphor analysis with real user interactions offers a framework, aligned with Bloom's Taxonomy, that reveals the multi-level benefits and potential risks of GenAI. It also provides actionable insights for AI literacy education, including strategies for effective prompt design.","PeriodicalId":10568,"journal":{"name":"Computers & Education","volume":"13 1","pages":""},"PeriodicalIF":12.0,"publicationDate":"2025-01-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143027365","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This study maps the evolution and state of Computer-Supported Collaborative Learning (CSCL) research, analyzing 6388 documents published between 1990 and 2022. The findings highlight the sustained engagement of a core group of scholars and the field's geographic concentration in Western countries, particularly the USA and Europe. While the field remains productive and diverse, recent trends reflect a growing emphasis on integrating emerging technologies such as learning analytics, augmented and virtual reality, and artificial intelligence (AI) into collaborative learning contexts. The study uncovers a tension within CSCL between its epistemological and methodological diversity and the need for theoretical coherence. This diversity has allowed the field to adapt and innovate but raises concerns about fragmentation and the risk of losing a unified identity. For example, while scripting remains a foundational topic, debates persist on balancing instructional guidance with learner agency to foster productive collaboration. Looking ahead, the study underscores the need for CSCL to develop integrative theoretical frameworks that bridge its rich historical foundations with the challenges posed by large-scale, distributed, and technology-mediated collaboration. Addressing these challenges, such as aligning AI innovations with existing theories and ensuring cumulative knowledge-building, will be critical for the field's ability to sustain its relevance and influence in understanding collaborative learning in complex educational environments.
{"title":"A Topical Review of Research in Computer-Supported Collaborative Learning: Questions and Possibilities","authors":"Rogers Kaliisa, Sonsoles López-Pernas, Kamila Misiejuk, Crina Damşa, Márta Sobocinski, Sanna Järvelä, Mohammed Saqr","doi":"10.1016/j.compedu.2025.105246","DOIUrl":"https://doi.org/10.1016/j.compedu.2025.105246","url":null,"abstract":"This study maps the evolution and state of Computer-Supported Collaborative Learning (CSCL) research, analyzing 6388 documents published between 1990 and 2022. The findings highlight the sustained engagement of a core group of scholars and the field's geographic concentration in Western countries, particularly the USA and Europe. While the field remains productive and diverse, recent trends reflect a growing emphasis on integrating emerging technologies such as learning analytics, augmented and virtual reality, and artificial intelligence (AI) into collaborative learning contexts. The study uncovers a tension within CSCL between its epistemological and methodological diversity and the need for theoretical coherence. This diversity has allowed the field to adapt and innovate but raises concerns about fragmentation and the risk of losing a unified identity. For example, while scripting remains a foundational topic, debates persist on balancing instructional guidance with learner agency to foster productive collaboration. Looking ahead, the study underscores the need for CSCL to develop integrative theoretical frameworks that bridge its rich historical foundations with the challenges posed by large-scale, distributed, and technology-mediated collaboration. Addressing these challenges, such as aligning AI innovations with existing theories and ensuring cumulative knowledge-building, will be critical for the field's ability to sustain its relevance and influence in understanding collaborative learning in complex educational environments.","PeriodicalId":10568,"journal":{"name":"Computers & Education","volume":"44 1","pages":""},"PeriodicalIF":12.0,"publicationDate":"2025-01-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142988417","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-13DOI: 10.1016/j.compedu.2025.105250
Chan Wang, Hongbiao Yin
The rapid development of generative AI (GenAI) has sparked increasing interest in its potential impact on student learning. Drawing on cognitive appraisal theory and AI device use acceptance (AIDUA) model, this study aims to investigate the complex dynamics of GenAI adoption among university undergraduates. Especially, our study examined how social, personal, and technological factors (social influence, hedonic motivation, and anthropomorphism) contributed to their cognitive appraisal and emotional experience with GenAI, analyzed how these cognitive-emotional responses affected their GenAI-powered learning productivity, and explored how this influential mechanism differs across various groups. We surveyed 2,026 Chinese undergraduates who had used GenAI in their studies. The findings indicated that the influence from others on using GenAI and hedonic motivation toward using GenAI were positively related to undergraduates’ appraisal of using GenAI as a challenge and negatively related to their view of it as a hindrance. Conversely, the anthropomorphic features of GenAI showed the opposite pattern. Undergraduates who viewed using GenAI as a challenge reported more positive emotions and demonstrated higher GenAI-powered learning productivity. Those who saw it as a hindrance experienced more negative emotions but showed no significant relationship with GenAI-powered learning productivity. Moreover, undergraduate students’ appraisal of using GenAI as a challenge or a hindrance and their emotional experiences played mediating roles in this influential process. Importantly, these relationships varied significantly across different student groups, highlighting the need for tailored approaches in GenAI integration. Our study provides insights for educators and policymakers on effectively integrating GenAI in higher education.
{"title":"How do Chinese undergraduates harness the potential of appraisal and emotions in generative AI-Powered learning? A multigroup analysis based on appraisal theory","authors":"Chan Wang, Hongbiao Yin","doi":"10.1016/j.compedu.2025.105250","DOIUrl":"https://doi.org/10.1016/j.compedu.2025.105250","url":null,"abstract":"The rapid development of generative AI (GenAI) has sparked increasing interest in its potential impact on student learning. Drawing on cognitive appraisal theory and AI device use acceptance (AIDUA) model, this study aims to investigate the complex dynamics of GenAI adoption among university undergraduates. Especially, our study examined how social, personal, and technological factors (social influence, hedonic motivation, and anthropomorphism) contributed to their cognitive appraisal and emotional experience with GenAI, analyzed how these cognitive-emotional responses affected their GenAI-powered learning productivity, and explored how this influential mechanism differs across various groups. We surveyed 2,026 Chinese undergraduates who had used GenAI in their studies. The findings indicated that the influence from others on using GenAI and hedonic motivation toward using GenAI were positively related to undergraduates’ appraisal of using GenAI as a challenge and negatively related to their view of it as a hindrance. Conversely, the anthropomorphic features of GenAI showed the opposite pattern. Undergraduates who viewed using GenAI as a challenge reported more positive emotions and demonstrated higher GenAI-powered learning productivity. Those who saw it as a hindrance experienced more negative emotions but showed no significant relationship with GenAI-powered learning productivity. Moreover, undergraduate students’ appraisal of using GenAI as a challenge or a hindrance and their emotional experiences played mediating roles in this influential process. Importantly, these relationships varied significantly across different student groups, highlighting the need for tailored approaches in GenAI integration. Our study provides insights for educators and policymakers on effectively integrating GenAI in higher education.","PeriodicalId":10568,"journal":{"name":"Computers & Education","volume":"37 1","pages":""},"PeriodicalIF":12.0,"publicationDate":"2025-01-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143027361","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-11DOI: 10.1016/j.compedu.2025.105247
Fitore Morina, Tim Fütterer, Nicolas Hübner, Steffen Zitzmann, Christian Fischer
Teachers' professional development is crucial for effective classroom practice. Due to its advantages, many teachers have participated in online professional development (OPD) in recent years. Numerous studies have investigated the participation and effect of first to 12th grade in-service teachers' OPD participation on the teacher, classroom practice, or student level. However, a comprehensive meta-analysis of these studies is missing. This meta-analysis summarizes findings from 102 quantitative studies with a pre-posttest design. The results indicate medium effects of OPD participation on the teacher level (Hedges' g = 0.71) and on the classroom level (Hedges' g = 0.55) and a small effect on the student level (Hedges' g = 0.19). We included Desimones' (2009) core features of effective PD participation as moderators. We found that the core feature of collective participation positively influenced the effect size of the classroom level. Furthermore, we found that studies that employed a control-group design reported significantly lower effect sizes on teacher-level outcomes than studies that used a within-subject design. On the student's level, studies with asynchronous OPD reported significantly smaller effect sizes than studies with a synchronous OPD format. Our results aim to provide research, policymakers, educational stakeholders, and teachers with the clarity that OPD is effective on several levels and should, therefore, be encouraged.
{"title":"Effects of online teacher professional development on teacher-, classroom-, and student-level outcomes: A meta-analysis","authors":"Fitore Morina, Tim Fütterer, Nicolas Hübner, Steffen Zitzmann, Christian Fischer","doi":"10.1016/j.compedu.2025.105247","DOIUrl":"https://doi.org/10.1016/j.compedu.2025.105247","url":null,"abstract":"Teachers' professional development is crucial for effective classroom practice. Due to its advantages, many teachers have participated in online professional development (OPD) in recent years. Numerous studies have investigated the participation and effect of first to 12th grade in-service teachers' OPD participation on the teacher, classroom practice, or student level. However, a comprehensive meta-analysis of these studies is missing. This meta-analysis summarizes findings from 102 quantitative studies with a pre-posttest design. The results indicate medium effects of OPD participation on the teacher level (Hedges' <ce:italic>g</ce:italic> = 0.71) and on the classroom level (Hedges' <ce:italic>g</ce:italic> = 0.55) and a small effect on the student level (Hedges' <ce:italic>g</ce:italic> = 0.19). We included Desimones' (2009) core features of effective PD participation as moderators. We found that the core feature of collective participation positively influenced the effect size of the classroom level. Furthermore, we found that studies that employed a control-group design reported significantly lower effect sizes on teacher-level outcomes than studies that used a within-subject design. On the student's level, studies with asynchronous OPD reported significantly smaller effect sizes than studies with a synchronous OPD format. Our results aim to provide research, policymakers, educational stakeholders, and teachers with the clarity that OPD is effective on several levels and should, therefore, be encouraged.","PeriodicalId":10568,"journal":{"name":"Computers & Education","volume":"58 1","pages":""},"PeriodicalIF":12.0,"publicationDate":"2025-01-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143027369","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-10DOI: 10.1016/j.compedu.2025.105245
Surina He, Ying Cui
In recent decades, log-based process data has been increasingly used in computer-based assessments to examine test-takers' response patterns and latent traits. This study provides a systematic review of the use of log-based process data in computer-based assessments. Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guideline, we identified 2548 publications, of which 330 were finally included in this study after careful screening and full-text review. The results of this study can assist researchers in better understanding: (1) what are the trends in using log-based process data in computer-based assessments, (2) which process indicators have been constructed from raw log files, (3) what latent constructs have been inferred from process indicators and at what inferential levels, and (4) what are the benefits, challenges, and future recommendations for using log-based process data. By examining these questions, we conclude that the use of log-based process data in computer-based assessment shows many potentials for enhancing the assessment. Therefore, more study using log-based process data in various fields is encouraged to better understand test-takers’ underlying response processes during assessments. Additionally, there is also a considerable demand for validating process indicators and the generalizability of findings.
{"title":"A systematic review of the use of log-based process data in computer-based assessments","authors":"Surina He, Ying Cui","doi":"10.1016/j.compedu.2025.105245","DOIUrl":"https://doi.org/10.1016/j.compedu.2025.105245","url":null,"abstract":"In recent decades, log-based process data has been increasingly used in computer-based assessments to examine test-takers' response patterns and latent traits. This study provides a systematic review of the use of log-based process data in computer-based assessments. Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guideline, we identified 2548 publications, of which 330 were finally included in this study after careful screening and full-text review. The results of this study can assist researchers in better understanding: (1) what are the trends in using log-based process data in computer-based assessments, (2) which process indicators have been constructed from raw log files, (3) what latent constructs have been inferred from process indicators and at what inferential levels, and (4) what are the benefits, challenges, and future recommendations for using log-based process data. By examining these questions, we conclude that the use of log-based process data in computer-based assessment shows many potentials for enhancing the assessment. Therefore, more study using log-based process data in various fields is encouraged to better understand test-takers’ underlying response processes during assessments. Additionally, there is also a considerable demand for validating process indicators and the generalizability of findings.","PeriodicalId":10568,"journal":{"name":"Computers & Education","volume":"30 1","pages":""},"PeriodicalIF":12.0,"publicationDate":"2025-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142987974","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-10DOI: 10.1016/j.compedu.2025.105233
T.S. Cristea, S. Heikkinen, C. Snijders, M. Saqr, U. Matzat, R. Conijn, A. Kleingeld
Proper self-regulating skills are essential in the new reality of digital learning in higher education. Research has shown that the trace data of students’ learning management system activity can identify various online learning tactics and strategies, but also their transitional dynamics, which are linked to academic performance. This study builds on this work by examining how learning tactics and strategies change within individual courses and how this relates to academic performance. A substantial dataset of 41 courses over two academic years at one university is analyzed. Employing Markov models on trace data, we identify prevalent tactics and strategies students use throughout courses. Our study examines shifts in strategy usage, comparing patterns between the initial and latter stages of the courses. The results reveal distinct clusters of learning strategies and their impact on academic achievement. Notably, deep learning strategies show significantly superior performance to surface approaches, especially when maintained over time. Moreover, students who consistently apply the same strategy score higher than those who are inconsistent. However, consistent surface learners score significantly lower than inconsistent learners. Underscoring such longitudinal trends could help interventions, aiding educators in targeting students with weaker strategies at specific times to boost their effectiveness and efficiency. This research contributes to a nuanced understanding of self-regulated learning behaviors in online educational contexts by showing the importance of dynamic transition of learning strategies for educators, instructional designers, and policymakers to enhance student learning experiences and outcomes.
{"title":"Dynamics of self-regulated learning: The effectiveness of students’ strategies across course periods","authors":"T.S. Cristea, S. Heikkinen, C. Snijders, M. Saqr, U. Matzat, R. Conijn, A. Kleingeld","doi":"10.1016/j.compedu.2025.105233","DOIUrl":"https://doi.org/10.1016/j.compedu.2025.105233","url":null,"abstract":"Proper self-regulating skills are essential in the new reality of digital learning in higher education. Research has shown that the trace data of students’ learning management system activity can identify various online learning tactics and strategies, but also their transitional dynamics, which are linked to academic performance. This study builds on this work by examining how learning tactics and strategies change within individual courses and how this relates to academic performance. A substantial dataset of 41 courses over two academic years at one university is analyzed. Employing Markov models on trace data, we identify prevalent tactics and strategies students use throughout courses. Our study examines shifts in strategy usage, comparing patterns between the initial and latter stages of the courses. The results reveal distinct clusters of learning strategies and their impact on academic achievement. Notably, deep learning strategies show significantly superior performance to surface approaches, especially when maintained over time. Moreover, students who consistently apply the same strategy score higher than those who are inconsistent. However, consistent surface learners score significantly lower than inconsistent learners. Underscoring such longitudinal trends could help interventions, aiding educators in targeting students with weaker strategies at specific times to boost their effectiveness and efficiency. This research contributes to a nuanced understanding of self-regulated learning behaviors in online educational contexts by showing the importance of dynamic transition of learning strategies for educators, instructional designers, and policymakers to enhance student learning experiences and outcomes.","PeriodicalId":10568,"journal":{"name":"Computers & Education","volume":"24 1","pages":""},"PeriodicalIF":12.0,"publicationDate":"2025-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142987975","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-09DOI: 10.1016/j.compedu.2025.105243
Ning Ren, Barry Lee Reynolds, Brian V. Rusk
Study-with-me (SWM) videos offer virtual companionship for learning. Users claim that SWM videos simulate physical companionship, motivate their learning with manageable pressure, and enhance their performance with minimal cost. Despite burgeoning search interests and millions of accumulated views, the effects of SWM videos on learning lack rigorous examination. The comparative advantages of SWM videos over other learning environments, based on different theories and related evidence, are contradictory. Addressing these gaps, we used the Basic Psychological Needs Theory (BPNT) to explain SWM video users' intrinsic motivation, perceived pressure, and performance with perceived autonomy, competence, and relatedness (RQ1). We also compared users’ learning with the SWM video to two learning environments: learning with a physical companion or alone (RQ2). We employed a within-subject design with 60 university students, inviting each to speed-read five English articles in all three learning environments. For RQ1, results showed that when learning with an SWM video, autonomy was positively associated with intrinsic motivation and negatively associated with perceived pressure. Competence had a negative relationship with reading time. Thus, we advise SWM video creators to support autonomy and competence, and users to select videos that fulfill these needs. For RQ2, participants perceived significantly lower pressure and had higher task accuracy with an SWM video than a physical companion despite preference or prior use of SWM videos. No statistical advantage was found over learning alone. Our findings support SWM videos as viable alternatives to physical companions but not superior to learning alone.
{"title":"Virtual companionship or flesh and blood: The effects of study-with-me (SWM) videos on learners’ intrinsic motivation, perceived pressure, and performance","authors":"Ning Ren, Barry Lee Reynolds, Brian V. Rusk","doi":"10.1016/j.compedu.2025.105243","DOIUrl":"https://doi.org/10.1016/j.compedu.2025.105243","url":null,"abstract":"Study-with-me (SWM) videos offer virtual companionship for learning. Users claim that SWM videos simulate physical companionship, motivate their learning with manageable pressure, and enhance their performance with minimal cost. Despite burgeoning search interests and millions of accumulated views, the effects of SWM videos on learning lack rigorous examination. The comparative advantages of SWM videos over other learning environments, based on different theories and related evidence, are contradictory. Addressing these gaps, we used the Basic Psychological Needs Theory (BPNT) to explain SWM video users' intrinsic motivation, perceived pressure, and performance with perceived autonomy, competence, and relatedness (RQ1). We also compared users’ learning with the SWM video to two learning environments: learning with a physical companion or alone (RQ2). We employed a within-subject design with 60 university students, inviting each to speed-read five English articles in all three learning environments. For RQ1, results showed that when learning with an SWM video, autonomy was positively associated with intrinsic motivation and negatively associated with perceived pressure. Competence had a negative relationship with reading time. Thus, we advise SWM video creators to support autonomy and competence, and users to select videos that fulfill these needs. For RQ2, participants perceived significantly lower pressure and had higher task accuracy with an SWM video than a physical companion despite preference or prior use of SWM videos. No statistical advantage was found over learning alone. Our findings support SWM videos as viable alternatives to physical companions but not superior to learning alone.","PeriodicalId":10568,"journal":{"name":"Computers & Education","volume":"23 1","pages":""},"PeriodicalIF":12.0,"publicationDate":"2025-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142987976","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：