Within the field of education technology, learning analytics has increased in popularity over the past decade. Researchers conduct experiments and develop software, building on each other’s work to create more intricate systems. In parallel, open science — which describes a set of practices to make research more open, transparent, and reproducible — has exploded in recent years, resulting in more open data, code, and materials for researchers to use. However, without prior knowledge of open science, many researchers do not make their datasets, code, and materials openly available, and those that are available are often difficult, if not impossible, to reproduce. The purpose of the current study was to take a close look at our field by examining previous papers within the proceedings of the International Conference on Learning Analytics and Knowledge, and document the rate of open science adoption (e.g., preregistration, open data), as well as how well available data and code could be reproduced. Specifically, we examined 133 research papers, allowing ourselves 15 minutes for each paper to identify open science practices and attempt to reproduce the results according to their provided specifications. Our results showed that less than half of the research adopted standard open science principles, with approximately 5% fully meeting some of the defined principles. Further, we were unable to reproduce any of the papers successfully in the given time period. We conclude by providing recommendations on how to improve the reproducibility of our research as a field moving forward. All openly accessible work can be found in an Open Science Foundation project1.
{"title":"How to Open Science: A Principle and Reproducibility Review of the Learning Analytics and Knowledge Conference","authors":"Aaron Haim, S. Shaw, N. Heffernan","doi":"10.1145/3576050.3576071","DOIUrl":"https://doi.org/10.1145/3576050.3576071","url":null,"abstract":"Within the field of education technology, learning analytics has increased in popularity over the past decade. Researchers conduct experiments and develop software, building on each other’s work to create more intricate systems. In parallel, open science — which describes a set of practices to make research more open, transparent, and reproducible — has exploded in recent years, resulting in more open data, code, and materials for researchers to use. However, without prior knowledge of open science, many researchers do not make their datasets, code, and materials openly available, and those that are available are often difficult, if not impossible, to reproduce. The purpose of the current study was to take a close look at our field by examining previous papers within the proceedings of the International Conference on Learning Analytics and Knowledge, and document the rate of open science adoption (e.g., preregistration, open data), as well as how well available data and code could be reproduced. Specifically, we examined 133 research papers, allowing ourselves 15 minutes for each paper to identify open science practices and attempt to reproduce the results according to their provided specifications. Our results showed that less than half of the research adopted standard open science principles, with approximately 5% fully meeting some of the defined principles. Further, we were unable to reproduce any of the papers successfully in the given time period. We conclude by providing recommendations on how to improve the reproducibility of our research as a field moving forward. All openly accessible work can be found in an Open Science Foundation project1.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"27 1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116790152","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
René F. Kizilcec, Olga Viberg, I. Jivet, Alejandra Martínez Monés, Alice Oh, Stefan Hrastinski, Chantal Mutimukwe, Maren Scheffel
The protection of students’ privacy in learning analytics (LA) applications is critical for cultivating trust and effective implementations of LA in educational environments around the world. However, students’ privacy concerns and how they may vary along demographic dimensions that historically influence these concerns have yet to be studied in higher education. Gender differences, in particular, are known to be associated with people's information privacy concerns, including in educational settings. Building on an empirically validated model and survey instrument for student privacy concerns, their antecedents and their behavioral outcomes, we investigate the presence of gender differences in students’ privacy concerns about LA. We conducted a survey study of students in higher education across five countries (N = 762): Germany, South Korea, Spain, Sweden and the United States. Using multiple regression analysis, across all five countries, we find that female students have stronger trusting beliefs and they are more inclined to engage in self-disclosure behaviors compared to male students. However, at the country level, these gender differences are significant only in the German sample, for Bachelor's degree students, and for students between the ages of 18 and 24. Thus, national context, degree program, and age are important moderating factors for gender differences in student privacy concerns.
{"title":"The Role of Gender in Students’ Privacy Concerns about Learning Analytics: Evidence from five countries","authors":"René F. Kizilcec, Olga Viberg, I. Jivet, Alejandra Martínez Monés, Alice Oh, Stefan Hrastinski, Chantal Mutimukwe, Maren Scheffel","doi":"10.1145/3576050.3576142","DOIUrl":"https://doi.org/10.1145/3576050.3576142","url":null,"abstract":"The protection of students’ privacy in learning analytics (LA) applications is critical for cultivating trust and effective implementations of LA in educational environments around the world. However, students’ privacy concerns and how they may vary along demographic dimensions that historically influence these concerns have yet to be studied in higher education. Gender differences, in particular, are known to be associated with people's information privacy concerns, including in educational settings. Building on an empirically validated model and survey instrument for student privacy concerns, their antecedents and their behavioral outcomes, we investigate the presence of gender differences in students’ privacy concerns about LA. We conducted a survey study of students in higher education across five countries (N = 762): Germany, South Korea, Spain, Sweden and the United States. Using multiple regression analysis, across all five countries, we find that female students have stronger trusting beliefs and they are more inclined to engage in self-disclosure behaviors compared to male students. However, at the country level, these gender differences are significant only in the German sample, for Bachelor's degree students, and for students between the ages of 18 and 24. Thus, national context, degree program, and age are important moderating factors for gender differences in student privacy concerns.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123328487","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Andrew Zamecnik, Srécko Joksimovíc, Vitomir Kovanovíc, G. Grossmann, Djazia Ladjal, Abelardo Pardo
Industry supervisors play a pivotal role in ongoing learner support and guidance within a work-integrated learning context. Effective provisional feedback from industry supervisors in work-integrated learning environments is essential for increasing a team’s metacognitive awareness and ability to evaluate their performance. However, research that examines the usefulness and type of feedback from industry supervisors for teams remains limited. In this study, we investigate the quality of provisional feedback by comparing the teams’ helpfulness rating of the feedback from two types of industry supervisors (i.e., clients and mentors), based on the feedback type (task, process, regulatory and self-level oriented) using learning analytics. The results show that teams rated the perceived helpfulness scores of clients and mentors as very useful, with mentors providing slightly more helpful feedback. We also found that mentors provide more co-occurrences of feedback classifications than clients. The overall results show that teams perceive mentor feedback as more helpful than clients and that the mentor targets feedback that is more beneficial to the teams learning than the clients. Our findings can aid in developing guidelines that aim to validate and improve existing or new feedback quality frameworks by leveraging backward evaluation data.
{"title":"Exploring the Feedback Provision of Mentors and Clients for Teams in Work-Integrated Learning Environments","authors":"Andrew Zamecnik, Srécko Joksimovíc, Vitomir Kovanovíc, G. Grossmann, Djazia Ladjal, Abelardo Pardo","doi":"10.1145/3576050.3576069","DOIUrl":"https://doi.org/10.1145/3576050.3576069","url":null,"abstract":"Industry supervisors play a pivotal role in ongoing learner support and guidance within a work-integrated learning context. Effective provisional feedback from industry supervisors in work-integrated learning environments is essential for increasing a team’s metacognitive awareness and ability to evaluate their performance. However, research that examines the usefulness and type of feedback from industry supervisors for teams remains limited. In this study, we investigate the quality of provisional feedback by comparing the teams’ helpfulness rating of the feedback from two types of industry supervisors (i.e., clients and mentors), based on the feedback type (task, process, regulatory and self-level oriented) using learning analytics. The results show that teams rated the perceived helpfulness scores of clients and mentors as very useful, with mentors providing slightly more helpful feedback. We also found that mentors provide more co-occurrences of feedback classifications than clients. The overall results show that teams perceive mentor feedback as more helpful than clients and that the mentor targets feedback that is more beneficial to the teams learning than the clients. Our findings can aid in developing guidelines that aim to validate and improve existing or new feedback quality frameworks by leveraging backward evaluation data.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"55 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129751618","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Computing education researchers often study the impact of online help-seeking behaviors that occur across multiple online resources in isolation. Such separation fails to capture the interconnected nature of online help-seeking behaviors that occur across multiple online resources and its affect on course grades. This is particularly important for programming education, which arguably has more online resources to seek help from other people (e.g., computer-mediated conversations) than other majors. Using data from an introductory programming course (CS1) at a large US university, we found that students (n=301) sought help in multiple computer-mediated conversations, both Q&A forum and online office hours (OHQ), differently. Results showed the more prior knowledge about programming students had, the more they sought help in the Q&A compared to students with less prior knowledge. In general, higher-performing students sought help online in the Q&A more than the lower-performing groups on all the homework assignments, but not for the OHQ. By better understanding how students seek help online across multiple modalities of computer-mediated conversations and the relationship between help-seeking and grades, we can re-design online resources that best support all students in introductory programming courses at scale.
{"title":"Online help-seeking occurring in multiple computer-mediated conversations affects grades in an introductory programming course","authors":"Elizabeth B. Cloude, R. Baker, Eric Fouh","doi":"10.1145/3576050.3576106","DOIUrl":"https://doi.org/10.1145/3576050.3576106","url":null,"abstract":"Computing education researchers often study the impact of online help-seeking behaviors that occur across multiple online resources in isolation. Such separation fails to capture the interconnected nature of online help-seeking behaviors that occur across multiple online resources and its affect on course grades. This is particularly important for programming education, which arguably has more online resources to seek help from other people (e.g., computer-mediated conversations) than other majors. Using data from an introductory programming course (CS1) at a large US university, we found that students (n=301) sought help in multiple computer-mediated conversations, both Q&A forum and online office hours (OHQ), differently. Results showed the more prior knowledge about programming students had, the more they sought help in the Q&A compared to students with less prior knowledge. In general, higher-performing students sought help online in the Q&A more than the lower-performing groups on all the homework assignments, but not for the OHQ. By better understanding how students seek help online across multiple modalities of computer-mediated conversations and the relationship between help-seeking and grades, we can re-design online resources that best support all students in introductory programming courses at scale.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132066936","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Amber J. Dood, K. Das, Zhen Qian, S. Finkenstaedt-Quinn, A. Gere, G. Shultz
Writing-to-Learn (WTL) is an evidence-based instructional practice which can help students construct knowledge across many disciplines. Though it is known to be an effective practice, many instructors do not implement WTL in their courses due to time constraints and inability to provide students with personalized feedback. One way to address this is to include peer review, which allows students to receive feedback on their writing and benefits them as they act as reviewers. To further ease the implementation of peer review and provide instructors with feedback on their students’ work, we labeled students’ peer review comments across courses for type of feedback provided and trained a machine learning model to automatically classify those comments, improving upon models reported in prior work. We then created a dashboard which takes students’ comments, labels the comments using the model, and allows instructors to filter through their students’ comments based on how the model labels the comments. This dashboard can be used by instructors to monitor the peer review collaborations occurring in their courses. The dashboard will allow them to efficiently use information provided by peers to identify common issues in their students’ writing and better evaluate the quality of their students’ peer review.
{"title":"A Dashboard to Provide Instructors with Automated Feedback on Students’ Peer Review Comments","authors":"Amber J. Dood, K. Das, Zhen Qian, S. Finkenstaedt-Quinn, A. Gere, G. Shultz","doi":"10.1145/3576050.3576087","DOIUrl":"https://doi.org/10.1145/3576050.3576087","url":null,"abstract":"Writing-to-Learn (WTL) is an evidence-based instructional practice which can help students construct knowledge across many disciplines. Though it is known to be an effective practice, many instructors do not implement WTL in their courses due to time constraints and inability to provide students with personalized feedback. One way to address this is to include peer review, which allows students to receive feedback on their writing and benefits them as they act as reviewers. To further ease the implementation of peer review and provide instructors with feedback on their students’ work, we labeled students’ peer review comments across courses for type of feedback provided and trained a machine learning model to automatically classify those comments, improving upon models reported in prior work. We then created a dashboard which takes students’ comments, labels the comments using the model, and allows instructors to filter through their students’ comments based on how the model labels the comments. This dashboard can be used by instructors to monitor the peer review collaborations occurring in their courses. The dashboard will allow them to efficiently use information provided by peers to identify common issues in their students’ writing and better evaluate the quality of their students’ peer review.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"52 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129777538","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ashish Gurung, Sami Baral, Kirk P. Vanacore, Andrew A. Mcreynolds, Hilary Kreisberg, Anthony F. Botelho, S. Shaw, Neil T. Hefferna
Prior work analyzing tutoring sessions provided evidence that highly effective tutors, through their interaction with students and their experience, can perceptively recognize incorrect processes or “bugs” when students incorrectly answer problems. Researchers have studied these tutoring interactions examining instructional approaches to address incorrect processes and observed that the format of the feedback can influence learning outcomes. In this work, we recognize the incorrect answers caused by these buggy processes as Common Wrong Answers (CWAs). We examine the ability of teachers and instructional designers to identify CWAs proactively. As teachers and instructional designers deeply understand the common approaches and mistakes students make when solving mathematical problems, we examine the feasibility of proactively identifying CWAs and generating Common Wrong Answer Feedback (CWAFs) as a formative feedback intervention for addressing student learning needs. As such, we analyze CWAFs in three sets of analyses. We first report on the accuracy of the CWAs predicted by the teachers and instructional designers on the problems across two activities. We then measure the effectiveness of the CWAFs using an intent-to-treat analysis. Finally, we explore the existence of personalization effects of the CWAFs for the students working on the two mathematics activities.
{"title":"Identification, Exploration, and Remediation: Can Teachers Predict Common Wrong Answers?","authors":"Ashish Gurung, Sami Baral, Kirk P. Vanacore, Andrew A. Mcreynolds, Hilary Kreisberg, Anthony F. Botelho, S. Shaw, Neil T. Hefferna","doi":"10.1145/3576050.3576109","DOIUrl":"https://doi.org/10.1145/3576050.3576109","url":null,"abstract":"Prior work analyzing tutoring sessions provided evidence that highly effective tutors, through their interaction with students and their experience, can perceptively recognize incorrect processes or “bugs” when students incorrectly answer problems. Researchers have studied these tutoring interactions examining instructional approaches to address incorrect processes and observed that the format of the feedback can influence learning outcomes. In this work, we recognize the incorrect answers caused by these buggy processes as Common Wrong Answers (CWAs). We examine the ability of teachers and instructional designers to identify CWAs proactively. As teachers and instructional designers deeply understand the common approaches and mistakes students make when solving mathematical problems, we examine the feasibility of proactively identifying CWAs and generating Common Wrong Answer Feedback (CWAFs) as a formative feedback intervention for addressing student learning needs. As such, we analyze CWAFs in three sets of analyses. We first report on the accuracy of the CWAs predicted by the teachers and instructional designers on the problems across two activities. We then measure the effectiveness of the CWAFs using an intent-to-treat analysis. Finally, we explore the existence of personalization effects of the CWAFs for the students working on the two mathematics activities.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"67 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124521558","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Reading on digital devices has become more commonplace, while it often poses challenges to learners’ attention. In this study, we hypothesized that allowing learners to reflect on their reading phases with an empathic social robot companion might enhance learners’ attention in e-reading. To verify our assumption, we collected a novel dataset (SKEP) in an e-reading setting with social robot support. It contains 25 multimodal features from various sensors and logged data that are direct and indirect cues of attention. Based on the SKEP dataset, we comprehensively compared the difference between HRI-based (treatment) and GUI-based (control) feedback and obtained insights for intervention design. Based on the human annotation of the nearly 40 hours of video data streams from 60 subjects, we developed a machine learning model to capture attention-regulation behaviors in e-reading. We exploited a two-stage framework to recognize learners’ observable self-regulatory behaviors and conducted attention analysis. The proposed system showed a promising performance with high prediction results of e-reading with HRI, such as 72.97% accuracy in recognizing attention regulation behaviors, 74.29% accuracy in predicting knowledge gain, 75.00% for perceived interaction experience, and 75.00% for perceived social presence. We believe our work can inspire the future design of HRI-based e-reading and its analysis.
{"title":"Can We Empower Attentive E-reading with a Social Robot? An Introductory Study with a Novel Multimodal Dataset and Deep Learning Approaches","authors":"Yoon Lee, M. Specht","doi":"10.1145/3576050.3576122","DOIUrl":"https://doi.org/10.1145/3576050.3576122","url":null,"abstract":"Reading on digital devices has become more commonplace, while it often poses challenges to learners’ attention. In this study, we hypothesized that allowing learners to reflect on their reading phases with an empathic social robot companion might enhance learners’ attention in e-reading. To verify our assumption, we collected a novel dataset (SKEP) in an e-reading setting with social robot support. It contains 25 multimodal features from various sensors and logged data that are direct and indirect cues of attention. Based on the SKEP dataset, we comprehensively compared the difference between HRI-based (treatment) and GUI-based (control) feedback and obtained insights for intervention design. Based on the human annotation of the nearly 40 hours of video data streams from 60 subjects, we developed a machine learning model to capture attention-regulation behaviors in e-reading. We exploited a two-stage framework to recognize learners’ observable self-regulatory behaviors and conducted attention analysis. The proposed system showed a promising performance with high prediction results of e-reading with HRI, such as 72.97% accuracy in recognizing attention regulation behaviors, 74.29% accuracy in predicting knowledge gain, 75.00% for perceived interaction experience, and 75.00% for perceived social presence. We believe our work can inspire the future design of HRI-based e-reading and its analysis.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"218 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132274817","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
H. Almoubayyed, Stephen E. Fancsali, Steven Ritter
Adaptive educational software is likely to better support broader and more diverse sets of learners by considering more comprehensive views (or models) of such learners. For example, recent work proposed making inferences about “non-math” factors like reading comprehension while students used adaptive software for mathematics to better support and adapt to learners. We build on this proposed approach to more comprehensive learning modeling by providing an empirical basis for making inferences about students’ reading ability from their performance on activities in adaptive software for mathematics. We lay out an approach to predicting middle school students’ reading ability using their performance on activities within Carnegie Learning’s MATHia, a widely used intelligent tutoring system for mathematics. We focus on how performance in an early, introductory activity as an especially powerful place to consider instruction-embedded assessment of non-math factors like reading comprehension to guide adaptation based on factors like reading ability. We close by discussing opportunities to extend this work by focusing on particular knowledge components or skills tracked by MATHia that may provide important “levers” for driving adaptation based on students’ reading ability while they learn and practice mathematics.
{"title":"Instruction-Embedded Assessment for Reading Ability in Adaptive Mathematics Software","authors":"H. Almoubayyed, Stephen E. Fancsali, Steven Ritter","doi":"10.1145/3576050.3576105","DOIUrl":"https://doi.org/10.1145/3576050.3576105","url":null,"abstract":"Adaptive educational software is likely to better support broader and more diverse sets of learners by considering more comprehensive views (or models) of such learners. For example, recent work proposed making inferences about “non-math” factors like reading comprehension while students used adaptive software for mathematics to better support and adapt to learners. We build on this proposed approach to more comprehensive learning modeling by providing an empirical basis for making inferences about students’ reading ability from their performance on activities in adaptive software for mathematics. We lay out an approach to predicting middle school students’ reading ability using their performance on activities within Carnegie Learning’s MATHia, a widely used intelligent tutoring system for mathematics. We focus on how performance in an early, introductory activity as an especially powerful place to consider instruction-embedded assessment of non-math factors like reading comprehension to guide adaptation based on factors like reading ability. We close by discussing opportunities to extend this work by focusing on particular knowledge components or skills tracked by MATHia that may provide important “levers” for driving adaptation based on students’ reading ability while they learn and practice mathematics.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"55 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127122419","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Daniel Biedermann, J. Schneider, George-Petru Ciordas-Hertel, B. Eichmann, Carolin Hahnel, Frank Goldhammer, H. Drachsler
When reading long and complex texts, students may disengage and miss out on relevant content. In order to prevent disengaged behavior or to counteract it by means of an intervention, it is ideally detected an early stage. In this paper, we present a method for early disengagement detection that relies only on the classification of scrolling data. The presented method transforms scrolling data into a time series representation, where each point of the series represents the vertical position of the viewport in the text document. This time series representation is then classified using time series classification algorithms. We evaluated the method on a dataset of 565 university students reading eight different texts. We compared the algorithm performance with different time series lengths, data sampling strategies, the texts that make up the training data, and classification algorithms. The method can classify disengagement early with up to 70% accuracy. However, we also observe differences in the performance depending on which of the texts are included in the training dataset. We discuss our results and propose several possible improvements to enhance the method.
{"title":"Detecting the Disengaged Reader - Using Scrolling Data to Predict Disengagement during Reading","authors":"Daniel Biedermann, J. Schneider, George-Petru Ciordas-Hertel, B. Eichmann, Carolin Hahnel, Frank Goldhammer, H. Drachsler","doi":"10.1145/3576050.3576078","DOIUrl":"https://doi.org/10.1145/3576050.3576078","url":null,"abstract":"When reading long and complex texts, students may disengage and miss out on relevant content. In order to prevent disengaged behavior or to counteract it by means of an intervention, it is ideally detected an early stage. In this paper, we present a method for early disengagement detection that relies only on the classification of scrolling data. The presented method transforms scrolling data into a time series representation, where each point of the series represents the vertical position of the viewport in the text document. This time series representation is then classified using time series classification algorithms. We evaluated the method on a dataset of 565 university students reading eight different texts. We compared the algorithm performance with different time series lengths, data sampling strategies, the texts that make up the training data, and classification algorithms. The method can classify disengagement early with up to 70% accuracy. However, we also observe differences in the performance depending on which of the texts are included in the training dataset. We discuss our results and propose several possible improvements to enhance the method.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127180900","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This study explored two aspects of temporality in students’ SRL behaviours to understand the dynamics of SRL phase transitions. In the first aspect, which refers to the temporal order of Self-regulated learning (SRL) phases, we characterized four types of SRL processes based on phase transitions and the cyclical nature of SRL. The SRL types were mapped into the kinds of iterative behaviours over SRL phases which correspond to the theorized self-regulatory behaviours of students at different levels of SRL skills. We found a significant association between SRL types and the assignment grades that suggests the higher achieved learning outcomes, i.e., programming skills demonstrated in the assignments, being associated with more advanced SRL processes. This study also focused on the second aspect of temporality, which refers to the instance of time. We revealed the temporal dynamics between SRL phase transitions by analyzing time profiles for transitions in each SRL process type. Next, we showed that a two-day interval is a threshold by which most students iteratively transition from adapting to enactment phases, which provides a suitable time to intervene if the transition is not observed.
{"title":"When to Intervene? Utilizing Two Facets of Temporality in Students’ SRL Processes in a Programming Course","authors":"Sina Nazeri, M. Hatala, Fatemeh Salehian Kia","doi":"10.1145/3576050.3576095","DOIUrl":"https://doi.org/10.1145/3576050.3576095","url":null,"abstract":"This study explored two aspects of temporality in students’ SRL behaviours to understand the dynamics of SRL phase transitions. In the first aspect, which refers to the temporal order of Self-regulated learning (SRL) phases, we characterized four types of SRL processes based on phase transitions and the cyclical nature of SRL. The SRL types were mapped into the kinds of iterative behaviours over SRL phases which correspond to the theorized self-regulatory behaviours of students at different levels of SRL skills. We found a significant association between SRL types and the assignment grades that suggests the higher achieved learning outcomes, i.e., programming skills demonstrated in the assignments, being associated with more advanced SRL processes. This study also focused on the second aspect of temporality, which refers to the instance of time. We revealed the temporal dynamics between SRL phase transitions by analyzing time profiles for transitions in each SRL process type. Next, we showed that a two-day interval is a threshold by which most students iteratively transition from adapting to enactment phases, which provides a suitable time to intervene if the transition is not observed.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115477687","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：