Within the field of education technology, learning analytics has increased in popularity over the past decade. Researchers conduct experiments and develop software, building on each other’s work to create more intricate systems. In parallel, open science — which describes a set of practices to make research more open, transparent, and reproducible — has exploded in recent years, resulting in more open data, code, and materials for researchers to use. However, without prior knowledge of open science, many researchers do not make their datasets, code, and materials openly available, and those that are available are often difficult, if not impossible, to reproduce. The purpose of the current study was to take a close look at our field by examining previous papers within the proceedings of the International Conference on Learning Analytics and Knowledge, and document the rate of open science adoption (e.g., preregistration, open data), as well as how well available data and code could be reproduced. Specifically, we examined 133 research papers, allowing ourselves 15 minutes for each paper to identify open science practices and attempt to reproduce the results according to their provided specifications. Our results showed that less than half of the research adopted standard open science principles, with approximately 5% fully meeting some of the defined principles. Further, we were unable to reproduce any of the papers successfully in the given time period. We conclude by providing recommendations on how to improve the reproducibility of our research as a field moving forward. All openly accessible work can be found in an Open Science Foundation project1.
{"title":"How to Open Science: A Principle and Reproducibility Review of the Learning Analytics and Knowledge Conference","authors":"Aaron Haim, S. Shaw, N. Heffernan","doi":"10.1145/3576050.3576071","DOIUrl":"https://doi.org/10.1145/3576050.3576071","url":null,"abstract":"Within the field of education technology, learning analytics has increased in popularity over the past decade. Researchers conduct experiments and develop software, building on each other’s work to create more intricate systems. In parallel, open science — which describes a set of practices to make research more open, transparent, and reproducible — has exploded in recent years, resulting in more open data, code, and materials for researchers to use. However, without prior knowledge of open science, many researchers do not make their datasets, code, and materials openly available, and those that are available are often difficult, if not impossible, to reproduce. The purpose of the current study was to take a close look at our field by examining previous papers within the proceedings of the International Conference on Learning Analytics and Knowledge, and document the rate of open science adoption (e.g., preregistration, open data), as well as how well available data and code could be reproduced. Specifically, we examined 133 research papers, allowing ourselves 15 minutes for each paper to identify open science practices and attempt to reproduce the results according to their provided specifications. Our results showed that less than half of the research adopted standard open science principles, with approximately 5% fully meeting some of the defined principles. Further, we were unable to reproduce any of the papers successfully in the given time period. We conclude by providing recommendations on how to improve the reproducibility of our research as a field moving forward. All openly accessible work can be found in an Open Science Foundation project1.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"27 1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116790152","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
René F. Kizilcec, Olga Viberg, I. Jivet, Alejandra Martínez Monés, Alice Oh, Stefan Hrastinski, Chantal Mutimukwe, Maren Scheffel
The protection of students’ privacy in learning analytics (LA) applications is critical for cultivating trust and effective implementations of LA in educational environments around the world. However, students’ privacy concerns and how they may vary along demographic dimensions that historically influence these concerns have yet to be studied in higher education. Gender differences, in particular, are known to be associated with people's information privacy concerns, including in educational settings. Building on an empirically validated model and survey instrument for student privacy concerns, their antecedents and their behavioral outcomes, we investigate the presence of gender differences in students’ privacy concerns about LA. We conducted a survey study of students in higher education across five countries (N = 762): Germany, South Korea, Spain, Sweden and the United States. Using multiple regression analysis, across all five countries, we find that female students have stronger trusting beliefs and they are more inclined to engage in self-disclosure behaviors compared to male students. However, at the country level, these gender differences are significant only in the German sample, for Bachelor's degree students, and for students between the ages of 18 and 24. Thus, national context, degree program, and age are important moderating factors for gender differences in student privacy concerns.
{"title":"The Role of Gender in Students’ Privacy Concerns about Learning Analytics: Evidence from five countries","authors":"René F. Kizilcec, Olga Viberg, I. Jivet, Alejandra Martínez Monés, Alice Oh, Stefan Hrastinski, Chantal Mutimukwe, Maren Scheffel","doi":"10.1145/3576050.3576142","DOIUrl":"https://doi.org/10.1145/3576050.3576142","url":null,"abstract":"The protection of students’ privacy in learning analytics (LA) applications is critical for cultivating trust and effective implementations of LA in educational environments around the world. However, students’ privacy concerns and how they may vary along demographic dimensions that historically influence these concerns have yet to be studied in higher education. Gender differences, in particular, are known to be associated with people's information privacy concerns, including in educational settings. Building on an empirically validated model and survey instrument for student privacy concerns, their antecedents and their behavioral outcomes, we investigate the presence of gender differences in students’ privacy concerns about LA. We conducted a survey study of students in higher education across five countries (N = 762): Germany, South Korea, Spain, Sweden and the United States. Using multiple regression analysis, across all five countries, we find that female students have stronger trusting beliefs and they are more inclined to engage in self-disclosure behaviors compared to male students. However, at the country level, these gender differences are significant only in the German sample, for Bachelor's degree students, and for students between the ages of 18 and 24. Thus, national context, degree program, and age are important moderating factors for gender differences in student privacy concerns.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123328487","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Andrew Zamecnik, Srécko Joksimovíc, Vitomir Kovanovíc, G. Grossmann, Djazia Ladjal, Abelardo Pardo
Industry supervisors play a pivotal role in ongoing learner support and guidance within a work-integrated learning context. Effective provisional feedback from industry supervisors in work-integrated learning environments is essential for increasing a team’s metacognitive awareness and ability to evaluate their performance. However, research that examines the usefulness and type of feedback from industry supervisors for teams remains limited. In this study, we investigate the quality of provisional feedback by comparing the teams’ helpfulness rating of the feedback from two types of industry supervisors (i.e., clients and mentors), based on the feedback type (task, process, regulatory and self-level oriented) using learning analytics. The results show that teams rated the perceived helpfulness scores of clients and mentors as very useful, with mentors providing slightly more helpful feedback. We also found that mentors provide more co-occurrences of feedback classifications than clients. The overall results show that teams perceive mentor feedback as more helpful than clients and that the mentor targets feedback that is more beneficial to the teams learning than the clients. Our findings can aid in developing guidelines that aim to validate and improve existing or new feedback quality frameworks by leveraging backward evaluation data.
{"title":"Exploring the Feedback Provision of Mentors and Clients for Teams in Work-Integrated Learning Environments","authors":"Andrew Zamecnik, Srécko Joksimovíc, Vitomir Kovanovíc, G. Grossmann, Djazia Ladjal, Abelardo Pardo","doi":"10.1145/3576050.3576069","DOIUrl":"https://doi.org/10.1145/3576050.3576069","url":null,"abstract":"Industry supervisors play a pivotal role in ongoing learner support and guidance within a work-integrated learning context. Effective provisional feedback from industry supervisors in work-integrated learning environments is essential for increasing a team’s metacognitive awareness and ability to evaluate their performance. However, research that examines the usefulness and type of feedback from industry supervisors for teams remains limited. In this study, we investigate the quality of provisional feedback by comparing the teams’ helpfulness rating of the feedback from two types of industry supervisors (i.e., clients and mentors), based on the feedback type (task, process, regulatory and self-level oriented) using learning analytics. The results show that teams rated the perceived helpfulness scores of clients and mentors as very useful, with mentors providing slightly more helpful feedback. We also found that mentors provide more co-occurrences of feedback classifications than clients. The overall results show that teams perceive mentor feedback as more helpful than clients and that the mentor targets feedback that is more beneficial to the teams learning than the clients. Our findings can aid in developing guidelines that aim to validate and improve existing or new feedback quality frameworks by leveraging backward evaluation data.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"55 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129751618","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Computing education researchers often study the impact of online help-seeking behaviors that occur across multiple online resources in isolation. Such separation fails to capture the interconnected nature of online help-seeking behaviors that occur across multiple online resources and its affect on course grades. This is particularly important for programming education, which arguably has more online resources to seek help from other people (e.g., computer-mediated conversations) than other majors. Using data from an introductory programming course (CS1) at a large US university, we found that students (n=301) sought help in multiple computer-mediated conversations, both Q&A forum and online office hours (OHQ), differently. Results showed the more prior knowledge about programming students had, the more they sought help in the Q&A compared to students with less prior knowledge. In general, higher-performing students sought help online in the Q&A more than the lower-performing groups on all the homework assignments, but not for the OHQ. By better understanding how students seek help online across multiple modalities of computer-mediated conversations and the relationship between help-seeking and grades, we can re-design online resources that best support all students in introductory programming courses at scale.
{"title":"Online help-seeking occurring in multiple computer-mediated conversations affects grades in an introductory programming course","authors":"Elizabeth B. Cloude, R. Baker, Eric Fouh","doi":"10.1145/3576050.3576106","DOIUrl":"https://doi.org/10.1145/3576050.3576106","url":null,"abstract":"Computing education researchers often study the impact of online help-seeking behaviors that occur across multiple online resources in isolation. Such separation fails to capture the interconnected nature of online help-seeking behaviors that occur across multiple online resources and its affect on course grades. This is particularly important for programming education, which arguably has more online resources to seek help from other people (e.g., computer-mediated conversations) than other majors. Using data from an introductory programming course (CS1) at a large US university, we found that students (n=301) sought help in multiple computer-mediated conversations, both Q&A forum and online office hours (OHQ), differently. Results showed the more prior knowledge about programming students had, the more they sought help in the Q&A compared to students with less prior knowledge. In general, higher-performing students sought help online in the Q&A more than the lower-performing groups on all the homework assignments, but not for the OHQ. By better understanding how students seek help online across multiple modalities of computer-mediated conversations and the relationship between help-seeking and grades, we can re-design online resources that best support all students in introductory programming courses at scale.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132066936","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ashish Gurung, Sami Baral, Kirk P. Vanacore, Andrew A. Mcreynolds, Hilary Kreisberg, Anthony F. Botelho, S. Shaw, Neil T. Hefferna
Prior work analyzing tutoring sessions provided evidence that highly effective tutors, through their interaction with students and their experience, can perceptively recognize incorrect processes or “bugs” when students incorrectly answer problems. Researchers have studied these tutoring interactions examining instructional approaches to address incorrect processes and observed that the format of the feedback can influence learning outcomes. In this work, we recognize the incorrect answers caused by these buggy processes as Common Wrong Answers (CWAs). We examine the ability of teachers and instructional designers to identify CWAs proactively. As teachers and instructional designers deeply understand the common approaches and mistakes students make when solving mathematical problems, we examine the feasibility of proactively identifying CWAs and generating Common Wrong Answer Feedback (CWAFs) as a formative feedback intervention for addressing student learning needs. As such, we analyze CWAFs in three sets of analyses. We first report on the accuracy of the CWAs predicted by the teachers and instructional designers on the problems across two activities. We then measure the effectiveness of the CWAFs using an intent-to-treat analysis. Finally, we explore the existence of personalization effects of the CWAFs for the students working on the two mathematics activities.
{"title":"Identification, Exploration, and Remediation: Can Teachers Predict Common Wrong Answers?","authors":"Ashish Gurung, Sami Baral, Kirk P. Vanacore, Andrew A. Mcreynolds, Hilary Kreisberg, Anthony F. Botelho, S. Shaw, Neil T. Hefferna","doi":"10.1145/3576050.3576109","DOIUrl":"https://doi.org/10.1145/3576050.3576109","url":null,"abstract":"Prior work analyzing tutoring sessions provided evidence that highly effective tutors, through their interaction with students and their experience, can perceptively recognize incorrect processes or “bugs” when students incorrectly answer problems. Researchers have studied these tutoring interactions examining instructional approaches to address incorrect processes and observed that the format of the feedback can influence learning outcomes. In this work, we recognize the incorrect answers caused by these buggy processes as Common Wrong Answers (CWAs). We examine the ability of teachers and instructional designers to identify CWAs proactively. As teachers and instructional designers deeply understand the common approaches and mistakes students make when solving mathematical problems, we examine the feasibility of proactively identifying CWAs and generating Common Wrong Answer Feedback (CWAFs) as a formative feedback intervention for addressing student learning needs. As such, we analyze CWAFs in three sets of analyses. We first report on the accuracy of the CWAs predicted by the teachers and instructional designers on the problems across two activities. We then measure the effectiveness of the CWAFs using an intent-to-treat analysis. Finally, we explore the existence of personalization effects of the CWAFs for the students working on the two mathematics activities.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"67 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124521558","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Reading on digital devices has become more commonplace, while it often poses challenges to learners’ attention. In this study, we hypothesized that allowing learners to reflect on their reading phases with an empathic social robot companion might enhance learners’ attention in e-reading. To verify our assumption, we collected a novel dataset (SKEP) in an e-reading setting with social robot support. It contains 25 multimodal features from various sensors and logged data that are direct and indirect cues of attention. Based on the SKEP dataset, we comprehensively compared the difference between HRI-based (treatment) and GUI-based (control) feedback and obtained insights for intervention design. Based on the human annotation of the nearly 40 hours of video data streams from 60 subjects, we developed a machine learning model to capture attention-regulation behaviors in e-reading. We exploited a two-stage framework to recognize learners’ observable self-regulatory behaviors and conducted attention analysis. The proposed system showed a promising performance with high prediction results of e-reading with HRI, such as 72.97% accuracy in recognizing attention regulation behaviors, 74.29% accuracy in predicting knowledge gain, 75.00% for perceived interaction experience, and 75.00% for perceived social presence. We believe our work can inspire the future design of HRI-based e-reading and its analysis.
{"title":"Can We Empower Attentive E-reading with a Social Robot? An Introductory Study with a Novel Multimodal Dataset and Deep Learning Approaches","authors":"Yoon Lee, M. Specht","doi":"10.1145/3576050.3576122","DOIUrl":"https://doi.org/10.1145/3576050.3576122","url":null,"abstract":"Reading on digital devices has become more commonplace, while it often poses challenges to learners’ attention. In this study, we hypothesized that allowing learners to reflect on their reading phases with an empathic social robot companion might enhance learners’ attention in e-reading. To verify our assumption, we collected a novel dataset (SKEP) in an e-reading setting with social robot support. It contains 25 multimodal features from various sensors and logged data that are direct and indirect cues of attention. Based on the SKEP dataset, we comprehensively compared the difference between HRI-based (treatment) and GUI-based (control) feedback and obtained insights for intervention design. Based on the human annotation of the nearly 40 hours of video data streams from 60 subjects, we developed a machine learning model to capture attention-regulation behaviors in e-reading. We exploited a two-stage framework to recognize learners’ observable self-regulatory behaviors and conducted attention analysis. The proposed system showed a promising performance with high prediction results of e-reading with HRI, such as 72.97% accuracy in recognizing attention regulation behaviors, 74.29% accuracy in predicting knowledge gain, 75.00% for perceived interaction experience, and 75.00% for perceived social presence. We believe our work can inspire the future design of HRI-based e-reading and its analysis.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"218 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132274817","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Amber J. Dood, K. Das, Zhen Qian, S. Finkenstaedt-Quinn, A. Gere, G. Shultz
Writing-to-Learn (WTL) is an evidence-based instructional practice which can help students construct knowledge across many disciplines. Though it is known to be an effective practice, many instructors do not implement WTL in their courses due to time constraints and inability to provide students with personalized feedback. One way to address this is to include peer review, which allows students to receive feedback on their writing and benefits them as they act as reviewers. To further ease the implementation of peer review and provide instructors with feedback on their students’ work, we labeled students’ peer review comments across courses for type of feedback provided and trained a machine learning model to automatically classify those comments, improving upon models reported in prior work. We then created a dashboard which takes students’ comments, labels the comments using the model, and allows instructors to filter through their students’ comments based on how the model labels the comments. This dashboard can be used by instructors to monitor the peer review collaborations occurring in their courses. The dashboard will allow them to efficiently use information provided by peers to identify common issues in their students’ writing and better evaluate the quality of their students’ peer review.
{"title":"A Dashboard to Provide Instructors with Automated Feedback on Students’ Peer Review Comments","authors":"Amber J. Dood, K. Das, Zhen Qian, S. Finkenstaedt-Quinn, A. Gere, G. Shultz","doi":"10.1145/3576050.3576087","DOIUrl":"https://doi.org/10.1145/3576050.3576087","url":null,"abstract":"Writing-to-Learn (WTL) is an evidence-based instructional practice which can help students construct knowledge across many disciplines. Though it is known to be an effective practice, many instructors do not implement WTL in their courses due to time constraints and inability to provide students with personalized feedback. One way to address this is to include peer review, which allows students to receive feedback on their writing and benefits them as they act as reviewers. To further ease the implementation of peer review and provide instructors with feedback on their students’ work, we labeled students’ peer review comments across courses for type of feedback provided and trained a machine learning model to automatically classify those comments, improving upon models reported in prior work. We then created a dashboard which takes students’ comments, labels the comments using the model, and allows instructors to filter through their students’ comments based on how the model labels the comments. This dashboard can be used by instructors to monitor the peer review collaborations occurring in their courses. The dashboard will allow them to efficiently use information provided by peers to identify common issues in their students’ writing and better evaluate the quality of their students’ peer review.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"52 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129777538","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
H. Almoubayyed, Stephen E. Fancsali, Steven Ritter
Adaptive educational software is likely to better support broader and more diverse sets of learners by considering more comprehensive views (or models) of such learners. For example, recent work proposed making inferences about “non-math” factors like reading comprehension while students used adaptive software for mathematics to better support and adapt to learners. We build on this proposed approach to more comprehensive learning modeling by providing an empirical basis for making inferences about students’ reading ability from their performance on activities in adaptive software for mathematics. We lay out an approach to predicting middle school students’ reading ability using their performance on activities within Carnegie Learning’s MATHia, a widely used intelligent tutoring system for mathematics. We focus on how performance in an early, introductory activity as an especially powerful place to consider instruction-embedded assessment of non-math factors like reading comprehension to guide adaptation based on factors like reading ability. We close by discussing opportunities to extend this work by focusing on particular knowledge components or skills tracked by MATHia that may provide important “levers” for driving adaptation based on students’ reading ability while they learn and practice mathematics.
{"title":"Instruction-Embedded Assessment for Reading Ability in Adaptive Mathematics Software","authors":"H. Almoubayyed, Stephen E. Fancsali, Steven Ritter","doi":"10.1145/3576050.3576105","DOIUrl":"https://doi.org/10.1145/3576050.3576105","url":null,"abstract":"Adaptive educational software is likely to better support broader and more diverse sets of learners by considering more comprehensive views (or models) of such learners. For example, recent work proposed making inferences about “non-math” factors like reading comprehension while students used adaptive software for mathematics to better support and adapt to learners. We build on this proposed approach to more comprehensive learning modeling by providing an empirical basis for making inferences about students’ reading ability from their performance on activities in adaptive software for mathematics. We lay out an approach to predicting middle school students’ reading ability using their performance on activities within Carnegie Learning’s MATHia, a widely used intelligent tutoring system for mathematics. We focus on how performance in an early, introductory activity as an especially powerful place to consider instruction-embedded assessment of non-math factors like reading comprehension to guide adaptation based on factors like reading ability. We close by discussing opportunities to extend this work by focusing on particular knowledge components or skills tracked by MATHia that may provide important “levers” for driving adaptation based on students’ reading ability while they learn and practice mathematics.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"55 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127122419","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Language learning applications usually estimate the learner’s language knowledge over time to provide personalized practice content for each learner at the optimal timing. However, accurately predicting language knowledge or linguistic skills is much more challenging than math or science knowledge, as many language tasks involve memorization and retrieval. Learners must memorize a large number of words and meanings, which are prone to be forgotten without practice. Although a few studies consider forgetting when modeling learners’ language knowledge, they tend to apply traditional models, consider only partial information about forgetting, and ignore linguistic features that may significantly influence learning and forgetting. This paper focuses on modeling and predicting learners’ knowledge by considering their forgetting behavior and linguistic features in language learning. Specifically, we first explore the existence of forgetting behavior and cross-effects in real-world language learning datasets through empirical studies. Based on these, we propose a model for predicting the probability of recalling a word given a learner’s practice history. The model incorporates key information related to forgetting, question formats, and semantic similarities between words using the attention mechanism. Experiments on two real-world datasets show that the proposed model improves performance compared to baselines. Moreover, the results indicate that combining multiple types of forgetting information and item format improves performance. In addition, we find that incorporating semantic features, such as word embeddings, to model similarities between words in a learner’s practice history and their effects on memory also improves the model.
{"title":"Each Encounter Counts: Modeling Language Learning and Forgetting","authors":"B. Ma, G. Hettiarachchi, Sora Fukui, Yuji Ando","doi":"10.1145/3576050.3576062","DOIUrl":"https://doi.org/10.1145/3576050.3576062","url":null,"abstract":"Language learning applications usually estimate the learner’s language knowledge over time to provide personalized practice content for each learner at the optimal timing. However, accurately predicting language knowledge or linguistic skills is much more challenging than math or science knowledge, as many language tasks involve memorization and retrieval. Learners must memorize a large number of words and meanings, which are prone to be forgotten without practice. Although a few studies consider forgetting when modeling learners’ language knowledge, they tend to apply traditional models, consider only partial information about forgetting, and ignore linguistic features that may significantly influence learning and forgetting. This paper focuses on modeling and predicting learners’ knowledge by considering their forgetting behavior and linguistic features in language learning. Specifically, we first explore the existence of forgetting behavior and cross-effects in real-world language learning datasets through empirical studies. Based on these, we propose a model for predicting the probability of recalling a word given a learner’s practice history. The model incorporates key information related to forgetting, question formats, and semantic similarities between words using the attention mechanism. Experiments on two real-world datasets show that the proposed model improves performance compared to baselines. Moreover, the results indicate that combining multiple types of forgetting information and item format improves performance. In addition, we find that incorporating semantic features, such as word embeddings, to model similarities between words in a learner’s practice history and their effects on memory also improves the model.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"49 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128920024","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Writing-to-learn (WTL) pedagogy supports the implementation of writing assignments in STEM courses to engage students in conceptual learning. Recent studies in the undergraduate STEM context demonstrate the value of implementing WTL, with findings that WTL can support meaningful learning and elicit students’ reasoning. However, the need for instructors to provide feedback on students’ writing poses a significant barrier to implementing WTL; this barrier is especially notable in the context of introductory organic chemistry courses at large universities, which often have large enrollments. This work describes one approach to overcome this barrier by presenting the development of an automated feedback tool for providing students with formative feedback on their responses to an organic chemistry WTL assignment. This approach leverages machine learning models to identify features of students’ mechanistic reasoning in response to WTL assignments in a second-semester, introductory organic chemistry laboratory course. The automated feedback tool development was guided by a framework for designing automated feedback, theories of self-regulated learning, and the components of effective WTL pedagogy. Herein, we describe the design of the automated feedback tool and report our initial evaluation of the tool through pilot interviews with organic chemistry students.
{"title":"Automated, content-focused feedback for a writing-to-learn assignment in an undergraduate organic chemistry course","authors":"Field M. Watts, Amber J. Dood, G. Shultz","doi":"10.1145/3576050.3576053","DOIUrl":"https://doi.org/10.1145/3576050.3576053","url":null,"abstract":"Writing-to-learn (WTL) pedagogy supports the implementation of writing assignments in STEM courses to engage students in conceptual learning. Recent studies in the undergraduate STEM context demonstrate the value of implementing WTL, with findings that WTL can support meaningful learning and elicit students’ reasoning. However, the need for instructors to provide feedback on students’ writing poses a significant barrier to implementing WTL; this barrier is especially notable in the context of introductory organic chemistry courses at large universities, which often have large enrollments. This work describes one approach to overcome this barrier by presenting the development of an automated feedback tool for providing students with formative feedback on their responses to an organic chemistry WTL assignment. This approach leverages machine learning models to identify features of students’ mechanistic reasoning in response to WTL assignments in a second-semester, introductory organic chemistry laboratory course. The automated feedback tool development was guided by a framework for designing automated feedback, theories of self-regulated learning, and the components of effective WTL pedagogy. Herein, we describe the design of the automated feedback tool and report our initial evaluation of the tool through pilot interviews with organic chemistry students.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128893426","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：