K. Sharma, Hamed S. Alavi, Patrick Jermann, P. Dillenbourg
In the context of MOOCs, "With-me-ness" refers to the extent to which the learner succeeds in following the teacher, specifically in terms of looking at the area in the video that the teacher is explaining. In our previous works, we employed eye-tracking methods to quantify learners' With-me-ness and showed that it is positively correlated with their learning gains. In this contribution, we describe a tool that is designed to improve With-me-ness by providing a visual-aid superimposed on the video. The position of the visual-aid is suggested by the teachers' dialogue and deixis, and it is displayed when the learner's With-me-ness is under the average value, which is computed from the other students' gaze behavior. We report on a user-study that examines the effectiveness of the proposed tool. The results show that it significantly improves the learning gain and it significantly increases the extent to which the students follow the teacher. Finally, we demonstrate how With-me-ness can create a complete theoretical framework for conducting gaze-based learning analytics in the context of MOOCs.
{"title":"A gaze-based learning analytics model: in-video visual feedback to improve learner's attention in MOOCs","authors":"K. Sharma, Hamed S. Alavi, Patrick Jermann, P. Dillenbourg","doi":"10.1145/2883851.2883902","DOIUrl":"https://doi.org/10.1145/2883851.2883902","url":null,"abstract":"In the context of MOOCs, \"With-me-ness\" refers to the extent to which the learner succeeds in following the teacher, specifically in terms of looking at the area in the video that the teacher is explaining. In our previous works, we employed eye-tracking methods to quantify learners' With-me-ness and showed that it is positively correlated with their learning gains. In this contribution, we describe a tool that is designed to improve With-me-ness by providing a visual-aid superimposed on the video. The position of the visual-aid is suggested by the teachers' dialogue and deixis, and it is displayed when the learner's With-me-ness is under the average value, which is computed from the other students' gaze behavior. We report on a user-study that examines the effectiveness of the proposed tool. The results show that it significantly improves the learning gain and it significantly increases the extent to which the students follow the teacher. Finally, we demonstrate how With-me-ness can create a complete theoretical framework for conducting gaze-based learning analytics in the context of MOOCs.","PeriodicalId":343844,"journal":{"name":"Proceedings of the Sixth International Conference on Learning Analytics & Knowledge","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131016166","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Korinn S. Ostrow, Douglas Selent, Yan Wang, E. V. Inwegen, N. Heffernan, J. Williams
Researchers invested in K-12 education struggle not just to enhance pedagogy, curriculum, and student engagement, but also to harness the power of technology in ways that will optimize learning. Online learning platforms offer a powerful environment for educational research at scale. The present work details the creation of an automated system designed to provide researchers with insights regarding data logged from randomized controlled experiments conducted within the ASSISTments TestBed. The Assessment of Learning Infrastructure (ALI) builds upon existing technologies to foster a symbiotic relationship beneficial to students, researchers, the platform and its content, and the learning analytics community. ALI is a sophisticated automated reporting system that provides an overview of sample distributions and basic analyses for researchers to consider when assessing their data. ALI's benefits can also be felt at scale through analyses that crosscut multiple studies to drive iterative platform improvements while promoting personalized learning.
{"title":"The assessment of learning infrastructure (ALI): the theory, practice, and scalability of automated assessment","authors":"Korinn S. Ostrow, Douglas Selent, Yan Wang, E. V. Inwegen, N. Heffernan, J. Williams","doi":"10.1145/2883851.2883872","DOIUrl":"https://doi.org/10.1145/2883851.2883872","url":null,"abstract":"Researchers invested in K-12 education struggle not just to enhance pedagogy, curriculum, and student engagement, but also to harness the power of technology in ways that will optimize learning. Online learning platforms offer a powerful environment for educational research at scale. The present work details the creation of an automated system designed to provide researchers with insights regarding data logged from randomized controlled experiments conducted within the ASSISTments TestBed. The Assessment of Learning Infrastructure (ALI) builds upon existing technologies to foster a symbiotic relationship beneficial to students, researchers, the platform and its content, and the learning analytics community. ALI is a sophisticated automated reporting system that provides an overview of sample distributions and basic analyses for researchers to consider when assessing their data. ALI's benefits can also be felt at scale through analyses that crosscut multiple studies to drive iterative platform improvements while promoting personalized learning.","PeriodicalId":343844,"journal":{"name":"Proceedings of the Sixth International Conference on Learning Analytics & Knowledge","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129759273","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Prerequisite skill structures have been closely studied in past years leading to many data-intensive methods aimed at refining such structures. While many of these proposed methods have yielded success, defining and refining hierarchies of skill relationships are often difficult tasks. The relationship between skills in a graph could either be causal, therefore, a prerequisite relationship (skill A must be learned before skill B). The relationship may be non-causal, in which case the ordering of skills does not matter and may indicate that both skills are prerequisites of another skill. In this study, we propose a simple, effective method of determining the strength of pre-to-post-requisite skill relationships. We then compare our results with a teacher-level survey about the strength of the relationships of the observed skills and find that the survey results largely confirm our findings in the data-driven approach.
{"title":"Predicting student performance on post-requisite skills using prerequisite skill data: an alternative method for refining prerequisite skill structures","authors":"Seth A. Adjei, Anthony F. Botelho, N. Heffernan","doi":"10.1145/2883851.2883867","DOIUrl":"https://doi.org/10.1145/2883851.2883867","url":null,"abstract":"Prerequisite skill structures have been closely studied in past years leading to many data-intensive methods aimed at refining such structures. While many of these proposed methods have yielded success, defining and refining hierarchies of skill relationships are often difficult tasks. The relationship between skills in a graph could either be causal, therefore, a prerequisite relationship (skill A must be learned before skill B). The relationship may be non-causal, in which case the ordering of skills does not matter and may indicate that both skills are prerequisites of another skill. In this study, we propose a simple, effective method of determining the strength of pre-to-post-requisite skill relationships. We then compare our results with a teacher-level survey about the strength of the relationships of the observed skills and find that the survey results largely confirm our findings in the data-driven approach.","PeriodicalId":343844,"journal":{"name":"Proceedings of the Sixth International Conference on Learning Analytics & Knowledge","volume":"85 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130451217","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Even though the recent influx of tablets in primary education goes together with the vision that educational technology empowered with learning analytics will revolutionize education, empirical results supporting this claim are scares. Adaptive educational technology Snappet combines extracted and embedded learning analytics daily in classrooms. While students make exercises on the tablet this technology displays real-time data of learner performance in a teacher dashboard (extracted analytics). At the same time, learner performance is used to adaptively adjust exercises to students' progress (embedded analytics). This quasiexperimental study compares the development of students' arithmetic skills over one schoolyear (grade 2 and 4) in a traditional paper based setting to learning with the adaptive educational technology Snappet. The results indicate that students in the Snappet condition make significantly more progress on arithmetic skills in grade 4. Moreover, in this grade students with a high ability level, benefit the most from working with this adaptive educational technology. Overall the development pattern of students with different abilities was more divergent in the AET condition compared to the control condition. These results indicate that adaptive educational technologies combining extracted and embedded learning analytics are indeed creating new education scenarios that contribute to personalized learning in primary education.
{"title":"Learning analytics in practice: the effects of adaptive educational technology Snappet on students' arithmetic skills","authors":"I. Molenaar, C. K. Campen","doi":"10.1145/2883851.2883892","DOIUrl":"https://doi.org/10.1145/2883851.2883892","url":null,"abstract":"Even though the recent influx of tablets in primary education goes together with the vision that educational technology empowered with learning analytics will revolutionize education, empirical results supporting this claim are scares. Adaptive educational technology Snappet combines extracted and embedded learning analytics daily in classrooms. While students make exercises on the tablet this technology displays real-time data of learner performance in a teacher dashboard (extracted analytics). At the same time, learner performance is used to adaptively adjust exercises to students' progress (embedded analytics). This quasiexperimental study compares the development of students' arithmetic skills over one schoolyear (grade 2 and 4) in a traditional paper based setting to learning with the adaptive educational technology Snappet. The results indicate that students in the Snappet condition make significantly more progress on arithmetic skills in grade 4. Moreover, in this grade students with a high ability level, benefit the most from working with this adaptive educational technology. Overall the development pattern of students with different abilities was more divergent in the AET condition compared to the control condition. These results indicate that adaptive educational technologies combining extracted and embedded learning analytics are indeed creating new education scenarios that contribute to personalized learning in primary education.","PeriodicalId":343844,"journal":{"name":"Proceedings of the Sixth International Conference on Learning Analytics & Knowledge","volume":"38 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132015431","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The focus of the learning analytics community bridges the gap between controlled educational research and data mining. Online learning platforms can be used to conduct randomized controlled trials to assist in the development of interventions that increase learning gains; datasets from such research can act as a treasure trove for inquisitive data miners. The present work employs a data mining approach on randomized controlled trial data from ASSISTments, a popular online learning platform, to assess the benefits of incorporating additional student performance data when attempting to differentiate between two user groups. Through a resampling technique, we show that partial credit, defined as an algorithmic combination of binary correctness, hint usage, and attempt count, can benefit assessment and group differentiation. Partial credit reduces sample sizes required to reliably differentiate between groups that are known to differ by 58%, and reduces sample sizes required to reliably differentiate between less distinct groups by 9%.
{"title":"Enhancing the efficiency and reliability of group differentiation through partial credit","authors":"Yan Wang, Korinn S. Ostrow, J. Beck, N. Heffernan","doi":"10.1145/2883851.2883910","DOIUrl":"https://doi.org/10.1145/2883851.2883910","url":null,"abstract":"The focus of the learning analytics community bridges the gap between controlled educational research and data mining. Online learning platforms can be used to conduct randomized controlled trials to assist in the development of interventions that increase learning gains; datasets from such research can act as a treasure trove for inquisitive data miners. The present work employs a data mining approach on randomized controlled trial data from ASSISTments, a popular online learning platform, to assess the benefits of incorporating additional student performance data when attempting to differentiate between two user groups. Through a resampling technique, we show that partial credit, defined as an algorithmic combination of binary correctness, hint usage, and attempt count, can benefit assessment and group differentiation. Partial credit reduces sample sizes required to reliably differentiate between groups that are known to differ by 58%, and reduces sample sizes required to reliably differentiate between less distinct groups by 9%.","PeriodicalId":343844,"journal":{"name":"Proceedings of the Sixth International Conference on Learning Analytics & Knowledge","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124292977","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This study attempts to investigate to what extent indicators of academic writing and cognitive thinking can help measure the writing quality of group collaborative writings on Wikis. Particularly, comparisons were made on Wiki content in different stages of the projects. Preliminary results from a multiple linear regression analysis reveal that linguistic indicators such as engagement markers and self-mention were significant predictors in earlier stages to the projects, whereas verbs indicating cognitive thinking in the evaluation level were significant in later project stages.
{"title":"Automating assessment of collaborative writing quality in multiple stages: the case of wiki","authors":"Xiao Hu, Tzi-Dong Jeremy Ng, L. Tian, Chi-Un Lei","doi":"10.1145/2883851.2883963","DOIUrl":"https://doi.org/10.1145/2883851.2883963","url":null,"abstract":"This study attempts to investigate to what extent indicators of academic writing and cognitive thinking can help measure the writing quality of group collaborative writings on Wikis. Particularly, comparisons were made on Wiki content in different stages of the projects. Preliminary results from a multiple linear regression analysis reveal that linguistic indicators such as engagement markers and self-mention were significant predictors in earlier stages to the projects, whereas verbs indicating cognitive thinking in the evaluation level were significant in later project stages.","PeriodicalId":343844,"journal":{"name":"Proceedings of the Sixth International Conference on Learning Analytics & Knowledge","volume":"39 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117117545","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
M. Wells, A. Wollenschlaeger, D. Lefevre, G. D. Magoulas, A. Poulovassilis
We analyse engagement and performance data arising from participants' interactions with an in-house LMS at Imperial College London while a cohort of students follow two courses on a new online postgraduate degree in Management. We identify and investigate two main questions relating to the relationships between engagement and performance, drawing recommendations for improved guidelines to inform the design of such courses.
我们分析了参与者与伦敦帝国理工学院(Imperial College London)内部管理学管理系统(LMS)互动产生的参与度和绩效数据，同时让一群学生学习新的在线管理学研究生学位的两门课程。我们确定并调查了与参与和绩效之间关系有关的两个主要问题，并为改进指导方针提出建议，为此类课程的设计提供信息。
{"title":"Analysing engagement in an online management programme and implications for course design","authors":"M. Wells, A. Wollenschlaeger, D. Lefevre, G. D. Magoulas, A. Poulovassilis","doi":"10.1145/2883851.2883894","DOIUrl":"https://doi.org/10.1145/2883851.2883894","url":null,"abstract":"We analyse engagement and performance data arising from participants' interactions with an in-house LMS at Imperial College London while a cohort of students follow two courses on a new online postgraduate degree in Management. We identify and investigate two main questions relating to the relationships between engagement and performance, drawing recommendations for improved guidelines to inform the design of such courses.","PeriodicalId":343844,"journal":{"name":"Proceedings of the Sixth International Conference on Learning Analytics & Knowledge","volume":"70 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121981983","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Student mistakes are often not random but, rather, reflect thoughtful yet incorrect strategies. In order for educational technologies to make full use of students' performance data to estimate the knowledge of a student, it is important to model not only the conceptions but also the misconceptions that a student's particular pattern of successes and errors may indicate. The student models that drive the "outer loop" of Intelligent Tutoring Systems typically do not represent or track misconceptions. Here, we present a method of representing misconceptions in the Knowledge Component models, or Q-Matrices, that are used by student models to estimate latent knowledge. We show, in a case study on a fraction arithmetic dataset, that incorporating a misconception into the Knowledge Component model dramatically improves the overall model's fit to data. We also derive qualitative insights from comparing predicted learning curves across models that incorporate varying misconception-related parameters. Finally, we show that the inclusion of a misconception in the Knowledge Component model can yield individual student estimates of misconception strength that are significantly correlated with out-of-tutor measures of student errors.
{"title":"Modeling common misconceptions in learning process data","authors":"Ran Liu, Rony Patel, K. Koedinger","doi":"10.1145/2883851.2883967","DOIUrl":"https://doi.org/10.1145/2883851.2883967","url":null,"abstract":"Student mistakes are often not random but, rather, reflect thoughtful yet incorrect strategies. In order for educational technologies to make full use of students' performance data to estimate the knowledge of a student, it is important to model not only the conceptions but also the misconceptions that a student's particular pattern of successes and errors may indicate. The student models that drive the \"outer loop\" of Intelligent Tutoring Systems typically do not represent or track misconceptions. Here, we present a method of representing misconceptions in the Knowledge Component models, or Q-Matrices, that are used by student models to estimate latent knowledge. We show, in a case study on a fraction arithmetic dataset, that incorporating a misconception into the Knowledge Component model dramatically improves the overall model's fit to data. We also derive qualitative insights from comparing predicted learning curves across models that incorporate varying misconception-related parameters. Finally, we show that the inclusion of a misconception in the Knowledge Component model can yield individual student estimates of misconception strength that are significantly correlated with out-of-tutor measures of student errors.","PeriodicalId":343844,"journal":{"name":"Proceedings of the Sixth International Conference on Learning Analytics & Knowledge","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128606574","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Tobias Ley, R. Klamma, Stefanie N. Lindstaedt, Fridolin Wild
Recognizing the need for addressing the rather fragmented character of research in this field, we have held a workshop on learning analytics for workplace and professional learning at the Learning Analytics and Knowledge (LAK) Conference. The workshop has taken a broad perspective, encompassing approaches from a number of previous traditions, such as adaptive learning, professional online communities, workplace learning and performance analytics. Being co-located with the LAK conference has provided an ideal venue for addressing common challenges and for benefiting from the strong research on learning analytics in other sectors that LAK has established. Learning Analytics for Workplace and Professional Learning is now on the research agenda of several ongoing EU projects, and therefore a number of follow-up activities are planned for strengthening integration in this emerging field.
{"title":"Learning analytics for workplace and professional learning","authors":"Tobias Ley, R. Klamma, Stefanie N. Lindstaedt, Fridolin Wild","doi":"10.1145/2883851.2883860","DOIUrl":"https://doi.org/10.1145/2883851.2883860","url":null,"abstract":"Recognizing the need for addressing the rather fragmented character of research in this field, we have held a workshop on learning analytics for workplace and professional learning at the Learning Analytics and Knowledge (LAK) Conference. The workshop has taken a broad perspective, encompassing approaches from a number of previous traditions, such as adaptive learning, professional online communities, workplace learning and performance analytics. Being co-located with the LAK conference has provided an ideal venue for addressing common challenges and for benefiting from the strong research on learning analytics in other sectors that LAK has established. Learning Analytics for Workplace and Professional Learning is now on the research agenda of several ongoing EU projects, and therefore a number of follow-up activities are planned for strengthening integration in this emerging field.","PeriodicalId":343844,"journal":{"name":"Proceedings of the Sixth International Conference on Learning Analytics & Knowledge","volume":"116 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128086777","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Computerized educational systems are increasingly provided as open online services which provide adaptive personalized learning experience. To fully exploit potential of such systems, it is necessary to thoroughly evaluate different design choices. However, both openness and adaptivity make proper evaluation difficult. We provide a detailed report on evaluation of an online system for adaptive practice of geography, and use this case study to highlight methodological issues with evaluation of open online learning systems, particularly attrition bias. To facilitate evaluation of learning, we propose to use randomized reference questions. We illustrate application of survival analysis and learning curves for declarative knowledge. The result provide an interesting insight into the impact of adaptivity on learner behaviour and learning.
{"title":"Evaluation of an adaptive practice system for learning geography facts","authors":"Jan Papousek, V. Stanislav, Radek Pelánek","doi":"10.1145/2883851.2883884","DOIUrl":"https://doi.org/10.1145/2883851.2883884","url":null,"abstract":"Computerized educational systems are increasingly provided as open online services which provide adaptive personalized learning experience. To fully exploit potential of such systems, it is necessary to thoroughly evaluate different design choices. However, both openness and adaptivity make proper evaluation difficult. We provide a detailed report on evaluation of an online system for adaptive practice of geography, and use this case study to highlight methodological issues with evaluation of open online learning systems, particularly attrition bias. To facilitate evaluation of learning, we propose to use randomized reference questions. We illustrate application of survival analysis and learning curves for declarative knowledge. The result provide an interesting insight into the impact of adaptivity on learner behaviour and learning.","PeriodicalId":343844,"journal":{"name":"Proceedings of the Sixth International Conference on Learning Analytics & Knowledge","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114367158","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：