Will Hudgins, M. Lynch, Ash Schmal, Harsh Sikka, Michael Swenson, David A. Joyner
While the literature on learning at scale has largely focused on MOOCs, online degree programs, and AI techniques for supporting scalable learning experiences, informal learning communities have been relatively underrepresented. None-theless, these massive open online learning communities regularly draw far more engaged users than the typical MOOC. Their informal structure, however, makes them significantly more difficult to study. In this work, we take a first step toward attempting to understand these communi-ties specifically from the perspective of scale. Taking a sample of 62 such communities, we develop a tagging sys-tem for understanding the specific features and how they relate to scale. For example, just as a MOOC cannot man-ually grade every assignment, so also an informal learning community cannot approve every contribution; and just as MOOCs therefore employ autograding, informal learning communities employ crowd-sourced moderation or plat-form-driven enforcement. Using these tags, we then select several communities for deeper case studies. We also use these tags to make sense of learning-based subreddits from the popular community site Reddit, which offers an API for programmatic analysis. Based on these techniques, we offer findings about the performance of informal learning communities at scale and issue a call to include these envi-ronments more fully in future research on learning at scale.
{"title":"Informal Learning Communities: The Other Massive Open Online 'C'","authors":"Will Hudgins, M. Lynch, Ash Schmal, Harsh Sikka, Michael Swenson, David A. Joyner","doi":"10.1145/3386527.3405926","DOIUrl":"https://doi.org/10.1145/3386527.3405926","url":null,"abstract":"While the literature on learning at scale has largely focused on MOOCs, online degree programs, and AI techniques for supporting scalable learning experiences, informal learning communities have been relatively underrepresented. None-theless, these massive open online learning communities regularly draw far more engaged users than the typical MOOC. Their informal structure, however, makes them significantly more difficult to study. In this work, we take a first step toward attempting to understand these communi-ties specifically from the perspective of scale. Taking a sample of 62 such communities, we develop a tagging sys-tem for understanding the specific features and how they relate to scale. For example, just as a MOOC cannot man-ually grade every assignment, so also an informal learning community cannot approve every contribution; and just as MOOCs therefore employ autograding, informal learning communities employ crowd-sourced moderation or plat-form-driven enforcement. Using these tags, we then select several communities for deeper case studies. We also use these tags to make sense of learning-based subreddits from the popular community site Reddit, which offers an API for programmatic analysis. Based on these techniques, we offer findings about the performance of informal learning communities at scale and issue a call to include these envi-ronments more fully in future research on learning at scale.","PeriodicalId":20608,"journal":{"name":"Proceedings of the Seventh ACM Conference on Learning @ Scale","volume":"5 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88796111","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This work reports on progress made towards building an equitable model to predict the success of an applicant to Georgia Tech's Online Master's in Analytics program. As a first step, we have collected and processed data on 9,044 applications and have trained a predictive model with a ROC-AUC score of 0.81, which predicts whether an applicant would be admitted to the program. Our next steps will include using applicant data to model the successful completion of the Analytics program's three core courses, graduation, and finally job placement. In addition, we plan to expand our feature processing and incorporate techniques to ensure that our models do not discriminate based on demographic factors. In the long run, we hope that the results of this study can be used to improve the course contents, selection of offered courses, and prerequisite training, and even give guidance toward the selection of the applicants.
{"title":"Predicting Applicant Admission Status for Georgia Tech's Online Master's in Analytics Program","authors":"S. Staudaher, Jeonghyun Lee, F. Soleimani","doi":"10.1145/3386527.3406735","DOIUrl":"https://doi.org/10.1145/3386527.3406735","url":null,"abstract":"This work reports on progress made towards building an equitable model to predict the success of an applicant to Georgia Tech's Online Master's in Analytics program. As a first step, we have collected and processed data on 9,044 applications and have trained a predictive model with a ROC-AUC score of 0.81, which predicts whether an applicant would be admitted to the program. Our next steps will include using applicant data to model the successful completion of the Analytics program's three core courses, graduation, and finally job placement. In addition, we plan to expand our feature processing and incorporate techniques to ensure that our models do not discriminate based on demographic factors. In the long run, we hope that the results of this study can be used to improve the course contents, selection of offered courses, and prerequisite training, and even give guidance toward the selection of the applicants.","PeriodicalId":20608,"journal":{"name":"Proceedings of the Seventh ACM Conference on Learning @ Scale","volume":"67 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84016222","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Peter Brusilovsky, K. Koedinger, David A. Joyner, T. Price
The goal of this workshop is to bring together the existing community of researchers working on Infrastructure Design for Data-Intensive Research in Computer Science Education and a community of Learning at Scale researchers focused on Computer Science Education. While both communities share many similar goals and could greatly benefit from each other work, the interaction between the communities is small. We hope that the proposed workshop will be instrumental in bringing together like-minded researchers from different communities, establishing collaboration, and expanding the scope of infrastructure project to address critical scaling issues.
{"title":"Building an Infrastructure for Computer Science Education Research and Practice at Scale","authors":"Peter Brusilovsky, K. Koedinger, David A. Joyner, T. Price","doi":"10.1145/3386527.3405936","DOIUrl":"https://doi.org/10.1145/3386527.3405936","url":null,"abstract":"The goal of this workshop is to bring together the existing community of researchers working on Infrastructure Design for Data-Intensive Research in Computer Science Education and a community of Learning at Scale researchers focused on Computer Science Education. While both communities share many similar goals and could greatly benefit from each other work, the interaction between the communities is small. We hope that the proposed workshop will be instrumental in bringing together like-minded researchers from different communities, establishing collaboration, and expanding the scope of infrastructure project to address critical scaling issues.","PeriodicalId":20608,"journal":{"name":"Proceedings of the Seventh ACM Conference on Learning @ Scale","volume":"42 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81022583","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Anish Khazane, Jia Mao, India Irish, Rocko Graziano, Thad Starner
As online educational programs scale, monitoring peer collaboration in platforms like BlueJeans for plagiarism becomes difficult. Recent studies indicate that students are less likely to cheat if presented with direct warning messages prior to engaging in online activities. In this work, we present Bluejeans codE Leak deTection (BELT), a system that monitors online BlueJeans meetings for shared code and sends timely warning messages to meeting participants. To test BELT's robustness as an online proctor, we evaluate its code-text disambiguation, code detection from images of varying quality, and code detection from videos of varying resolution. We conclude this work by pinpointing areas of improvement and briefly discuss possible extensions for future work.
{"title":"BELT","authors":"Anish Khazane, Jia Mao, India Irish, Rocko Graziano, Thad Starner","doi":"10.1145/3386527.3406727","DOIUrl":"https://doi.org/10.1145/3386527.3406727","url":null,"abstract":"As online educational programs scale, monitoring peer collaboration in platforms like BlueJeans for plagiarism becomes difficult. Recent studies indicate that students are less likely to cheat if presented with direct warning messages prior to engaging in online activities. In this work, we present Bluejeans codE Leak deTection (BELT), a system that monitors online BlueJeans meetings for shared code and sends timely warning messages to meeting participants. To test BELT's robustness as an online proctor, we evaluate its code-text disambiguation, code detection from images of varying quality, and code detection from videos of varying resolution. We conclude this work by pinpointing areas of improvement and briefly discuss possible extensions for future work.","PeriodicalId":20608,"journal":{"name":"Proceedings of the Seventh ACM Conference on Learning @ Scale","volume":"39 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"89490634","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ashank Verma, Timothy Bretl, Matthew West, C. Zilles
In this paper, we study a computerized exam system that allows students to attempt the same question multiple times. This system permits students either to receive feedback on their submitted answer immediately or to defer the feedback and grade questions in bulk. An analysis of student behavior in three courses across two semesters found similar student behaviors across courses and student groups. We found that only a small minority of students used the deferred feedback option. A clustering analysis that considered both when students chose to receive feedback and either to immediately retry incorrect problems or to attempt other unfinished problems identified four main student strategies. These strategies were correlated to statistically significant differences in exam scores, but it was not clear if some strategies improved outcomes or if stronger students tended to prefer certain strategies.
{"title":"A Quantitative Analysis of When Students Choose to Grade Questions on Computerized Exams with Multiple Attempts","authors":"Ashank Verma, Timothy Bretl, Matthew West, C. Zilles","doi":"10.1145/3386527.3406740","DOIUrl":"https://doi.org/10.1145/3386527.3406740","url":null,"abstract":"In this paper, we study a computerized exam system that allows students to attempt the same question multiple times. This system permits students either to receive feedback on their submitted answer immediately or to defer the feedback and grade questions in bulk. An analysis of student behavior in three courses across two semesters found similar student behaviors across courses and student groups. We found that only a small minority of students used the deferred feedback option. A clustering analysis that considered both when students chose to receive feedback and either to immediately retry incorrect problems or to attempt other unfinished problems identified four main student strategies. These strategies were correlated to statistically significant differences in exam scores, but it was not clear if some strategies improved outcomes or if stronger students tended to prefer certain strategies.","PeriodicalId":20608,"journal":{"name":"Proceedings of the Seventh ACM Conference on Learning @ Scale","volume":"23 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"90159857","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Explanations are used to provide an understanding of a concept, procedure, or reasoning to others. Although explanations are present online ubiquitously within textbooks, discussion forums, and many more, there is no way to mine them automatically to assist learners in seeking an explanation. To address this problem, we propose the task of Explanation Mining. To mine explanations of educational concepts, we propose a baseline approach based on the Language Modeling approach of information retrieval. Preliminary results suggest that incorporating knowledge from a model trained on the ELI5 (Explain Like I'm Five) dataset in the form of a document prior helps increase the performance of a standard retrieval model. This is encouraging because our method requires minimal in-domain supervision, as a result, it can be deployed for multiple online courses. We also suggest some interesting future work in the computational analysis of explanations.
解释用于向他人提供对概念、程序或推理的理解。尽管在线教科书、讨论论坛和其他很多地方都有解释，但没有办法自动挖掘它们来帮助学习者寻找解释。为了解决这个问题，我们提出了解释挖掘的任务。为了挖掘教育概念的解释，我们提出了一种基于信息检索的语言建模方法的基线方法。初步结果表明，将ELI5 (Explain Like I’m Five)数据集上训练的模型中的知识以文档的形式合并在一起，有助于提高标准检索模型的性能。这是令人鼓舞的，因为我们的方法需要最少的域内监督，因此，它可以部署到多个在线课程中。我们还建议在解释的计算分析方面进行一些有趣的未来工作。
{"title":"Explanation Mining","authors":"Bhavya, Chengxiang Zhai","doi":"10.1145/3386527.3406738","DOIUrl":"https://doi.org/10.1145/3386527.3406738","url":null,"abstract":"Explanations are used to provide an understanding of a concept, procedure, or reasoning to others. Although explanations are present online ubiquitously within textbooks, discussion forums, and many more, there is no way to mine them automatically to assist learners in seeking an explanation. To address this problem, we propose the task of Explanation Mining. To mine explanations of educational concepts, we propose a baseline approach based on the Language Modeling approach of information retrieval. Preliminary results suggest that incorporating knowledge from a model trained on the ELI5 (Explain Like I'm Five) dataset in the form of a document prior helps increase the performance of a standard retrieval model. This is encouraging because our method requires minimal in-domain supervision, as a result, it can be deployed for multiple online courses. We also suggest some interesting future work in the computational analysis of explanations.","PeriodicalId":20608,"journal":{"name":"Proceedings of the Seventh ACM Conference on Learning @ Scale","volume":"32 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"73251132","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Rich models of students' learning and problem-solving behaviors can support tailored interventions by instructors and scaffolding of complex learning activities. Our goal in this paper is to identify students' reading behaviors as they engage with instructional texts in domain-specific activities. In this work, we apply theory and methodology from the learning sciences to a large-scale middle school dataset within a digital literacy platform, Actively Learn. We compare students' reading behaviors both within and across domains for 12,566 science and 16,240 social studies students. Our findings show that higher-performing students in science engaged in more metacognitively-rich reading activities, such as text annotation; whereas lower-performing students relied more on simple highlighting and took longer to respond to embedded questions. Higher-performing students in social studies, by contrast, engaged more with the vocabulary and took longer to read before attempting question responses. Our finding may be used as recommendations to help both teachers and students engage in and support more effective behaviors.
{"title":"Understanding Reading Behaviors of Middle School Students","authors":"Effat Farhana, Teomara Rutherford, Collin Lynch","doi":"10.1145/3386527.3405948","DOIUrl":"https://doi.org/10.1145/3386527.3405948","url":null,"abstract":"Rich models of students' learning and problem-solving behaviors can support tailored interventions by instructors and scaffolding of complex learning activities. Our goal in this paper is to identify students' reading behaviors as they engage with instructional texts in domain-specific activities. In this work, we apply theory and methodology from the learning sciences to a large-scale middle school dataset within a digital literacy platform, Actively Learn. We compare students' reading behaviors both within and across domains for 12,566 science and 16,240 social studies students. Our findings show that higher-performing students in science engaged in more metacognitively-rich reading activities, such as text annotation; whereas lower-performing students relied more on simple highlighting and took longer to respond to embedded questions. Higher-performing students in social studies, by contrast, engaged more with the vocabulary and took longer to read before attempting question responses. Our finding may be used as recommendations to help both teachers and students engage in and support more effective behaviors.","PeriodicalId":20608,"journal":{"name":"Proceedings of the Seventh ACM Conference on Learning @ Scale","volume":"78 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"75671970","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Massive open online courses (MOOCs) provide a great opportunity to use multiple means of information representation through a mixture of various media such as text, graphics, and video, among others. However, most research on MOOCs focused on learning analytics and not much attention is given to content analysis. We gathered all text corpora and video transcripts of selected MOOCs using a web crawler and looked at word counts, clustered by distribution, and measured readability of the crawled data. Analyzing content distribution allows for a comparison of MOOCs regardless of topics, thus giving us an idea of what most course developers might think is ideal in terms of content distribution. This comparison along with readability analysis can be useful for course pre-run quality assessment and gauging content sufficiency.
{"title":"Content Type Distribution and Readability of MOOCs","authors":"M. Carlon, Nopphon Keerativoranan, J. Cross","doi":"10.1145/3386527.3405950","DOIUrl":"https://doi.org/10.1145/3386527.3405950","url":null,"abstract":"Massive open online courses (MOOCs) provide a great opportunity to use multiple means of information representation through a mixture of various media such as text, graphics, and video, among others. However, most research on MOOCs focused on learning analytics and not much attention is given to content analysis. We gathered all text corpora and video transcripts of selected MOOCs using a web crawler and looked at word counts, clustered by distribution, and measured readability of the crawled data. Analyzing content distribution allows for a comparison of MOOCs regardless of topics, thus giving us an idea of what most course developers might think is ideal in terms of content distribution. This comparison along with readability analysis can be useful for course pre-run quality assessment and gauging content sufficiency.","PeriodicalId":20608,"journal":{"name":"Proceedings of the Seventh ACM Conference on Learning @ Scale","volume":"21 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84163759","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Effective driver education techniques can greatly benefit from the use of state-of-the-art technologies for driving training and tutoring in classroom environment. Such environment includes simulation systems that are designed based on the Intelligent Tutoring System concepts and framework. This research analyzed simulator data for both simulation and vehicle environments to identify factors for driver training guidelines. Based on the results of this study, one of the recommendations is that current ITS based driver training systems be calibrated to accurately measure the steering input which is found to be the most significant parameter influencing time headway (distances between simulated vehicles). The findings also support the modern intelligent tutoring system used at scale that leverages human feedback to improve their design.
{"title":"Understanding the Implications of the Use of Intelligent Tutoring Systems in Driver Training","authors":"Al-Ahad Ekram","doi":"10.1145/3386527.3406756","DOIUrl":"https://doi.org/10.1145/3386527.3406756","url":null,"abstract":"Effective driver education techniques can greatly benefit from the use of state-of-the-art technologies for driving training and tutoring in classroom environment. Such environment includes simulation systems that are designed based on the Intelligent Tutoring System concepts and framework. This research analyzed simulator data for both simulation and vehicle environments to identify factors for driver training guidelines. Based on the results of this study, one of the recommendations is that current ITS based driver training systems be calibrated to accurately measure the steering input which is found to be the most significant parameter influencing time headway (distances between simulated vehicles). The findings also support the modern intelligent tutoring system used at scale that leverages human feedback to improve their design.","PeriodicalId":20608,"journal":{"name":"Proceedings of the Seventh ACM Conference on Learning @ Scale","volume":"37 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88453544","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Crowdsourcing has shown promise in education domains. For example, researchers have leveraged the wisdom of the crowd to process grading in MOOCs and develop learning resources. An untapped domain is harnessing the crowd to systematically process educational data in classrooms -- data that provide key instructional insights but take time to process, such as paper-based assessments. In this paper, we describe an experiment of a crowdsourcing task to effectively process classroom-based data and explore the potential of crowdsourcing as a learning opportunity for the crowdworkers. We discuss implications for designing crowdsourced assessment tasks to yield both high quality output and enriching learning experiences for those involved in the crowdsourcing task.
{"title":"Where's the Learning in Education Crowdsourcing?","authors":"Ha Nguyen, June Ahn, William Young, Fabio Campos","doi":"10.1145/3386527.3406734","DOIUrl":"https://doi.org/10.1145/3386527.3406734","url":null,"abstract":"Crowdsourcing has shown promise in education domains. For example, researchers have leveraged the wisdom of the crowd to process grading in MOOCs and develop learning resources. An untapped domain is harnessing the crowd to systematically process educational data in classrooms -- data that provide key instructional insights but take time to process, such as paper-based assessments. In this paper, we describe an experiment of a crowdsourcing task to effectively process classroom-based data and explore the potential of crowdsourcing as a learning opportunity for the crowdworkers. We discuss implications for designing crowdsourced assessment tasks to yield both high quality output and enriching learning experiences for those involved in the crowdsourcing task.","PeriodicalId":20608,"journal":{"name":"Proceedings of the Seventh ACM Conference on Learning @ Scale","volume":"104 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"75987058","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：