Faced with growing class sizes and the dawn of the MOOC, educators are in need of tools to help them cope with the growing number of questions asked in large classes since manually answering all the questions in a timely manner is infeasible. In this paper, we propose to exploit historical question/answer data accumulated for the same or similar classes as a basis for automatically answering previously asked questions via the use of information retrieval techniques. We further propose to leverage resolved questions to create test collections for quantitative evaluation of a question retrieval algorithm without requiring additional human effort. Using this evaluation methodology, we study the effectiveness of state of the art retrieval techniques for this special retrieval task, and perform error analysis to inform future directions.
{"title":"Scaling up Online Question Answering via Similar Question Retrieval","authors":"Chase Geigle, ChengXiang Zhai","doi":"10.1145/2876034.2893428","DOIUrl":"https://doi.org/10.1145/2876034.2893428","url":null,"abstract":"Faced with growing class sizes and the dawn of the MOOC, educators are in need of tools to help them cope with the growing number of questions asked in large classes since manually answering all the questions in a timely manner is infeasible. In this paper, we propose to exploit historical question/answer data accumulated for the same or similar classes as a basis for automatically answering previously asked questions via the use of information retrieval techniques. We further propose to leverage resolved questions to create test collections for quantitative evaluation of a question retrieval algorithm without requiring additional human effort. Using this evaluation methodology, we study the effectiveness of state of the art retrieval techniques for this special retrieval task, and perform error analysis to inform future directions.","PeriodicalId":20739,"journal":{"name":"Proceedings of the Third (2016) ACM Conference on Learning @ Scale","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2016-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"79779279","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Stephen Cummins, A. Stead, Lisa Jardine-Wright, Ian P. Davies, A. Beresford, A. Rice
We investigate the use of hints as a form of scaffolding for 4,652 eligible users on a large-scale online learning environment called Isaac, which allows users to answer physics questions with up to five hints. We investigate user behaviour when using hints, users' engagement with fading (the process of gradually becoming less reliant on the hints provided), and hint strategies including Decomposition, Correction, Verification, or Comparison. Finally, we present recommendations for the design and development of online teaching tools that provide open access to hints, including a mechanism that may improve the speed at which users begin fading.
{"title":"Investigating the Use of Hints in Online Problem Solving","authors":"Stephen Cummins, A. Stead, Lisa Jardine-Wright, Ian P. Davies, A. Beresford, A. Rice","doi":"10.1145/2876034.2893379","DOIUrl":"https://doi.org/10.1145/2876034.2893379","url":null,"abstract":"We investigate the use of hints as a form of scaffolding for 4,652 eligible users on a large-scale online learning environment called Isaac, which allows users to answer physics questions with up to five hints. We investigate user behaviour when using hints, users' engagement with fading (the process of gradually becoming less reliant on the hints provided), and hint strategies including Decomposition, Correction, Verification, or Comparison. Finally, we present recommendations for the design and development of online teaching tools that provide open access to hints, including a mechanism that may improve the speed at which users begin fading.","PeriodicalId":20739,"journal":{"name":"Proceedings of the Third (2016) ACM Conference on Learning @ Scale","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2016-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77119852","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We conducted two studies on the effect of visual reward mechanisms for increasing engagement with an online learning material. In the first study, we studied the effect of showing cat pictures as a reward to correct and incorrect answers to multiple choice questions, and in the second study, we created an illusion of progress using a progress bar that showed step-wise increments as students answered to the questions. Our results show the use of cat pictures as a visual reward mechanism does not significantly increase students' engagement with learning materials. At the same time, students who were shown progress bars had a statistically significant increase in the quantity of answers -- on average 88% more answers per day. However, our results also indicate that this effect declines over time, meaning that students catch up to the illusion.
{"title":"Illusion of Progress is Moar Addictive than Cat Pictures","authors":"Leo Leppänen, Lassi Vapaakallio, Arto Vihavainen","doi":"10.1145/2876034.2893388","DOIUrl":"https://doi.org/10.1145/2876034.2893388","url":null,"abstract":"We conducted two studies on the effect of visual reward mechanisms for increasing engagement with an online learning material. In the first study, we studied the effect of showing cat pictures as a reward to correct and incorrect answers to multiple choice questions, and in the second study, we created an illusion of progress using a progress bar that showed step-wise increments as students answered to the questions. Our results show the use of cat pictures as a visual reward mechanism does not significantly increase students' engagement with learning materials. At the same time, students who were shown progress bars had a statistically significant increase in the quantity of answers -- on average 88% more answers per day. However, our results also indicate that this effect declines over time, meaning that students catch up to the illusion.","PeriodicalId":20739,"journal":{"name":"Proceedings of the Third (2016) ACM Conference on Learning @ Scale","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2016-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"74883009","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Online courses on sites such as Coursera use quizzes embedded inside lecture videos (in-video quizzes) to help learners test their understanding of the video. This paper analyzes how users interact with in-video quizzes, and how in-video quizzes influence users' lecture viewing behavior. We analyze the viewing logs of users who took the Machine Learning course on Coursera. Users engage heavily with in-video quizzes -- 74% of viewers who start watching a video will attempt its corresponding in-video quiz. We observe spikes in seek activity surrounding in-video quizzes, particularly seeks from the in-video quiz to the preceding section. We show that this is likely due to users reviewing the preceding section to help them answer the quiz, as the majority of users who seek backwards from in-video quizzes have not yet submitted a correct answer, but will later attempt the quiz. Some users appear to use quiz-oriented navigation strategies, such as seeking directly from the start of the video to in-video quizzes, or skipping from one quiz to the next. We discuss implications of our findings on the design of lecture-viewing platforms.
{"title":"Effects of In-Video Quizzes on MOOC Lecture Viewing","authors":"G. Kovács","doi":"10.1145/2876034.2876041","DOIUrl":"https://doi.org/10.1145/2876034.2876041","url":null,"abstract":"Online courses on sites such as Coursera use quizzes embedded inside lecture videos (in-video quizzes) to help learners test their understanding of the video. This paper analyzes how users interact with in-video quizzes, and how in-video quizzes influence users' lecture viewing behavior. We analyze the viewing logs of users who took the Machine Learning course on Coursera. Users engage heavily with in-video quizzes -- 74% of viewers who start watching a video will attempt its corresponding in-video quiz. We observe spikes in seek activity surrounding in-video quizzes, particularly seeks from the in-video quiz to the preceding section. We show that this is likely due to users reviewing the preceding section to help them answer the quiz, as the majority of users who seek backwards from in-video quizzes have not yet submitted a correct answer, but will later attempt the quiz. Some users appear to use quiz-oriented navigation strategies, such as seeking directly from the start of the video to in-video quizzes, or skipping from one quiz to the next. We discuss implications of our findings on the design of lecture-viewing platforms.","PeriodicalId":20739,"journal":{"name":"Proceedings of the Third (2016) ACM Conference on Learning @ Scale","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2016-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"74866795","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jan Renz, Gerado Navarro-Suarez, Rowshan Sathi, T. Staubitz, C. Meinel
This paper at hand describes the design and implementation of an analytics service to retrieve live usage data from students enrolled in a service-oriented MOOC platform for the purpose of learning analytics (LA) research. A real-time and extensible architecture for consolidating and processing data in versatile analytics stores is introduced.
{"title":"Enabling Schema Agnostic Learning Analytics in a Service-Oriented MOOC Platform","authors":"Jan Renz, Gerado Navarro-Suarez, Rowshan Sathi, T. Staubitz, C. Meinel","doi":"10.1145/2876034.2893389","DOIUrl":"https://doi.org/10.1145/2876034.2893389","url":null,"abstract":"This paper at hand describes the design and implementation of an analytics service to retrieve live usage data from students enrolled in a service-oriented MOOC platform for the purpose of learning analytics (LA) research. A real-time and extensible architecture for consolidating and processing data in versatile analytics stores is introduced.","PeriodicalId":20739,"journal":{"name":"Proceedings of the Third (2016) ACM Conference on Learning @ Scale","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2016-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"74841492","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
V. Aleven, R. Baker, Y. Wang, J. Sewall, Octav Popescu
Learning-by-doing in MOOCs may be enhanced by embedding intelligent tutoring systems (ITSs). ITSs support learning-by-doing by guiding learners through complex practice problems while adapting to differences among learners. We extended the Cognitive Tutor Authoring Tools (CTAT), a widely-used non-programmer tool kit for building intelligent tutors, so that CTAT-built tutors can be embedded in MOOCs and e-learning platforms. We demonstrated the technical feasibility of this integration by adding simple CTAT-built tutors to an edX MOOC, "Big Data in Education." To the best of our knowledge, this integration is the first occasion that material created through an open-access non-programmer authoring tool for full-fledged ITS has been integrated in a MOOC. The work offers examples of key steps that may be useful in other ITS-MOOC integration efforts, together with reflections on strengths, weaknesses, and future possibilities.
{"title":"Bringing Non-programmer Authoring of Intelligent Tutors to MOOCs","authors":"V. Aleven, R. Baker, Y. Wang, J. Sewall, Octav Popescu","doi":"10.1145/2876034.2893442","DOIUrl":"https://doi.org/10.1145/2876034.2893442","url":null,"abstract":"Learning-by-doing in MOOCs may be enhanced by embedding intelligent tutoring systems (ITSs). ITSs support learning-by-doing by guiding learners through complex practice problems while adapting to differences among learners. We extended the Cognitive Tutor Authoring Tools (CTAT), a widely-used non-programmer tool kit for building intelligent tutors, so that CTAT-built tutors can be embedded in MOOCs and e-learning platforms. We demonstrated the technical feasibility of this integration by adding simple CTAT-built tutors to an edX MOOC, \"Big Data in Education.\" To the best of our knowledge, this integration is the first occasion that material created through an open-access non-programmer authoring tool for full-fledged ITS has been integrated in a MOOC. The work offers examples of key steps that may be useful in other ITS-MOOC integration efforts, together with reflections on strengths, weaknesses, and future possibilities.","PeriodicalId":20739,"journal":{"name":"Proceedings of the Third (2016) ACM Conference on Learning @ Scale","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2016-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"75746119","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. Reich, Brandon M Stewart, Kimia Mavon, D. Tingley
In this study, we develop methods for computationally measuring the degree to which students engage in MOOC forums with other students holding different political beliefs. We examine a case study of a single MOOC about education policy, Saving Schools, where we obtain measures of student education policy preferences that correlate with political ideology. Contrary to assertions that online spaces often become echo chambers or ideological silos, we find that students in this case hold diverse political beliefs, participate equitably in forum discussions, directly engage (through replies and upvotes) with students holding opposing beliefs, and converge on a shared language rather than talking past one another. Research that focuses on the civic mission of MOOCs helps ensure that open online learning engages the same breadth of purposes that higher education aspires to serve.
{"title":"The Civic Mission of MOOCs: Measuring Engagement across Political Differences in Forums","authors":"J. Reich, Brandon M Stewart, Kimia Mavon, D. Tingley","doi":"10.1145/2876034.2876045","DOIUrl":"https://doi.org/10.1145/2876034.2876045","url":null,"abstract":"In this study, we develop methods for computationally measuring the degree to which students engage in MOOC forums with other students holding different political beliefs. We examine a case study of a single MOOC about education policy, Saving Schools, where we obtain measures of student education policy preferences that correlate with political ideology. Contrary to assertions that online spaces often become echo chambers or ideological silos, we find that students in this case hold diverse political beliefs, participate equitably in forum discussions, directly engage (through replies and upvotes) with students holding opposing beliefs, and converge on a shared language rather than talking past one another. Research that focuses on the civic mission of MOOCs helps ensure that open online learning engages the same breadth of purposes that higher education aspires to serve.","PeriodicalId":20739,"journal":{"name":"Proceedings of the Third (2016) ACM Conference on Learning @ Scale","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2016-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77480837","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
C. Breazeal, R. Morris, S. Gottwald, Tinsley A. Galyean, M. Wolf
Extensive work focuses on the uses of technology at scale for post-literate populations (e.g., MOOC, learning games, Learning Management Systems). Little attention is afforded to non-literate populations, particularly in the developing world. This paper presents an approach using mobile devices with the ultimate goal to reach 770 million people. We developed a novel platform with a cloud backend to deliver educational content to over a thousand marginalized children in different countries: specifically, in remote villages without schools, urban slums with overcrowded schools, and at-risk, rural schools. Here we describe the theoretical basis of our system and results from case studies in three educational contexts. This model will help researchers and designers understand how mobile devices can help children acquire basic skills and aid each other's learning when the benefit of teachers is limited or non-existent.
{"title":"Mobile Devices for Early Literacy Intervention and Research with Global Reach","authors":"C. Breazeal, R. Morris, S. Gottwald, Tinsley A. Galyean, M. Wolf","doi":"10.1145/2876034.2876046","DOIUrl":"https://doi.org/10.1145/2876034.2876046","url":null,"abstract":"Extensive work focuses on the uses of technology at scale for post-literate populations (e.g., MOOC, learning games, Learning Management Systems). Little attention is afforded to non-literate populations, particularly in the developing world. This paper presents an approach using mobile devices with the ultimate goal to reach 770 million people. We developed a novel platform with a cloud backend to deliver educational content to over a thousand marginalized children in different countries: specifically, in remote villages without schools, urban slums with overcrowded schools, and at-risk, rural schools. Here we describe the theoretical basis of our system and results from case studies in three educational contexts. This model will help researchers and designers understand how mobile devices can help children acquire basic skills and aid each other's learning when the benefit of teachers is limited or non-existent.","PeriodicalId":20739,"journal":{"name":"Proceedings of the Third (2016) ACM Conference on Learning @ Scale","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2016-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82309557","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We will demonstrate the Beetle-Grow intelligent tutoring system, which combines active experimentation, self-explanation, and formative feedback using natural language interaction. It runs in a standard web browser and has a fresh, engaging design. The underlying back-end system has previously been shown to be highly effective in teaching basic electricity and electronics concepts. Beetle-Grow has been designed to capture student interaction and indicators of learning in a form suitable for data mining, and to support future work on building tools for interactive tutoring that improve after experiencing interaction with students, as human tutors do. We are interested in partnering with teachers and other education researchers to carry out large-scale user trials with Beetle-Grow in the classroom and remotely.
{"title":"Beetle-Grow: An Effective Intelligent Tutoring System to Support Conceptual Change","authors":"Elaine Farrow, Johanna D. Moore","doi":"10.1145/2876034.2893403","DOIUrl":"https://doi.org/10.1145/2876034.2893403","url":null,"abstract":"We will demonstrate the Beetle-Grow intelligent tutoring system, which combines active experimentation, self-explanation, and formative feedback using natural language interaction. It runs in a standard web browser and has a fresh, engaging design. The underlying back-end system has previously been shown to be highly effective in teaching basic electricity and electronics concepts. Beetle-Grow has been designed to capture student interaction and indicators of learning in a form suitable for data mining, and to support future work on building tools for interactive tutoring that improve after experiencing interaction with students, as human tutors do. We are interested in partnering with teachers and other education researchers to carry out large-scale user trials with Beetle-Grow in the classroom and remotely.","PeriodicalId":20739,"journal":{"name":"Proceedings of the Third (2016) ACM Conference on Learning @ Scale","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2016-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"85384269","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Many ongoing efforts in online education aim to increase accessibility through affordability and flexibility, but some critics have noted that pedagogy often suffers during these efforts. In contrast, in the low-cost for-credit Georgia Tech Online Masters of Science in Computer Science (OMSCS) program, we have observed that the features that make the program accessible also lead to pedagogical benefits. In this paper, we discuss the pedagogical benefits, and draw a causal link between those benefits and the factors that increase the program's accessibility.
{"title":"The Unexpected Pedagogical Benefits of Making Higher Education Accessible","authors":"David A. Joyner, Ashok K. Goel, C. Isbell","doi":"10.1145/2876034.2893383","DOIUrl":"https://doi.org/10.1145/2876034.2893383","url":null,"abstract":"Many ongoing efforts in online education aim to increase accessibility through affordability and flexibility, but some critics have noted that pedagogy often suffers during these efforts. In contrast, in the low-cost for-credit Georgia Tech Online Masters of Science in Computer Science (OMSCS) program, we have observed that the features that make the program accessible also lead to pedagogical benefits. In this paper, we discuss the pedagogical benefits, and draw a causal link between those benefits and the factors that increase the program's accessibility.","PeriodicalId":20739,"journal":{"name":"Proceedings of the Third (2016) ACM Conference on Learning @ Scale","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2016-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"85661080","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：