In recent years, non-credit options for learning at scale have outpaced for-credit options. To scale for-credit options, workflows and policies must be devised to preserve the characteristics of accredited higher education---such as the presumption of human evaluation and an assertion of academic integrity---despite increased scale. These efforts must follow as well with shifting from offering isolated courses (or informal collections thereof) to offering full degree programs with additional administrative elements. We see this shift as one from Massive Open Online Courses (MOOCs) to Large, Internet-Mediated Asynchronous Degrees (Limeades). In this work, we perform a qualitative research study on one such program that has scaled to 6,500 students while retaining full accreditation. We report a typology of policies and workflows employed by the individual classes to deliver this experience.
{"title":"Squeezing the limeade: policies and workflows for scalable online degrees","authors":"David A. Joyner","doi":"10.1145/3231644.3231649","DOIUrl":"https://doi.org/10.1145/3231644.3231649","url":null,"abstract":"In recent years, non-credit options for learning at scale have outpaced for-credit options. To scale for-credit options, workflows and policies must be devised to preserve the characteristics of accredited higher education---such as the presumption of human evaluation and an assertion of academic integrity---despite increased scale. These efforts must follow as well with shifting from offering isolated courses (or informal collections thereof) to offering full degree programs with additional administrative elements. We see this shift as one from Massive Open Online Courses (MOOCs) to Large, Internet-Mediated Asynchronous Degrees (Limeades). In this work, we perform a qualitative research study on one such program that has scaled to 6,500 students while retaining full accreditation. We report a typology of policies and workflows employed by the individual classes to deliver this experience.","PeriodicalId":20634,"journal":{"name":"Proceedings of the Fifth Annual ACM Conference on Learning at Scale","volume":"9 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2018-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86743479","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper investigates randomization on asynchronous exams as a defense against collaborative cheating. Asynchronous exams are those for which students take the exam at different times, potentially across a multi-day exam period. Collaborative cheating occurs when one student (the information producer) takes the exam early and passes information about the exam to other students (the information consumers) that are taking the exam later. Using a dataset of computerized exam and homework problems in a single course with 425 students, we identified 5.5% of students (on average) as information consumers by their disproportionate studying of problems that were on the exam. These information consumers ("cheaters") had a significant advantage (13 percentage points on average) when every student was given the same exam problem (even when the parameters are randomized for each student), but that advantage dropped to almost negligible levels (2--3 percentage points) when students were given a random problem from a pool of two or four problems. We conclude that randomization with pools of four (or even three) problems, which also contain randomized parameters, is an effective mitigation for collaborative cheating. Our analysis suggests that this mitigation is in part explained by cheating students having less complete information about larger pools.
{"title":"How much randomization is needed to deter collaborative cheating on asynchronous exams?","authors":"Binglin Chen, Matthew West, C. Zilles","doi":"10.1145/3231644.3231664","DOIUrl":"https://doi.org/10.1145/3231644.3231664","url":null,"abstract":"This paper investigates randomization on asynchronous exams as a defense against collaborative cheating. Asynchronous exams are those for which students take the exam at different times, potentially across a multi-day exam period. Collaborative cheating occurs when one student (the information producer) takes the exam early and passes information about the exam to other students (the information consumers) that are taking the exam later. Using a dataset of computerized exam and homework problems in a single course with 425 students, we identified 5.5% of students (on average) as information consumers by their disproportionate studying of problems that were on the exam. These information consumers (\"cheaters\") had a significant advantage (13 percentage points on average) when every student was given the same exam problem (even when the parameters are randomized for each student), but that advantage dropped to almost negligible levels (2--3 percentage points) when students were given a random problem from a pool of two or four problems. We conclude that randomization with pools of four (or even three) problems, which also contain randomized parameters, is an effective mitigation for collaborative cheating. Our analysis suggests that this mitigation is in part explained by cheating students having less complete information about larger pools.","PeriodicalId":20634,"journal":{"name":"Proceedings of the Fifth Annual ACM Conference on Learning at Scale","volume":"33 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2018-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87612934","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
It often takes years of hands-on practice to build operational problem solving skills for a data scientist to be sufficiently competent to tackle real world problems. In this research, we explore a new scalable technology-enhanced learning (TEL) platform that enables accelerated apprenticeship process via a repository of caselets - small but focused case studies with scaffolding questions and feedback. In this paper, we report rationales of the design, caselet authoring process, and the planned experiment with cohorts of students who will use caselets while taking graduate level data science courses.
{"title":"Accelerated apprenticeship: teaching data science problem solving skills at scale","authors":"Lujie Chen, A. Dubrawski","doi":"10.1145/3231644.3231697","DOIUrl":"https://doi.org/10.1145/3231644.3231697","url":null,"abstract":"It often takes years of hands-on practice to build operational problem solving skills for a data scientist to be sufficiently competent to tackle real world problems. In this research, we explore a new scalable technology-enhanced learning (TEL) platform that enables accelerated apprenticeship process via a repository of caselets - small but focused case studies with scaffolding questions and feedback. In this paper, we report rationales of the design, caselet authoring process, and the planned experiment with cohorts of students who will use caselets while taking graduate level data science courses.","PeriodicalId":20634,"journal":{"name":"Proceedings of the Fifth Annual ACM Conference on Learning at Scale","volume":"21 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2018-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"85975302","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Social media sites are increasingly being adopted to support teaching practice in higher education. Learning Analytics (LA) dashboards can be used to reveal how students engage with course material and others in the class. However, research on the best practices of designing, developing, and evaluating such dashboards to support teaching and learning with social media has been limited. Considering the increasing use of Twitter for both formal and informal learning processes, this paper presents our design process and a LA prototype dashboard developed based on a comprehensive literature review and an online survey among 54 higher education instructors who have used Twitter in their teaching.
{"title":"Designing a learning analytics dashboard for twitter-facilitated teaching","authors":"A. Gruzd, Nadia Conroy","doi":"10.1145/3231644.3231704","DOIUrl":"https://doi.org/10.1145/3231644.3231704","url":null,"abstract":"Social media sites are increasingly being adopted to support teaching practice in higher education. Learning Analytics (LA) dashboards can be used to reveal how students engage with course material and others in the class. However, research on the best practices of designing, developing, and evaluating such dashboards to support teaching and learning with social media has been limited. Considering the increasing use of Twitter for both formal and informal learning processes, this paper presents our design process and a LA prototype dashboard developed based on a comprehensive literature review and an online survey among 54 higher education instructors who have used Twitter in their teaching.","PeriodicalId":20634,"journal":{"name":"Proceedings of the Fifth Annual ACM Conference on Learning at Scale","volume":"46 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2018-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87353373","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Over two iterations of a Massive Open Online Course (MOOC) for school leaders, Launching Innovation in Schools, we developed and tested design elements to support the transfer of online learning into offline action. Effective professional learning is job-embedded: learners should employ new skills and knowledge at work. We aimed to get participants to both plan and actually launch new change efforts, and a subset of our most engaged participants were willing to do so during the course. Assessments, instructor calls to action, and exemplars supported student actions. We found that participants led change initiatives, held stakeholder meetings, collected new data about their contexts, and shared and used course materials collaboratively. Collecting data about participant learning and behavior outside the MOOC environment is essential for researchers and designers looking to create effective online environments for professional learning.
{"title":"From online learning to offline action: using MOOCs for job-embedded teacher professional development","authors":"Alyssa Napier, Elizabeth Huttner-Loan, J. Reich","doi":"10.1145/3231644.3231674","DOIUrl":"https://doi.org/10.1145/3231644.3231674","url":null,"abstract":"Over two iterations of a Massive Open Online Course (MOOC) for school leaders, Launching Innovation in Schools, we developed and tested design elements to support the transfer of online learning into offline action. Effective professional learning is job-embedded: learners should employ new skills and knowledge at work. We aimed to get participants to both plan and actually launch new change efforts, and a subset of our most engaged participants were willing to do so during the course. Assessments, instructor calls to action, and exemplars supported student actions. We found that participants led change initiatives, held stakeholder meetings, collected new data about their contexts, and shared and used course materials collaboratively. Collecting data about participant learning and behavior outside the MOOC environment is essential for researchers and designers looking to create effective online environments for professional learning.","PeriodicalId":20634,"journal":{"name":"Proceedings of the Fifth Annual ACM Conference on Learning at Scale","volume":"84 9","pages":""},"PeriodicalIF":0.0,"publicationDate":"2018-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"91483724","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Learning at scale (LAS) systems like Massive Open Online Classes (MOOCs) have hugely expanded access to high quality educational materials however, such material are frequently time and resource expensive to create. In this work we propose a new approach for automatically and adaptively sequencing practice activities for a particular learner and explore its application for foreign language learning. We evaluate our system through simulation and are in the process of running an experiment. Our simulation results suggest that such an approach may be significantly better than an expert system when there is high variability in the rate of learning among the students and if mastering prerequisites before advancing is important, and is likely to be no worse than an expert system if our generated curriculum approximately describes the necessary structure of learning in students.
{"title":"Combining adaptivity with progression ordering for intelligent tutoring systems","authors":"Tong Mu, Shuhan Wang, Erik Andersen, E. Brunskill","doi":"10.1145/3231644.3231672","DOIUrl":"https://doi.org/10.1145/3231644.3231672","url":null,"abstract":"Learning at scale (LAS) systems like Massive Open Online Classes (MOOCs) have hugely expanded access to high quality educational materials however, such material are frequently time and resource expensive to create. In this work we propose a new approach for automatically and adaptively sequencing practice activities for a particular learner and explore its application for foreign language learning. We evaluate our system through simulation and are in the process of running an experiment. Our simulation results suggest that such an approach may be significantly better than an expert system when there is high variability in the rate of learning among the students and if mastering prerequisites before advancing is important, and is likely to be no worse than an expert system if our generated curriculum approximately describes the necessary structure of learning in students.","PeriodicalId":20634,"journal":{"name":"Proceedings of the Fifth Annual ACM Conference on Learning at Scale","volume":"444 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2018-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"78253091","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Vineet Pandey, Justine W. Debelius, Embriette R. Hyde, T. Kosciólek, Rob Knight, Scott R. Klemmer
Leidraad uit Bonset (2010): Door middel van fictie is het mogelijk kennis te maken met de (fantasie) gedachten van anderen, is het mogelijk om (verzonnen) belevenissen in andere tijden, omgevingen en omstandigheden ‘mee te maken’. Fictie verruimt je blik, leert je nieuwe werelden kennen en stimuleert je fantasie. p. 269 Kerndoel 8: De leerling leert verhalen, gedichten en informatieve teksten te lezen die aan zijn belangstelling tegemoetkomen en zijn belevingswereld uitbreiden. p. 271 Niveau 1F: Kan jeugdliteratuur belevend lezen. Interpreteren: Kan relaties leggen tussen de tekst en de werkelijkheid. Kan spannende, humoristische of dramatische passages in de tekst aanwijzen. Herkent verschillende emoties in de tekst zoals verdriet, boosheid en blijdschap. Evalueren: Evalueert de tekst met emotieve argumenten. Kan met medeleerlingen leeservaringen uitwisselen. Kan interesse in bepaalde fictievormen aangeven. p. 271 – 272 – 273
{"title":"Docent","authors":"Vineet Pandey, Justine W. Debelius, Embriette R. Hyde, T. Kosciólek, Rob Knight, Scott R. Klemmer","doi":"10.1145/3231644.3231646","DOIUrl":"https://doi.org/10.1145/3231644.3231646","url":null,"abstract":"Leidraad uit Bonset (2010): Door middel van fictie is het mogelijk kennis te maken met de (fantasie) gedachten van anderen, is het mogelijk om (verzonnen) belevenissen in andere tijden, omgevingen en omstandigheden ‘mee te maken’. Fictie verruimt je blik, leert je nieuwe werelden kennen en stimuleert je fantasie. p. 269 Kerndoel 8: De leerling leert verhalen, gedichten en informatieve teksten te lezen die aan zijn belangstelling tegemoetkomen en zijn belevingswereld uitbreiden. p. 271 Niveau 1F: Kan jeugdliteratuur belevend lezen. Interpreteren: Kan relaties leggen tussen de tekst en de werkelijkheid. Kan spannende, humoristische of dramatische passages in de tekst aanwijzen. Herkent verschillende emoties in de tekst zoals verdriet, boosheid en blijdschap. Evalueren: Evalueert de tekst met emotieve argumenten. Kan met medeleerlingen leeservaringen uitwisselen. Kan interesse in bepaalde fictievormen aangeven. p. 271 – 272 – 273","PeriodicalId":20634,"journal":{"name":"Proceedings of the Fifth Annual ACM Conference on Learning at Scale","volume":"11 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2018-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"73701997","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Teamwork and collaborative learning are considered superior to learning individually by many instructors and didactical theories. Particularly, in the context of e-learning and Massive Open Online Courses (MOOCs) we see great benefits but also great challenges for both, learners and instructors. We discuss our experience with six team based assignments on the openHPI and openSAP1 MOOC platforms.
{"title":"Team based assignments in MOOCs: results and observations","authors":"T. Staubitz, C. Meinel","doi":"10.1145/3231644.3231705","DOIUrl":"https://doi.org/10.1145/3231644.3231705","url":null,"abstract":"Teamwork and collaborative learning are considered superior to learning individually by many instructors and didactical theories. Particularly, in the context of e-learning and Massive Open Online Courses (MOOCs) we see great benefits but also great challenges for both, learners and instructors. We discuss our experience with six team based assignments on the openHPI and openSAP1 MOOC platforms.","PeriodicalId":20634,"journal":{"name":"Proceedings of the Fifth Annual ACM Conference on Learning at Scale","volume":"28 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2018-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"73497111","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Josh Gardner, Christopher A. Brooks, J. M. Andres, R. Baker
We present a case study in predictive model replication for student dropout in Massive Open Online Courses (MOOCs) using a large and diverse dataset (133 sessions of 28 unique courses offered by two institutions). This experiment was run on the MOOC Replication Framework (MORF), which makes it feasible to fully replicate complex machine learned models, from raw data to model evaluation. We provide an overview of the MORF platform architecture and functionality, and demonstrate its use through a case study. In this replication of [41], we contextualize and evaluate the results of the previous work using statistical tests and a more effective model evaluation scheme. We find that only some of the original findings replicate across this larger and more diverse sample of MOOCs, with others replicating significantly in the opposite direction. Our analysis also reveals results which are highly relevant to the prediction task which were not reported in the original experiment. This work demonstrates the importance of replication of predictive modeling research in MOOCs using large and diverse datasets, illuminates the challenges of doing so, and describes our freely available, open-source software framework to overcome barriers to replication.
{"title":"Replicating MOOC predictive models at scale","authors":"Josh Gardner, Christopher A. Brooks, J. M. Andres, R. Baker","doi":"10.1145/3231644.3231656","DOIUrl":"https://doi.org/10.1145/3231644.3231656","url":null,"abstract":"We present a case study in predictive model replication for student dropout in Massive Open Online Courses (MOOCs) using a large and diverse dataset (133 sessions of 28 unique courses offered by two institutions). This experiment was run on the MOOC Replication Framework (MORF), which makes it feasible to fully replicate complex machine learned models, from raw data to model evaluation. We provide an overview of the MORF platform architecture and functionality, and demonstrate its use through a case study. In this replication of [41], we contextualize and evaluate the results of the previous work using statistical tests and a more effective model evaluation scheme. We find that only some of the original findings replicate across this larger and more diverse sample of MOOCs, with others replicating significantly in the opposite direction. Our analysis also reveals results which are highly relevant to the prediction task which were not reported in the original experiment. This work demonstrates the importance of replication of predictive modeling research in MOOCs using large and diverse datasets, illuminates the challenges of doing so, and describes our freely available, open-source software framework to overcome barriers to replication.","PeriodicalId":20634,"journal":{"name":"Proceedings of the Fifth Annual ACM Conference on Learning at Scale","volume":"20 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2018-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87380501","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Brent Morgan, Andrew J. Hampton, Zhiqiang Cai, Andrew Tackett, Lijia Wang, Xiangen Hu, A. Graesser
ElectronixTutor is a new Intelligent Tutoring System for electronics that integrates multiple intelligent learning resources, including AutoTutor, Dragoon, LearnForm, ASSISTments, and BEETLE-II, as well as Point & Query hotspots on diagrams and numerous text documents on the subject of electronics. ElectronixTutor's student model contains a set of electronics knowledge components (e.g., "transistor behavior"), each of which are taught by multiple learning resources. ElectronixTutor also features a recommender system, which suggests topics and resources for the student to try based on the student model. ElectronixTutor uses a Moodle interface, and is accessible to anyone via a web browser. Currently, ElectronixTutor is being tested by undergraduate electronics students before supplementing Naval Apprentice Technician Training coursework in the fall of 2018.
{"title":"Electronixtutor integrates multiple learning resources to teach electronics on the web","authors":"Brent Morgan, Andrew J. Hampton, Zhiqiang Cai, Andrew Tackett, Lijia Wang, Xiangen Hu, A. Graesser","doi":"10.1145/3231644.3231691","DOIUrl":"https://doi.org/10.1145/3231644.3231691","url":null,"abstract":"ElectronixTutor is a new Intelligent Tutoring System for electronics that integrates multiple intelligent learning resources, including AutoTutor, Dragoon, LearnForm, ASSISTments, and BEETLE-II, as well as Point & Query hotspots on diagrams and numerous text documents on the subject of electronics. ElectronixTutor's student model contains a set of electronics knowledge components (e.g., \"transistor behavior\"), each of which are taught by multiple learning resources. ElectronixTutor also features a recommender system, which suggests topics and resources for the student to try based on the student model. ElectronixTutor uses a Moodle interface, and is accessible to anyone via a web browser. Currently, ElectronixTutor is being tested by undergraduate electronics students before supplementing Naval Apprentice Technician Training coursework in the fall of 2018.","PeriodicalId":20634,"journal":{"name":"Proceedings of the Fifth Annual ACM Conference on Learning at Scale","volume":"42 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2018-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"89305920","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：