Jihyun Park, K. Denaro, F. Rodriguez, Padhraic Smyth, M. Warschauer
Student clickstream data can provide valuable insights about student activities in an online learning environment and how these activities inform their learning outcomes. However, given the noisy and complex nature of this data, an on-going challenge involves devising statistical techniques that capture clear and meaningful aspects of students' click patterns. In this paper, we utilize statistical change detection techniques to investigate students' online behaviors. Using clickstream data from two large university courses, one face-to-face and one online, we illustrate how this methodology can be used to detect when students change their previewing and reviewing behavior, and how these changes can be related to other aspects of students' activity and performance.
{"title":"Detecting changes in student behavior from clickstream data","authors":"Jihyun Park, K. Denaro, F. Rodriguez, Padhraic Smyth, M. Warschauer","doi":"10.1145/3027385.3027430","DOIUrl":"https://doi.org/10.1145/3027385.3027430","url":null,"abstract":"Student clickstream data can provide valuable insights about student activities in an online learning environment and how these activities inform their learning outcomes. However, given the noisy and complex nature of this data, an on-going challenge involves devising statistical techniques that capture clear and meaningful aspects of students' click patterns. In this paper, we utilize statistical change detection techniques to investigate students' online behaviors. Using clickstream data from two large university courses, one face-to-face and one online, we illustrate how this methodology can be used to detect when students change their previewing and reviewing behavior, and how these changes can be related to other aspects of students' activity and performance.","PeriodicalId":160897,"journal":{"name":"Proceedings of the Seventh International Learning Analytics & Knowledge Conference","volume":"27 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124969947","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. Schulte, Pedro Fernandez de Mendonca, Roberto Martínez Maldonado, S. B. Shum
For students, in particular freshmen, the degree pathway from semester to semester is not that transparent, although students have a reasonable idea what courses are expected to be taken each semester. An often-pondered question by students is: "what can I expect in the next semester?" More precisely, given the commitment and engagement I presented in this particular course and the respective performance I achieved, can I expect a similar outcome in the next semester in the particular course I selected? Are the demands and expectations in this course much higher so that I need to adjust my commitment and engagement and overall workload if I expect a similar outcome? Is it better to drop a course to manage expectations rather than to (predictably) fail, and perhaps have to leave the degree altogether? Degree and course advisors and student support units find it challenging to provide evidence based advise to students. This paper presents research into educational process mining and student data analytics in a whole university scale approach with the aim of providing insight into the degree pathway questions raised above. The beta-version of our course level degree pathway tool has been used to shed light for university staff and students alike into our university's 1,300 degrees and associated 6 million course enrolments over the past 20 years.
{"title":"Large scale predictive process mining and analytics of university degree course data","authors":"J. Schulte, Pedro Fernandez de Mendonca, Roberto Martínez Maldonado, S. B. Shum","doi":"10.1145/3027385.3029446","DOIUrl":"https://doi.org/10.1145/3027385.3029446","url":null,"abstract":"For students, in particular freshmen, the degree pathway from semester to semester is not that transparent, although students have a reasonable idea what courses are expected to be taken each semester. An often-pondered question by students is: \"what can I expect in the next semester?\" More precisely, given the commitment and engagement I presented in this particular course and the respective performance I achieved, can I expect a similar outcome in the next semester in the particular course I selected? Are the demands and expectations in this course much higher so that I need to adjust my commitment and engagement and overall workload if I expect a similar outcome? Is it better to drop a course to manage expectations rather than to (predictably) fail, and perhaps have to leave the degree altogether? Degree and course advisors and student support units find it challenging to provide evidence based advise to students. This paper presents research into educational process mining and student data analytics in a whole university scale approach with the aim of providing insight into the degree pathway questions raised above. The beta-version of our course level degree pathway tool has been used to shed light for university staff and students alike into our university's 1,300 degrees and associated 6 million course enrolments over the past 20 years.","PeriodicalId":160897,"journal":{"name":"Proceedings of the Seventh International Learning Analytics & Knowledge Conference","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125423968","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A mixed methods approach was undertaken in this exploratory study to better understand how learners perceive and utilize learning analytics visualizations during online discussions activities. Internal conditions such as goal orientation and numeracy were measured alongside the external conditions created by the discussion structure and learning analytics. Our results emphasize key factors that should be considered when designing learning analytics tools.
{"title":"Best intentions: learner feedback on learning analytics visualization design","authors":"Halimat I. Alabi, M. Hatala","doi":"10.1145/3027385.3029487","DOIUrl":"https://doi.org/10.1145/3027385.3029487","url":null,"abstract":"A mixed methods approach was undertaken in this exploratory study to better understand how learners perceive and utilize learning analytics visualizations during online discussions activities. Internal conditions such as goal orientation and numeracy were measured alongside the external conditions created by the discussion structure and learning analytics. Our results emphasize key factors that should be considered when designing learning analytics tools.","PeriodicalId":160897,"journal":{"name":"Proceedings of the Seventh International Learning Analytics & Knowledge Conference","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125819238","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Intelligent tutoring systems (ITSs) are commonly designed to enhance student learning. However, they are not typically designed to meet the needs of teachers who use them in their classrooms. ITSs generate a wealth of analytics about student learning and behavior, opening a rich design space for real-time teacher support tools such as dashboards. Whereas real-time dashboards for teachers have become popular with many learning technologies, we are not aware of projects that have designed dashboards for ITSs based on a broad investigation of teachers' needs. We conducted design interviews with ten middle school math teachers to explore their needs for on-the-spot support during blended class sessions, as a first step in a user-centered design process of a real-time dashboard. Based on multi-methods analyses of this interview data, we identify several opportunities for ITSs to better support teachers' needs, noting that the analytics commonly generated by existing teacher support tools do not strongly align with the analytics teachers expect to be most useful. We highlight key tensions and tradeoffs in the design of such real-time supports for teachers, as revealed by "Speed Dating" possible futures with teachers. This paper has implications for our ongoing co-design of a real-time dashboard for ITSs, as well as broader implications for the design of ITSs that can effectively collaborate with teachers in classroom settings.
{"title":"Intelligent tutors as teachers' aides: exploring teacher needs for real-time analytics in blended classrooms","authors":"Kenneth Holstein, B. McLaren, V. Aleven","doi":"10.1145/3027385.3027451","DOIUrl":"https://doi.org/10.1145/3027385.3027451","url":null,"abstract":"Intelligent tutoring systems (ITSs) are commonly designed to enhance student learning. However, they are not typically designed to meet the needs of teachers who use them in their classrooms. ITSs generate a wealth of analytics about student learning and behavior, opening a rich design space for real-time teacher support tools such as dashboards. Whereas real-time dashboards for teachers have become popular with many learning technologies, we are not aware of projects that have designed dashboards for ITSs based on a broad investigation of teachers' needs. We conducted design interviews with ten middle school math teachers to explore their needs for on-the-spot support during blended class sessions, as a first step in a user-centered design process of a real-time dashboard. Based on multi-methods analyses of this interview data, we identify several opportunities for ITSs to better support teachers' needs, noting that the analytics commonly generated by existing teacher support tools do not strongly align with the analytics teachers expect to be most useful. We highlight key tensions and tradeoffs in the design of such real-time supports for teachers, as revealed by \"Speed Dating\" possible futures with teachers. This paper has implications for our ongoing co-design of a real-time dashboard for ITSs, as well as broader implications for the design of ITSs that can effectively collaborate with teachers in classroom settings.","PeriodicalId":160897,"journal":{"name":"Proceedings of the Seventh International Learning Analytics & Knowledge Conference","volume":"34 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126028036","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Josh Gardner, Ogechi Onuoha, Christopher A. Brooks
In this work, we present (1) a methodology for collecting, evaluating, and utilizing human-annotated data about course syllabi in predictive models of student success, and (2) an empirical analysis of the predictiveness of such features as they relate to others in modeling end-of-course grades in traditional higher education courses. We present a two-stage approach to (1) that addresses several challenges unique to the annotation task, and address (2) using variable importance metrics from a series of exploratory models. We demonstrate that the process of supplementing traditional course data with human-annotated data can potentially improve predictive models with information not contained in university records, and highlight specific features that demonstrate these potential information gains.
{"title":"Integrating syllabus data into student success models","authors":"Josh Gardner, Ogechi Onuoha, Christopher A. Brooks","doi":"10.1145/3027385.3029473","DOIUrl":"https://doi.org/10.1145/3027385.3029473","url":null,"abstract":"In this work, we present (1) a methodology for collecting, evaluating, and utilizing human-annotated data about course syllabi in predictive models of student success, and (2) an empirical analysis of the predictiveness of such features as they relate to others in modeling end-of-course grades in traditional higher education courses. We present a two-stage approach to (1) that addresses several challenges unique to the annotation task, and address (2) using variable importance metrics from a series of exploratory models. We demonstrate that the process of supplementing traditional course data with human-annotated data can potentially improve predictive models with information not contained in university records, and highlight specific features that demonstrate these potential information gains.","PeriodicalId":160897,"journal":{"name":"Proceedings of the Seventh International Learning Analytics & Knowledge Conference","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130163518","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Nicholas Diana, Michael Eagle, John C. Stamper, Shuchi Grover, M. Bienkowski, Satabdi Basu
Many introductory programming environments generate a large amount of log data, but making insights from these data accessible to instructors remains a challenge. This research demonstrates that student outcomes can be accurately predicted from student program states at various time points throughout the course, and integrates the resulting predictive models into an instructor dashboard. The effectiveness of the dashboard is evaluated by measuring how well the dashboard analytics correctly suggest that the instructor help students classified as most in need. Finally, we describe a method of matching low-performing students with high-performing peer tutors, and show that the inclusion of peer tutors not only increases the amount of help given, but the consistency of help availability as well.
{"title":"An instructor dashboard for real-time analytics in interactive programming assignments","authors":"Nicholas Diana, Michael Eagle, John C. Stamper, Shuchi Grover, M. Bienkowski, Satabdi Basu","doi":"10.1145/3027385.3027441","DOIUrl":"https://doi.org/10.1145/3027385.3027441","url":null,"abstract":"Many introductory programming environments generate a large amount of log data, but making insights from these data accessible to instructors remains a challenge. This research demonstrates that student outcomes can be accurately predicted from student program states at various time points throughout the course, and integrates the resulting predictive models into an instructor dashboard. The effectiveness of the dashboard is evaluated by measuring how well the dashboard analytics correctly suggest that the instructor help students classified as most in need. Finally, we describe a method of matching low-performing students with high-performing peer tutors, and show that the inclusion of peer tutors not only increases the amount of help given, but the consistency of help availability as well.","PeriodicalId":160897,"journal":{"name":"Proceedings of the Seventh International Learning Analytics & Knowledge Conference","volume":"201 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116430356","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Fumiya Okubo, Takayoshi Yamashita, Atsushi Shimada, H. Ogata
In this paper, we propose a method for predicting final grades of students by a Recurrent Neural Network (RNN) from the log data stored in the educational systems. We applied this method to the log data from 108 students and examined the accuracy of prediction. From the experimental results, comparing with multiple regression analysis, it is confirmed that an RNN is effective to early prediction of final grades.
{"title":"A neural network approach for students' performance prediction","authors":"Fumiya Okubo, Takayoshi Yamashita, Atsushi Shimada, H. Ogata","doi":"10.1145/3027385.3029479","DOIUrl":"https://doi.org/10.1145/3027385.3029479","url":null,"abstract":"In this paper, we propose a method for predicting final grades of students by a Recurrent Neural Network (RNN) from the log data stored in the educational systems. We applied this method to the log data from 108 students and examined the accuracy of prediction. From the experimental results, comparing with multiple regression analysis, it is confirmed that an RNN is effective to early prediction of final grades.","PeriodicalId":160897,"journal":{"name":"Proceedings of the Seventh International Learning Analytics & Knowledge Conference","volume":"52 1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126921021","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The facilitation of interpersonal relationships within a respectful learning climate is an important aspect of teaching practice. However, in large-scale online contexts, such as MOOCs, the number of learners and highly asynchronous nature militates against the development of a sense of belonging and dyadic trust. Given these challenges, instead of conventional instruments that reflect learners' affective perceptions, we suggest a set of indicators that can be used to evaluate social activity in relation to the participation structure. These group-level indicators can then help teachers to gain insights into the evolution of social activity shaped by their facilitation choices. For this study, group-level indicators were derived from measuring information exchange activity between the returning MOOC posters. By conceptualizing this group as an identity-based community, we can apply exponential random graph modelling to explain the network's structure through the configurations of direct reciprocity, triadic-level exchange, and the effect of participants demonstrating super-posting behavior. The findings provide novel insights into network amplification, and highlight the differences between the courses with different facilitation strategies. Direct reciprocation was characteristic of non-facilitated groups. Exchange at the level of triads was more prominent in highly facilitated online communities with instructor's involvement. Super-posting activity was less pronounced in networks with higher triadic exchange, and more pronounced in networks with higher direct reciprocity.
{"title":"How effective is your facilitation?: group-level analytics of MOOC forums","authors":"Oleksandra Poquet, S. Dawson, Nia Dowell","doi":"10.1145/3027385.3027404","DOIUrl":"https://doi.org/10.1145/3027385.3027404","url":null,"abstract":"The facilitation of interpersonal relationships within a respectful learning climate is an important aspect of teaching practice. However, in large-scale online contexts, such as MOOCs, the number of learners and highly asynchronous nature militates against the development of a sense of belonging and dyadic trust. Given these challenges, instead of conventional instruments that reflect learners' affective perceptions, we suggest a set of indicators that can be used to evaluate social activity in relation to the participation structure. These group-level indicators can then help teachers to gain insights into the evolution of social activity shaped by their facilitation choices. For this study, group-level indicators were derived from measuring information exchange activity between the returning MOOC posters. By conceptualizing this group as an identity-based community, we can apply exponential random graph modelling to explain the network's structure through the configurations of direct reciprocity, triadic-level exchange, and the effect of participants demonstrating super-posting behavior. The findings provide novel insights into network amplification, and highlight the differences between the courses with different facilitation strategies. Direct reciprocation was characteristic of non-facilitated groups. Exchange at the level of triads was more prominent in highly facilitated online communities with instructor's involvement. Super-posting activity was less pronounced in networks with higher triadic exchange, and more pronounced in networks with higher direct reciprocity.","PeriodicalId":160897,"journal":{"name":"Proceedings of the Seventh International Learning Analytics & Knowledge Conference","volume":"59 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114896523","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Bodong Chen, Yizhou Fan, Guogang Zhang, Qiong Wang
The present study examines behavioral patterns, motivations, and self-regulated learning strategies of returning learners---a special learner subpopulation in massive open online courses (MOOCs). To this end, data were collected from a teacher professional development MOOC that has been offered for seven iterations during 2014--2016. Data analysis identified more than 15% of all registrants as returning learners. Findings from click log analysis identified possible motivations of re-enrollment including improving grades, refreshing theoretical understanding, and solving practical problems. Further analysis uncovered evidence of self-regulated learning strategies among returning learners. Taken together, this study contributes to ongoing inquiry into MOOCs learning pathways, informs future MOOC design, and sheds light on the exploration of MOOCs as a viable option for teacher professional development.
{"title":"Examining motivations and self-regulated learning strategies of returning MOOCs learners","authors":"Bodong Chen, Yizhou Fan, Guogang Zhang, Qiong Wang","doi":"10.1145/3027385.3029448","DOIUrl":"https://doi.org/10.1145/3027385.3029448","url":null,"abstract":"The present study examines behavioral patterns, motivations, and self-regulated learning strategies of returning learners---a special learner subpopulation in massive open online courses (MOOCs). To this end, data were collected from a teacher professional development MOOC that has been offered for seven iterations during 2014--2016. Data analysis identified more than 15% of all registrants as returning learners. Findings from click log analysis identified possible motivations of re-enrollment including improving grades, refreshing theoretical understanding, and solving practical problems. Further analysis uncovered evidence of self-regulated learning strategies among returning learners. Taken together, this study contributes to ongoing inquiry into MOOCs learning pathways, informs future MOOC design, and sheds light on the exploration of MOOCs as a viable option for teacher professional development.","PeriodicalId":160897,"journal":{"name":"Proceedings of the Seventh International Learning Analytics & Knowledge Conference","volume":"38 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130510573","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Quality assurance in any organization is important for ensuring that service users are satisfied with the service offered. For higher education institutes, the use of service quality measures allows for ideological gaps to be both identified and resolved. The learning analytic community, however, has rarely addressed the concept of service quality. A potential outcome of this is the provision of a learning analytics service that only meets the expectations of certain stakeholders (e.g., managers), whilst overlooking those who are most important (e.g., students). In order to resolve this issue, we outline a framework and our current progress towards developing a scale to assess student expectations and perceptions of learning analytics as a service.
{"title":"What do students want?: towards an instrument for students' evaluation of quality of learning analytics services","authors":"A. Whitelock-Wainwright, D. Gašević, R. Tejeiro","doi":"10.1145/3027385.3027419","DOIUrl":"https://doi.org/10.1145/3027385.3027419","url":null,"abstract":"Quality assurance in any organization is important for ensuring that service users are satisfied with the service offered. For higher education institutes, the use of service quality measures allows for ideological gaps to be both identified and resolved. The learning analytic community, however, has rarely addressed the concept of service quality. A potential outcome of this is the provision of a learning analytics service that only meets the expectations of certain stakeholders (e.g., managers), whilst overlooking those who are most important (e.g., students). In order to resolve this issue, we outline a framework and our current progress towards developing a scale to assess student expectations and perceptions of learning analytics as a service.","PeriodicalId":160897,"journal":{"name":"Proceedings of the Seventh International Learning Analytics & Knowledge Conference","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132371051","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：