The increased adoption of online learning environments has resulted in the availability of vast amounts of educationallog data, which raises questions that could be answered by a thorough and accurate examination of students’ onlinelearning behaviours. Event logs describe something that occurred on a platform and provide multiple dimensionsthat help to characterize what actions students take, when, and where (in which course and in which part of thecourse). Temporal analysis has been shown to be relevant in learning analytics (LA) research, and capturingtime-on-task as a proxy to model learning behaviour, predict performance, and prevent drop-out has been thesubject of several studies. In Moodle, one of the most used learning management systems, while most events arelogged at their beginning, other events are recorded at their end. The duration of an event is usually calculated asthe difference between two consecutive records assuming that a log records the action’s starting time. Therefore,when an event is logged at its end, the difference between the starting and the ending event identifies their sum,not the duration of the first. Moreover, in the pursuit of a better user experience, increasingly more online learningplatforms’ functions are shifted to the client, with the unintended effect of reducing significant logs and conceivablymisinterpreting student behaviour. The purpose of this study is to present Moodle’s logging system to illustratewhere the temporal dimension of Moodle log data can be difficult to interpret and how this knowledge can be usedto improve data processing. Starting from the correct extraction of Moodle logs, we focus on factors to considerwhen preparing data for temporal dimensional analysis. Considering the significance of the correct interpretation oflog data to the LA community, we intend to initiate a discussion on this domain understanding to prevent the loss ofdata-related knowledge.
{"title":"Processing and Understanding Moodle Log Data and Their Temporal Dimension","authors":"D. Rotelli, A. Monreale","doi":"10.18608/jla.2023.7867","DOIUrl":"https://doi.org/10.18608/jla.2023.7867","url":null,"abstract":"The increased adoption of online learning environments has resulted in the availability of vast amounts of educationallog data, which raises questions that could be answered by a thorough and accurate examination of students’ onlinelearning behaviours. Event logs describe something that occurred on a platform and provide multiple dimensionsthat help to characterize what actions students take, when, and where (in which course and in which part of thecourse). Temporal analysis has been shown to be relevant in learning analytics (LA) research, and capturingtime-on-task as a proxy to model learning behaviour, predict performance, and prevent drop-out has been thesubject of several studies. In Moodle, one of the most used learning management systems, while most events arelogged at their beginning, other events are recorded at their end. The duration of an event is usually calculated asthe difference between two consecutive records assuming that a log records the action’s starting time. Therefore,when an event is logged at its end, the difference between the starting and the ending event identifies their sum,not the duration of the first. Moreover, in the pursuit of a better user experience, increasingly more online learningplatforms’ functions are shifted to the client, with the unintended effect of reducing significant logs and conceivablymisinterpreting student behaviour. The purpose of this study is to present Moodle’s logging system to illustratewhere the temporal dimension of Moodle log data can be difficult to interpret and how this knowledge can be usedto improve data processing. Starting from the correct extraction of Moodle logs, we focus on factors to considerwhen preparing data for temporal dimensional analysis. Considering the significance of the correct interpretation oflog data to the LA community, we intend to initiate a discussion on this domain understanding to prevent the loss ofdata-related knowledge.","PeriodicalId":36754,"journal":{"name":"Journal of Learning Analytics","volume":" ","pages":""},"PeriodicalIF":3.9,"publicationDate":"2023-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45144839","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Learning analytics dashboard (LAD) development has been criticized for being too data-driven and for developers lacking an understanding of the nontechnical aspects of learning analytics (LA). The ability of developers to address their understanding of learners as well as systematic efforts to involve students in the development process are central to creating pedagogically grounded student-facing dashboards. However, limited research is available about developer perceptions on supporting students with LA. We examined an interdisciplinary LA development team’s (IDT) perceptions of and intentions to support student agency, and the student-facing LAD development process. Qualitative content analysis supported by a social cognitive theory framework was conducted on interviews (N = 12) to analyze the IDT’s perceptions of student agency. IDT members had differing conceptions of student agency but agreed that it manifests in strategic study progression and planning, as well as in active interpretation and use of LA-based feedback. IDT members had differing views on student involvement in the LAD development process. Communication challenges within an IDT and limited resources were mentioned, impeding development work. The results of this study highlight the importance of fostering communication among IDT members about guiding pedagogical design principles and the systematic use of educational concepts in LA development processes.
{"title":"Supporting Student Agency with a Student-Facing Learning Analytics Dashboard","authors":"Anceli Kaveri, Anni Silvola, H. Muukkonen","doi":"10.18608/jla.2023.7729","DOIUrl":"https://doi.org/10.18608/jla.2023.7729","url":null,"abstract":"Learning analytics dashboard (LAD) development has been criticized for being too data-driven and for developers lacking an understanding of the nontechnical aspects of learning analytics (LA). The ability of developers to address their understanding of learners as well as systematic efforts to involve students in the development process are central to creating pedagogically grounded student-facing dashboards. However, limited research is available about developer perceptions on supporting students with LA. We examined an interdisciplinary LA development team’s (IDT) perceptions of and intentions to support student agency, and the student-facing LAD development process. Qualitative content analysis supported by a social cognitive theory framework was conducted on interviews (N = 12) to analyze the IDT’s perceptions of student agency. IDT members had differing conceptions of student agency but agreed that it manifests in strategic study progression and planning, as well as in active interpretation and use of LA-based feedback. IDT members had differing views on student involvement in the LAD development process. Communication challenges within an IDT and limited resources were mentioned, impeding development work. The results of this study highlight the importance of fostering communication among IDT members about guiding pedagogical design principles and the systematic use of educational concepts in LA development processes.","PeriodicalId":36754,"journal":{"name":"Journal of Learning Analytics","volume":" ","pages":""},"PeriodicalIF":3.9,"publicationDate":"2023-08-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46953471","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
D. Fleur, M. Marshall, Miguel Pieters, N. Brouwer, Gerrit Oomens, Angelos Konstantinidis, K. Winnips, S. Moes, W. van den Bos, B. Bredeweg, E. V. van Vliet
Personalized feedback is important for the learning process, but it is time consuming and particularly problematic in large-scale courses. While automatic feedback may help for self-regulated learning, not all forms of feedback are effective. Social comparison offers powerful feedback but is often loosely designed. We propose that intertwining meaningful feedback with well-designed peer comparison using a learning analytics dashboard provides a solution. Third-year bachelor students were randomly assigned to have access to the learning analytics dashboard IguideME (treatment, n=31) or no access (control, n=31). Dashboard users were asked to indicate their desired grade, which was used to construct peer-comparison groups. Personalized peer-comparison feedback was provided via the dashboard. The effects were studied using quantitative and qualitative data, including the Motivated Strategies for Learning Questionnaire (MSLQ) and the Achievement Goal Questionnaire (AGQ). Compared to the control group, the treatment group achieved higher scores for the MSLQ components “metacognitive self-regulation” and “peer learning,” and for the AGQ component “other-approach” (do better than others). The treatment group performed better on reading assignments and achieved higher grades for high-level Bloom exam questions. These data support the hypothesis that personalized peer-comparison feedback can be used to improve self-regulated learning and academic achievement.
{"title":"IguideME:","authors":"D. Fleur, M. Marshall, Miguel Pieters, N. Brouwer, Gerrit Oomens, Angelos Konstantinidis, K. Winnips, S. Moes, W. van den Bos, B. Bredeweg, E. V. van Vliet","doi":"10.18608/jla.2023.7853","DOIUrl":"https://doi.org/10.18608/jla.2023.7853","url":null,"abstract":"Personalized feedback is important for the learning process, but it is time consuming and particularly problematic in large-scale courses. While automatic feedback may help for self-regulated learning, not all forms of feedback are effective. Social comparison offers powerful feedback but is often loosely designed. We propose that intertwining meaningful feedback with well-designed peer comparison using a learning analytics dashboard provides a solution. Third-year bachelor students were randomly assigned to have access to the learning analytics dashboard IguideME (treatment, n=31) or no access (control, n=31). Dashboard users were asked to indicate their desired grade, which was used to construct peer-comparison groups. Personalized peer-comparison feedback was provided via the dashboard. The effects were studied using quantitative and qualitative data, including the Motivated Strategies for Learning Questionnaire (MSLQ) and the Achievement Goal Questionnaire (AGQ). Compared to the control group, the treatment group achieved higher scores for the MSLQ components “metacognitive self-regulation” and “peer learning,” and for the AGQ component “other-approach” (do better than others). The treatment group performed better on reading assignments and achieved higher grades for high-level Bloom exam questions. These data support the hypothesis that personalized peer-comparison feedback can be used to improve self-regulated learning and academic achievement.","PeriodicalId":36754,"journal":{"name":"Journal of Learning Analytics","volume":" ","pages":""},"PeriodicalIF":3.9,"publicationDate":"2023-08-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44140949","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Daniel Biedermann, George-Petru Ciordas-Hertel, Marc Winter, Julia Mordel, H. Drachsler
Learners use digital media during learning for a variety of reasons. Sometimes media use can be considered “on-task,” e.g., to perform research or to collaborate with peers. In other cases, media use is “off-task,” meaning that learners use content unrelated to their current learning task. Given the well-known problems with self-reported data (incomplete memory, distorted perceptions, subjective attributions), exploring on-task and off-task usage of digital media in learning scenarios requires logging activity on digital devices. However, we argue that logging on- and off-task behaviour has challenges that are rarely addressed. First, logging must be active only during learning. Second, logging represents a potential invasion of privacy. Third, logging must incorporate multiple devices simultaneously to take the reality of media multitasking into account. Fourth, logging alone is insufficient to reveal what prompted learners to switch to a different digital activity. To address these issues, we present a contextually activated logging system that allows users to inspect and annotate the observed activities after a learning session. Data from a formative study show that our system works as intended, and furthermore supports our assumptions about the diverse intentions of media use in learning. We discuss the implications for learning analytics.
{"title":"Contextualized Logging of On-Task and Off-Task Behaviours During Learning","authors":"Daniel Biedermann, George-Petru Ciordas-Hertel, Marc Winter, Julia Mordel, H. Drachsler","doi":"10.18608/jla.2023.7837","DOIUrl":"https://doi.org/10.18608/jla.2023.7837","url":null,"abstract":"Learners use digital media during learning for a variety of reasons. Sometimes media use can be considered “on-task,” e.g., to perform research or to collaborate with peers. In other cases, media use is “off-task,” meaning that learners use content unrelated to their current learning task. Given the well-known problems with self-reported data (incomplete memory, distorted perceptions, subjective attributions), exploring on-task and off-task usage of digital media in learning scenarios requires logging activity on digital devices. However, we argue that logging on- and off-task behaviour has challenges that are rarely addressed. First, logging must be active only during learning. Second, logging represents a potential invasion of privacy. Third, logging must incorporate multiple devices simultaneously to take the reality of media multitasking into account. Fourth, logging alone is insufficient to reveal what prompted learners to switch to a different digital activity. To address these issues, we present a contextually activated logging system that allows users to inspect and annotate the observed activities after a learning session. Data from a formative study show that our system works as intended, and furthermore supports our assumptions about the diverse intentions of media use in learning. We discuss the implications for learning analytics.","PeriodicalId":36754,"journal":{"name":"Journal of Learning Analytics","volume":" ","pages":""},"PeriodicalIF":3.9,"publicationDate":"2023-08-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44956776","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dirk T. Tempelaar, B. Rienties, B. Giesbers, Quan Nguyen
Learning analytics needs to pay more attention to the temporal aspect of learning processes, especially in self-regulated learning (SRL) research. In doing so, learning analytics models should incorporate both the duration and frequency of learning activities, the passage of time, and the temporal order of learning activities. However, where this exhortation is widely supported, there is less agreement on its consequences. Does paying tribute to temporal aspects of learning processes necessarily imply that event-based models are to replace variable-based models, and analytic discovery methods substitute traditional statistical methods? We do not necessarily require such a paradigm shift to give temporal aspects their position. First, temporal aspects can be integrated into variable-based models that apply statistical methods by carefully choosing appropriate time windows and granularity levels. Second, in addressing temporality in learning analytic models that describe authentic learning settings, heterogeneity is of crucial importance in both variable- and event-based models. Variable-based person-centred modelling, where a heterogeneous sample is split into homogeneous subsamples, is suggested as a solution. Our conjecture is illustrated by an application of dispositional learning analytics, describing authentic learning processes over an eight-week full module of 2,360 students.
{"title":"Modelling Temporality in Person- and Variable-Centred Approaches","authors":"Dirk T. Tempelaar, B. Rienties, B. Giesbers, Quan Nguyen","doi":"10.18608/jla.2023.7841","DOIUrl":"https://doi.org/10.18608/jla.2023.7841","url":null,"abstract":"Learning analytics needs to pay more attention to the temporal aspect of learning processes, especially in self-regulated learning (SRL) research. In doing so, learning analytics models should incorporate both the duration and frequency of learning activities, the passage of time, and the temporal order of learning activities. However, where this exhortation is widely supported, there is less agreement on its consequences. Does paying tribute to temporal aspects of learning processes necessarily imply that event-based models are to replace variable-based models, and analytic discovery methods substitute traditional statistical methods? We do not necessarily require such a paradigm shift to give temporal aspects their position. First, temporal aspects can be integrated into variable-based models that apply statistical methods by carefully choosing appropriate time windows and granularity levels. Second, in addressing temporality in learning analytic models that describe authentic learning settings, heterogeneity is of crucial importance in both variable- and event-based models. Variable-based person-centred modelling, where a heterogeneous sample is split into homogeneous subsamples, is suggested as a solution. Our conjecture is illustrated by an application of dispositional learning analytics, describing authentic learning processes over an eight-week full module of 2,360 students.","PeriodicalId":36754,"journal":{"name":"Journal of Learning Analytics","volume":"1 1","pages":""},"PeriodicalIF":3.9,"publicationDate":"2023-08-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41711709","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Learning has a temporal characteristic in nature, which means that it occurs over the passage of time. The research on the temporal aspects of learning faces several challenges, one of which is utilizing appropriate analytical techniques to exploit the temporal data. There is no coherent guide to selecting certain temporal techniques to lead to results that truthfully uncover underlying phenomena. To fill this gap, this systematic mapping study contributes to understanding the type of questions and approaches in works in the area of temporal educational research. This study aims to analyze different components of published research and explores the current trends in educational studies that explicitly consider the temporal aspect. Using the thematic coding method, we identified trends in three components, including asked research questions, utilized methodological techniques, and inferred insight about learning. The distribution of codes regarding asked research questions showed that the highest number of studies focused on method development or proposing a methodological framework. We discussed that methodological development, with the underlying theory, led to identifying learning indicators that can provide the ability to identify individual students with respect to the learning concepts of interest. In terms of utilized techniques, there was a strong trend in visualization analysis and process mining. This study found that to discover insight into learning, it is important to utilize techniques that are interpretable to characterize temporal patterns.
{"title":"Associations of Research Questions, Analytical Techniques, and Learning Insight in Temporal Educational Research","authors":"Sina Nazeri, M. Hatala, Carman Neustaedter","doi":"10.18608/jla.2023.7745","DOIUrl":"https://doi.org/10.18608/jla.2023.7745","url":null,"abstract":"Learning has a temporal characteristic in nature, which means that it occurs over the passage of time. The research on the temporal aspects of learning faces several challenges, one of which is utilizing appropriate analytical techniques to exploit the temporal data. There is no coherent guide to selecting certain temporal techniques to lead to results that truthfully uncover underlying phenomena. To fill this gap, this systematic mapping study contributes to understanding the type of questions and approaches in works in the area of temporal educational research. This study aims to analyze different components of published research and explores the current trends in educational studies that explicitly consider the temporal aspect. Using the thematic coding method, we identified trends in three components, including asked research questions, utilized methodological techniques, and inferred insight about learning. The distribution of codes regarding asked research questions showed that the highest number of studies focused on method development or proposing a methodological framework. We discussed that methodological development, with the underlying theory, led to identifying learning indicators that can provide the ability to identify individual students with respect to the learning concepts of interest. In terms of utilized techniques, there was a strong trend in visualization analysis and process mining. This study found that to discover insight into learning, it is important to utilize techniques that are interpretable to characterize temporal patterns.","PeriodicalId":36754,"journal":{"name":"Journal of Learning Analytics","volume":" ","pages":""},"PeriodicalIF":3.9,"publicationDate":"2023-08-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43285789","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Egle Gedrimiene, Ismail Celik, Kati Mäkitalo, Hanni Muukkonen
Transparency and trustworthiness are among the key requirements for the ethical use of learning analytics (LA) and artificial intelligence (AI) in the context of social inclusion and equity. However, research on these issues pertaining to users is lacking, leaving it unclear as to how transparent and trustworthy current LA tools are for their users and how perceptions of these variables relate to user behaviour. In this study, we investigate user experiences of an LA tool in the context of career guidance, which plays a crucial role in supporting nonlinear career pathways for individuals. We review the ethical challenges of big data, AI, and LA in connection to career guidance and analyze the user experiences (N = 106) of the LA career guidance tool, which recommends study programs and institutions to users. Results indicate that the LA career guidance tool was evaluated as trustworthy but not transparent. Accuracy was found to be a stronger predictor for the intention to follow on the recommendations of the LA guidance tool than was understanding the origins of the recommendation. The user’s age emerged as an important factor in their assessment of transparency. We discuss the implications of these findings and suggest emphasizing accuracy in the development of LA tools for career guidance.
{"title":"Transparency and Trustworthiness in User Intentions to Follow Career Recommendations from a Learning Analytics Tool","authors":"Egle Gedrimiene, Ismail Celik, Kati Mäkitalo, Hanni Muukkonen","doi":"10.18608/jla.2023.7791","DOIUrl":"https://doi.org/10.18608/jla.2023.7791","url":null,"abstract":"Transparency and trustworthiness are among the key requirements for the ethical use of learning analytics (LA) and artificial intelligence (AI) in the context of social inclusion and equity. However, research on these issues pertaining to users is lacking, leaving it unclear as to how transparent and trustworthy current LA tools are for their users and how perceptions of these variables relate to user behaviour. In this study, we investigate user experiences of an LA tool in the context of career guidance, which plays a crucial role in supporting nonlinear career pathways for individuals. We review the ethical challenges of big data, AI, and LA in connection to career guidance and analyze the user experiences (N = 106) of the LA career guidance tool, which recommends study programs and institutions to users. Results indicate that the LA career guidance tool was evaluated as trustworthy but not transparent. Accuracy was found to be a stronger predictor for the intention to follow on the recommendations of the LA guidance tool than was understanding the origins of the recommendation. The user’s age emerged as an important factor in their assessment of transparency. We discuss the implications of these findings and suggest emphasizing accuracy in the development of LA tools for career guidance.","PeriodicalId":36754,"journal":{"name":"Journal of Learning Analytics","volume":"586 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136155870","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Benjamin A. Motz, Yoav Bergner, Christopher A. Brooks, Anna Gladden, G. Gray, Charles Lang, Warren Li, F. Marmolejo‐Ramos, Joshua D. Quick
Learning analytics defines itself with a focus on data from learners and learning environments, with corresponding goals of understanding and optimizing student learning. In this regard, learning analytics research, ideally, should be characterized by studies that make use of data from learners engaged in education systems, should measure student learning, and should make efforts to intervene and improve these learning environments. However, a common concern among members of the learning analytics research community is that these standards are not being met. In two analysis waves, we review a large and comprehensive sample of research articles from the proceedings of the three most recent Learning Analytics and Knowledge conferences, the premier conference venue for learning analytics research, and from articles published during the same time in the Journal of Learning Analytics (over the years of 2020, 2021, and 2022). We find that 37.4% of articles do not analyze data from learners in an education system, 71.1% do not include any measure of learning, and 89.0% of articles do not attempt to intervene in the learning environment. We contrast these findings with the stated definition of learning analytics and infer, like others before us, that scholarship in learning analytics research presently lacks clear direction toward its stated goals. We invite critical discussion of these findings from the learning analytics community, through open peer commentary.
{"title":"LAK of Direction","authors":"Benjamin A. Motz, Yoav Bergner, Christopher A. Brooks, Anna Gladden, G. Gray, Charles Lang, Warren Li, F. Marmolejo‐Ramos, Joshua D. Quick","doi":"10.18608/jla.2023.7913","DOIUrl":"https://doi.org/10.18608/jla.2023.7913","url":null,"abstract":"Learning analytics defines itself with a focus on data from learners and learning environments, with corresponding goals of understanding and optimizing student learning. In this regard, learning analytics research, ideally, should be characterized by studies that make use of data from learners engaged in education systems, should measure student learning, and should make efforts to intervene and improve these learning environments. However, a common concern among members of the learning analytics research community is that these standards are not being met. In two analysis waves, we review a large and comprehensive sample of research articles from the proceedings of the three most recent Learning Analytics and Knowledge conferences, the premier conference venue for learning analytics research, and from articles published during the same time in the Journal of Learning Analytics (over the years of 2020, 2021, and 2022). We find that 37.4% of articles do not analyze data from learners in an education system, 71.1% do not include any measure of learning, and 89.0% of articles do not attempt to intervene in the learning environment. We contrast these findings with the stated definition of learning analytics and infer, like others before us, that scholarship in learning analytics research presently lacks clear direction toward its stated goals. We invite critical discussion of these findings from the learning analytics community, through open peer commentary.","PeriodicalId":36754,"journal":{"name":"Journal of Learning Analytics","volume":" ","pages":""},"PeriodicalIF":3.9,"publicationDate":"2023-03-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47643542","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The use of online video lectures in universities, primarily for content delivery and learning, is on the rise. Instructors ’ ability to recognize and understand student learning experiences with online video lectures, identify particularly difficult or disengaging content and thereby assess overall lecture quality can inform their instructional practice related to such lectures. This paper introduces Tcherly, a teacher-facing dashboard that presents class-level aggregated time series data on student s’ self-reported cognitive-affective states they experienced during a lecture. Instructors can use the dashboard to evaluate and improve their instructional practice related to video lectures. We report the detailed iterative prototyping design process of the Tcherly Dashboard involving two stakeholders (instructors and designers) that informed various design decisions of the dashboard, and also provide usability and usefulness data. We demonstrate, with real-life examples of Tcherly Dashboard use generated by the researchers based on data collected from six courses and 11 lectures, how the dashboard can assist instructors in understanding thei r students’ learning experiences and evaluating the associated instructional materials. decision-making. This paper demonstrates how stakeholders (instructors and designers) can be involved in the design process of such a dashboard to inform microlevel design decisions such as visualization, the format of dashboard elements, and supports required by instructors to make sense of the presented information (analytics). The evolution of the dashboard design through iterative prototyping with instructors and designers is demonstrated along with usability and usefulness evaluation results. • We present guidelines for instructors to use the dashboard based on data gathered from six courses and 11 lectures. The real-life examples of dashboard use demonstrate how to use dashboard features and visualizations in tandem to understand student learning experiences and evaluate the associated instructional materials.
{"title":"Tcherly:: A Teacher-facing Dashboard for Online Video Lectures","authors":"Pankaj S. Chavan, R. Mitra","doi":"10.18608/jla.2022.7555","DOIUrl":"https://doi.org/10.18608/jla.2022.7555","url":null,"abstract":"The use of online video lectures in universities, primarily for content delivery and learning, is on the rise. Instructors ’ ability to recognize and understand student learning experiences with online video lectures, identify particularly difficult or disengaging content and thereby assess overall lecture quality can inform their instructional practice related to such lectures. This paper introduces Tcherly, a teacher-facing dashboard that presents class-level aggregated time series data on student s’ self-reported cognitive-affective states they experienced during a lecture. Instructors can use the dashboard to evaluate and improve their instructional practice related to video lectures. We report the detailed iterative prototyping design process of the Tcherly Dashboard involving two stakeholders (instructors and designers) that informed various design decisions of the dashboard, and also provide usability and usefulness data. We demonstrate, with real-life examples of Tcherly Dashboard use generated by the researchers based on data collected from six courses and 11 lectures, how the dashboard can assist instructors in understanding thei r students’ learning experiences and evaluating the associated instructional materials. decision-making. This paper demonstrates how stakeholders (instructors and designers) can be involved in the design process of such a dashboard to inform microlevel design decisions such as visualization, the format of dashboard elements, and supports required by instructors to make sense of the presented information (analytics). The evolution of the dashboard design through iterative prototyping with instructors and designers is demonstrated along with usability and usefulness evaluation results. • We present guidelines for instructors to use the dashboard based on data gathered from six courses and 11 lectures. The real-life examples of dashboard use demonstrate how to use dashboard features and visualizations in tandem to understand student learning experiences and evaluate the associated instructional materials.","PeriodicalId":36754,"journal":{"name":"Journal of Learning Analytics","volume":"9 1","pages":"125-151"},"PeriodicalIF":3.9,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"67542695","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Simon Knight, S. Abel, A. Shibani, Yoong Kuan Goh, Rianne Conijn, A. Gibson, Sowmya Vajjala, Elena Cotos, Ágnes Sándor, S. B. Shum
Writing analytics has emerged as a sub-field of learning analytics, with applications including the provision of formative feedback to students in developing their writing capacities. Rhetorical markers in writing have become a key feature in this feedback, with a number of tools being developed across research and teaching contexts. However, there is no shared corpus of texts annotated by these tools, nor is it clear how the tool annotations compare. Thus, resources are scarce for comparing tools for both tool development and pedagogic purposes. In this paper, we conduct such a comparison and introduce a sample corpus of texts representative of the particular genres, a subset of which has been annotated using three rhetorical analysis tools (one of which has two versions). This paper aims to provide both a description of the tools and a shared dataset in order to support extensions of existing analyses and tool design in support of writing skill development. We intend the description of these tools, which share a focus on rhetorical structures, alongside the corpus, to be a preliminary step to enable further research, with regard to both tool development and tool interaction
{"title":"Are You Being Rhetorical? A Description of Rhetorical Move Annotation Tools and Open Corpus of Sample Machine-Annotated Rhetorical Moves","authors":"Simon Knight, S. Abel, A. Shibani, Yoong Kuan Goh, Rianne Conijn, A. Gibson, Sowmya Vajjala, Elena Cotos, Ágnes Sándor, S. B. Shum","doi":"10.18608/jla.2020.73.10","DOIUrl":"https://doi.org/10.18608/jla.2020.73.10","url":null,"abstract":"Writing analytics has emerged as a sub-field of learning analytics, with applications including the provision of formative feedback to students in developing their writing capacities. Rhetorical markers in writing have become a key feature in this feedback, with a number of tools being developed across research and teaching contexts. However, there is no shared corpus of texts annotated by these tools, nor is it clear how the tool annotations compare. Thus, resources are scarce for comparing tools for both tool development and pedagogic purposes. In this paper, we conduct such a comparison and introduce a sample corpus of texts representative of the particular genres, a subset of which has been annotated using three rhetorical analysis tools (one of which has two versions). This paper aims to provide both a description of the tools and a shared dataset in order to support extensions of existing analyses and tool design in support of writing skill development. We intend the description of these tools, which share a focus on rhetorical structures, alongside the corpus, to be a preliminary step to enable further research, with regard to both tool development and tool interaction","PeriodicalId":36754,"journal":{"name":"Journal of Learning Analytics","volume":"7 1","pages":"138-154"},"PeriodicalIF":3.9,"publicationDate":"2020-12-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"67542589","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：