Pub Date : 2024-01-15DOI: 10.1109/TLT.2024.3354128
David P. Reid;Timothy D. Drysdale
The designs of many student-facing learning analytics (SFLA) dashboards are insufficiently informed by educational research and lack rigorous evaluation in authentic learning contexts, including during remote laboratory practical work. In this article, we present and evaluate an SFLA dashboard designed using the principles of formative assessment to provide feedback to students during remote lab activities. Feedback is based upon graphical visualizations of student actions performed during lab tasks and comparison to expected procedures using TaskCompare—our custom, asymmetric graph dissimilarity measure that distinguishes students who miss expected actions from those who perform additional actions, a capability missing in existing graph distance (symmetrical dissimilarity) measures. Using a total of $N = 235$ student graphs collected during authentic learning in two different engineering courses, we describe the validation of TaskCompare and evaluate the impact of the SFLA dashboard on task completion during remote lab activities. In addition, we use components of the motivated strategies for learning questionnaire as covariates for propensity score matching to account for potential bias in self-selection of use of the dashboard. We find that those students who used the SFLA dashboard achieved significantly better task completion rate (nearly double) than those who did not, with a significant difference in TaskCompare score between the two groups (Mann–Whitney $U = 453.5$ , $p < 0.01$ and Cliff's $delta = 0.43$ , large effect size). This difference remains after accounting for self-selection. We also report that students' positive rating of the usefulness of the SFLA dashboard for completing lab work is significantly above a neutral response ( $S = 21.0$ and $p < 0.01$ ). These findings provide evidence that our SFLA dashboard is an effective means of providing formative assessment during remote laboratory activities.
{"title":"Student-Facing Learning Analytics Dashboard for Remote Lab Practical Work","authors":"David P. Reid;Timothy D. Drysdale","doi":"10.1109/TLT.2024.3354128","DOIUrl":"https://doi.org/10.1109/TLT.2024.3354128","url":null,"abstract":"The designs of many student-facing learning analytics (SFLA) dashboards are insufficiently informed by educational research and lack rigorous evaluation in authentic learning contexts, including during remote laboratory practical work. In this article, we present and evaluate an SFLA dashboard designed using the principles of formative assessment to provide feedback to students during remote lab activities. Feedback is based upon graphical visualizations of student actions performed during lab tasks and comparison to expected procedures using TaskCompare—our custom, asymmetric graph dissimilarity measure that distinguishes students who miss expected actions from those who perform additional actions, a capability missing in existing graph distance (symmetrical dissimilarity) measures. Using a total of \u0000<inline-formula><tex-math>$N = 235$</tex-math></inline-formula>\u0000 student graphs collected during authentic learning in two different engineering courses, we describe the validation of TaskCompare and evaluate the impact of the SFLA dashboard on task completion during remote lab activities. In addition, we use components of the motivated strategies for learning questionnaire as covariates for propensity score matching to account for potential bias in self-selection of use of the dashboard. We find that those students who used the SFLA dashboard achieved significantly better task completion rate (nearly double) than those who did not, with a significant difference in TaskCompare score between the two groups (Mann–Whitney \u0000<inline-formula><tex-math>$U = 453.5$</tex-math></inline-formula>\u0000, \u0000<inline-formula><tex-math>$p < 0.01$</tex-math></inline-formula>\u0000 and Cliff's \u0000<inline-formula><tex-math>$delta = 0.43$</tex-math></inline-formula>\u0000, large effect size). This difference remains after accounting for self-selection. We also report that students' positive rating of the usefulness of the SFLA dashboard for completing lab work is significantly above a neutral response (\u0000<inline-formula><tex-math>$S = 21.0$</tex-math></inline-formula>\u0000 and \u0000<inline-formula><tex-math>$p < 0.01$</tex-math></inline-formula>\u0000). These findings provide evidence that our SFLA dashboard is an effective means of providing formative assessment during remote laboratory activities.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1037-1050"},"PeriodicalIF":3.7,"publicationDate":"2024-01-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139676214","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-12DOI: 10.1109/TLT.2024.3352743
Chen Li;Yue Jiang;Peter H. F. Ng;Yixin Dai;Francis Cheung;Henry C. B. Chan;Ping Li
Computer-supported collaborative learning aims to use information technologies to support collaborative knowledge construction by practicing the relevant pedagogical approaches, especially in the distance learning setting. The enabling technologies are fast advancing, and the need for solutions during the COVID-19 global pandemic led to the emergence of the Edu-Metaverse, which is conceptualized as a collection of networked virtual worlds (i.e., the Metaverse) for learning. There is a great necessity to investigate how these more recent enabling technologies can support collaborative learning. This empirical study aims to collect both quantitative and qualitative results to fill the knowledge gaps. Specifically, 20 undergraduate students (three females and 17 males) taking the Game Design and Development course voluntarily participated in this study. The participants used three representative collaboration platforms (i.e., AltSpace, Gather, and ZOOM) in our laboratory for discussing three course-specific topics, simulating the undertaking of collaborative learning tasks in the distance learning setting. The results suggest that the participants were more engaged in the learning activities using the Metaverse platforms that offer avatar-mediated communications and collaborations (i.e., AltSpace and Gather). These platforms also gave the participants a stronger sense of copresence and belonging to the learning community. Potential improvements to the usability and the participants' feedback are also discussed in the article. We hope the results can contribute to the fast-growing use of the Metaverse-enabling technologies for educational purposes.
{"title":"Collaborative Learning in the Edu-Metaverse Era: An Empirical Study on the Enabling Technologies","authors":"Chen Li;Yue Jiang;Peter H. F. Ng;Yixin Dai;Francis Cheung;Henry C. B. Chan;Ping Li","doi":"10.1109/TLT.2024.3352743","DOIUrl":"https://doi.org/10.1109/TLT.2024.3352743","url":null,"abstract":"Computer-supported collaborative learning aims to use information technologies to support collaborative knowledge construction by practicing the relevant pedagogical approaches, especially in the distance learning setting. The enabling technologies are fast advancing, and the need for solutions during the COVID-19 global pandemic led to the emergence of the Edu-Metaverse, which is conceptualized as a collection of networked virtual worlds (i.e., the Metaverse) for learning. There is a great necessity to investigate how these more recent enabling technologies can support collaborative learning. This empirical study aims to collect both quantitative and qualitative results to fill the knowledge gaps. Specifically, 20 undergraduate students (three females and 17 males) taking the Game Design and Development course voluntarily participated in this study. The participants used three representative collaboration platforms (i.e., AltSpace, Gather, and ZOOM) in our laboratory for discussing three course-specific topics, simulating the undertaking of collaborative learning tasks in the distance learning setting. The results suggest that the participants were more engaged in the learning activities using the Metaverse platforms that offer avatar-mediated communications and collaborations (i.e., AltSpace and Gather). These platforms also gave the participants a stronger sense of copresence and belonging to the learning community. Potential improvements to the usability and the participants' feedback are also discussed in the article. We hope the results can contribute to the fast-growing use of the Metaverse-enabling technologies for educational purposes.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1107-1119"},"PeriodicalIF":3.7,"publicationDate":"2024-01-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10399380","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139738983","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-10DOI: 10.1109/TLT.2023.3346645
Gina George;Anisha M. Lal
E-learning is increasingly being used by students in the higher education level for their university credit purpose and some for improving their knowledge. E-learning is also used for skill enhancement purpose by organizations. Due to the availability of wide-ranging options, recommender systems that provide personalized suggestions are much needed. The proposed methodology takes advantage of compact prediction tree (CPT), a popular sequence prediction algorithm. In this article, a new prediction model based on applying CPT over similar students which is found in a novel manner is proposed. The aim of the work is to recommend courses to students at university level. The methodology was evaluated in terms of accuracy and results show the proposed work performs better than applying only CPT, when applying fuzzy C-means with CPT, and when applying k nearest neighbors with CPT.
{"title":"PERKC: Personalized kNN With CPT for Course Recommendations in Higher Education","authors":"Gina George;Anisha M. Lal","doi":"10.1109/TLT.2023.3346645","DOIUrl":"https://doi.org/10.1109/TLT.2023.3346645","url":null,"abstract":"E-learning is increasingly being used by students in the higher education level for their university credit purpose and some for improving their knowledge. E-learning is also used for skill enhancement purpose by organizations. Due to the availability of wide-ranging options, recommender systems that provide personalized suggestions are much needed. The proposed methodology takes advantage of compact prediction tree (CPT), a popular sequence prediction algorithm. In this article, a new prediction model based on applying CPT over similar students which is found in a novel manner is proposed. The aim of the work is to recommend courses to students at university level. The methodology was evaluated in terms of accuracy and results show the proposed work performs better than applying only CPT, when applying fuzzy C-means with CPT, and when applying k nearest neighbors with CPT.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"885-892"},"PeriodicalIF":3.7,"publicationDate":"2024-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139419476","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-09DOI: 10.1109/TLT.2024.3351352
Oscar Blessed Deho;Lin Liu;Jiuyong Li;Jixue Liu;Chen Zhan;Srecko Joksimovic
Learning analytics (LA), like much of machine learning, assumes the training and test datasets come from the same distribution. Therefore, LA models built on past observations are (implicitly) expected to work well for future observations. However, this assumption does not always hold in practice because the dataset may drift. Recently, algorithmic fairness has gained significant attention. Nevertheless, algorithmic fairness research has paid little attention to dataset drift. Majority of the existing fairness algorithms are “statically” designed. Put another way, LA models tuned to be “fair” on past data are expected to still be “fair” when dealing with current/future data. However, it is counter-intuitive to deploy a statically fair algorithm to a nonstationary world. There is, therefore, a need to assess the impact of dataset drift on the unfairness of LA models. For this reason, we investigate the relationship between dataset drift and unfairness of LA models. Specifically, we first measure the degree of drift in the features (i.e., covariates) and target label of our dataset. After that, we train predictive models on the dataset and evaluate the relationship between the dataset drift and the unfairness of the predictive models. Our findings suggest a directly proportional relationship between dataset drift and unfairness. Further, we find covariate drift to have the most impact on unfairness of models as compared to target drift, and there are no guarantees that a once fair model would consistently remain fair. Our findings imply that “robustness” of fair LA models to dataset drift is necessary before deployment.
学习分析（LA）与大部分机器学习一样，都假定训练数据集和测试数据集来自相同的分布。因此，基于过去观察结果建立的学习分析模型（隐含地）有望在未来的观察结果中发挥良好的作用。然而，这一假设在实践中并不总是成立的，因为数据集可能会漂移。最近，算法公平性受到了广泛关注。然而，算法公平性研究很少关注数据集漂移问题。现有的大多数公平性算法都是 "静态 "设计的。换句话说，在过去的数据上调整为 "公平 "的洛杉矶模型，在处理当前/未来的数据时预计仍然是 "公平 "的。然而，将静态公平算法应用于非稳态世界是违背直觉的。因此，有必要评估数据集漂移对 LA 模型公平性的影响。为此，我们研究了数据集漂移与 LA 模型不公平性之间的关系。具体来说，我们首先测量数据集的特征（即协变量）和目标标签的漂移程度。然后，我们在数据集上训练预测模型，并评估数据集漂移与预测模型不公平程度之间的关系。我们的研究结果表明，数据集漂移与不公平之间存在正比关系。此外，我们发现与目标漂移相比，协变量漂移对模型不公平程度的影响最大，而且无法保证曾经公平的模型会一直保持公平。我们的研究结果表明，在部署公平的洛杉矶模型之前，必须使其对数据集漂移具有 "稳健性"。
{"title":"When the Past != The Future: Assessing the Impact of Dataset Drift on the Fairness of Learning Analytics Models","authors":"Oscar Blessed Deho;Lin Liu;Jiuyong Li;Jixue Liu;Chen Zhan;Srecko Joksimovic","doi":"10.1109/TLT.2024.3351352","DOIUrl":"https://doi.org/10.1109/TLT.2024.3351352","url":null,"abstract":"Learning analytics (LA), like much of machine learning, assumes the training and test datasets come from the same distribution. Therefore, LA models built on past observations are (implicitly) expected to work well for future observations. However, this assumption does not always hold in practice because the dataset may drift. Recently, algorithmic fairness has gained significant attention. Nevertheless, algorithmic fairness research has paid little attention to dataset drift. Majority of the existing fairness algorithms are “statically” designed. Put another way, LA models \u0000<italic>tuned</i>\u0000 to be “fair” on past data are expected to still be “fair” when dealing with current/future data. However, it is counter-intuitive to deploy a \u0000<italic>statically</i>\u0000 fair algorithm to a \u0000<italic>nonstationary</i>\u0000 world. There is, therefore, a need to assess the impact of dataset drift on the unfairness of LA models. For this reason, we investigate the relationship between dataset drift and unfairness of LA models. Specifically, we first measure the degree of drift in the features (i.e., covariates) and target label of our dataset. After that, we train predictive models on the dataset and evaluate the relationship between the dataset drift and the unfairness of the predictive models. Our findings suggest a directly proportional relationship between dataset drift and unfairness. Further, we find covariate drift to have the most impact on unfairness of models as compared to target drift, and there are no guarantees that a once fair model would consistently remain fair. Our findings imply that “robustness” of fair LA models to dataset drift is necessary before deployment.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1007-1020"},"PeriodicalIF":3.7,"publicationDate":"2024-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139572651","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-04DOI: 10.1109/TLT.2024.3349579
Furkan Yücel;Hasret Sultan Ünal;Elif Surer;Nejan Huvaj
Laboratory experience is an integral part of the undergraduate curriculum in most engineering courses. When physical learning is not feasible, and when the demand cannot be met through actual hands-on laboratory sessions, as has been during the COVID-19 pandemic, virtual laboratory courses can be considered as an alternative education medium. This study focuses on developing a generic modular virtual laboratory framework that allows engineers, game designers, and developers to build lab experiments as serious games—games with ulterior motives rather than only entertainment—without writing additional code. A virtual lab serious game for civil engineering's soil mechanics course was created in Unity3D as a WebGL game, and it was tested within the framework by 24 students (12 from the Civil Engineering Department, the rest from computer science-related degrees). Seven faculty members evaluated if the serious game met the learning outcomes. In addition, nine engineers and designers assessed the framework's capabilities and analyzed its flexibility and reuse aspects. To analyze the usability and acceptability of the created game, standard questionnaires such as the technology acceptance model, system usability scale, and presence were employed. The study was done in two phases: participants tested the first version of the game, and the second version was built based on their feedback on the first version. The findings indicate that the modular structure has significant potential for use in a variety of fields and laboratory courses. The proposed game has received very positive feedback and can be considered a use case for the potential of games in interactive virtual laboratories.
{"title":"A Modular Serious Game Development Framework for Virtual Laboratory Courses","authors":"Furkan Yücel;Hasret Sultan Ünal;Elif Surer;Nejan Huvaj","doi":"10.1109/TLT.2024.3349579","DOIUrl":"https://doi.org/10.1109/TLT.2024.3349579","url":null,"abstract":"Laboratory experience is an integral part of the undergraduate curriculum in most engineering courses. When physical learning is not feasible, and when the demand cannot be met through actual hands-on laboratory sessions, as has been during the COVID-19 pandemic, virtual laboratory courses can be considered as an alternative education medium. This study focuses on developing a generic modular virtual laboratory framework that allows engineers, game designers, and developers to build lab experiments as serious games—games with ulterior motives rather than only entertainment—without writing additional code. A virtual lab serious game for civil engineering's soil mechanics course was created in Unity3D as a WebGL game, and it was tested within the framework by 24 students (12 from the Civil Engineering Department, the rest from computer science-related degrees). Seven faculty members evaluated if the serious game met the learning outcomes. In addition, nine engineers and designers assessed the framework's capabilities and analyzed its flexibility and reuse aspects. To analyze the usability and acceptability of the created game, standard questionnaires such as the technology acceptance model, system usability scale, and presence were employed. The study was done in two phases: participants tested the first version of the game, and the second version was built based on their feedback on the first version. The findings indicate that the modular structure has significant potential for use in a variety of fields and laboratory courses. The proposed game has received very positive feedback and can be considered a use case for the potential of games in interactive virtual laboratories.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"966-981"},"PeriodicalIF":3.7,"publicationDate":"2024-01-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139504438","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-03DOI: 10.1109/TLT.2023.3349161
Sofia Seinfeld;Filippo Gabriele Pratticó;Chiara De Giorgi;Fabrizio Lamberti
Swapping visual perspective in virtual reality (VR) provides a unique means for embodying different virtual bodies and for self-distancing. Moreover, this technology is a powerful tool for experiential learning and for simulating realistic scenarios, with broad potential in the training of soft skills. However, there is scarce knowledge on how perspective swapping in VR might benefit the training of soft skills such as those required in a job interview. This article investigates the impact of virtual body swapping on the self-assessment of verbal and nonverbal communication skills, emotional states, and embodiment in a simulated job interview context. Three main conditions were compared: a baseline condition in which the participants practiced a job interview from the first-person perspective of a virtual interviewee (no swap condition); an external point of view condition where, first, the participants answered questions from the interviewee perspective, but then swap visual perspective to re-experience their responses from a nonembodied point of view (out of body condition); and a condition in which, after answering questions from the interviewee perspective, the participants re-experienced their responses from the embodied perspective of the virtual recruiter (recruiter condition). The experimental results indicated that the effectiveness of the out of body and recruiter conditions was superior to the no swap condition to self-assess the communication styles used during a job interview. Moreover, all the conditions led to a high level of embodiment toward the interviewee avatar when seen from the first-person perspective; in the case of the recruiter condition, the participants also felt embodied in the recruiter avatar. No differences in emotional states were found among conditions, with all sharing a positive valence.
{"title":"Virtual Reality Body Swapping to Improve Self-Assessment in Job Interview Training","authors":"Sofia Seinfeld;Filippo Gabriele Pratticó;Chiara De Giorgi;Fabrizio Lamberti","doi":"10.1109/TLT.2023.3349161","DOIUrl":"https://doi.org/10.1109/TLT.2023.3349161","url":null,"abstract":"Swapping visual perspective in virtual reality (VR) provides a unique means for embodying different virtual bodies and for self-distancing. Moreover, this technology is a powerful tool for experiential learning and for simulating realistic scenarios, with broad potential in the training of soft skills. However, there is scarce knowledge on how perspective swapping in VR might benefit the training of soft skills such as those required in a job interview. This article investigates the impact of virtual body swapping on the self-assessment of verbal and nonverbal communication skills, emotional states, and embodiment in a simulated job interview context. Three main conditions were compared: a baseline condition in which the participants practiced a job interview from the first-person perspective of a virtual interviewee (no swap condition); an external point of view condition where, first, the participants answered questions from the interviewee perspective, but then swap visual perspective to re-experience their responses from a nonembodied point of view (out of body condition); and a condition in which, after answering questions from the interviewee perspective, the participants re-experienced their responses from the embodied perspective of the virtual recruiter (recruiter condition). The experimental results indicated that the effectiveness of the out of body and recruiter conditions was superior to the no swap condition to self-assess the communication styles used during a job interview. Moreover, all the conditions led to a high level of embodiment toward the interviewee avatar when seen from the first-person perspective; in the case of the recruiter condition, the participants also felt embodied in the recruiter avatar. No differences in emotional states were found among conditions, with all sharing a positive valence.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"992-1006"},"PeriodicalIF":3.7,"publicationDate":"2024-01-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139504423","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-01DOI: 10.1109/TLT.2023.3348690
Andrea Zanellati;Daniele Di Mitri;Maurizio Gabbrielli;Olivia Levrini
Knowledge tracing is a well-known problem in AI for education, consisting of monitoring how the knowledge state of students changes during the learning process and accurately predicting their performance in future exercises. In recent years, many advances have been made thanks to various machine learning and deep learning techniques. Despite their satisfactory performances, they have some pitfalls, e.g., modeling one skill at a time, ignoring the relationships between different skills, or inconsistency for the predictions, i.e., sudden spikes and falls across time steps. For this reason, hybrid machine-learning techniques have also been explored. With this systematic literature review, we aim to illustrate the state of the art in this field. Specifically, we want to identify the potential and the frontiers in integrating prior knowledge sources in the traditional machine learning pipeline as a supplement to the normally considered data. We applied a qualitative analysis to distill a taxonomy with the following three dimensions: knowledge source, knowledge representation, and knowledge integration. Exploiting this taxonomy, we also conducted a quantitative analysis to detect the most common approaches.
{"title":"Hybrid Models for Knowledge Tracing: A Systematic Literature Review","authors":"Andrea Zanellati;Daniele Di Mitri;Maurizio Gabbrielli;Olivia Levrini","doi":"10.1109/TLT.2023.3348690","DOIUrl":"https://doi.org/10.1109/TLT.2023.3348690","url":null,"abstract":"Knowledge tracing is a well-known problem in AI for education, consisting of monitoring how the knowledge state of students changes during the learning process and accurately predicting their performance in future exercises. In recent years, many advances have been made thanks to various machine learning and deep learning techniques. Despite their satisfactory performances, they have some pitfalls, e.g., modeling one skill at a time, ignoring the relationships between different skills, or inconsistency for the predictions, i.e., sudden spikes and falls across time steps. For this reason, hybrid machine-learning techniques have also been explored. With this systematic literature review, we aim to illustrate the state of the art in this field. Specifically, we want to identify the potential and the frontiers in integrating prior knowledge sources in the traditional machine learning pipeline as a supplement to the normally considered data. We applied a qualitative analysis to distill a taxonomy with the following three dimensions: knowledge source, knowledge representation, and knowledge integration. Exploiting this taxonomy, we also conducted a quantitative analysis to detect the most common approaches.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1021-1036"},"PeriodicalIF":3.7,"publicationDate":"2024-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10379123","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139654998","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Knowledge tracing (KT), the task of tracking the knowledge state of a student over time, has been assessed actively by artificial intelligence researchers. Recent reports have described that Deep-IRT, which combines item response theory (IRT) with a deep learning method, provides superior performance. It can express the abilities of each student and the difficulty of each item such as IRT. Nevertheless, its interpretability is inadequate compared to that of IRT because the ability parameter depends on each item. Deep-IRT implicitly assumes that items with the same skills are equivalent, which does not hold when item difficulties for the same skills differ greatly. For identical skills, items that are not equivalent hinder the interpretation of a student's ability estimate. To overcome those difficulties, this study proposes a novel Deep-IRT that models a student response to an item using two independent networks: 1) a student network and 2) an item network. The proposed Deep-IRT method learns student parameters and item parameters independently to avoid impairing the predictive accuracy. Moreover, we propose a novel hypernetwork architecture for the proposed Deep-IRT to balance both the current and the past data in the latent variable storing student's knowledge states. Results of experiments with six benchmark datasets demonstrate that the proposed method improves the prediction accuracy by about 2.0%, on average. In addition, experiments for the simulation dataset demonstrated that the proposed method provides a stronger correlation with true parameters than the earlier Deep-IRT method does at the $p< 0.5$ significance level.
{"title":"Deep Knowledge Tracing Incorporating a Hypernetwork With Independent Student and Item Networks","authors":"Emiko Tsutsumi;Yiming Guo;Ryo Kinoshita;Maomi Ueno","doi":"10.1109/TLT.2023.3346671","DOIUrl":"https://doi.org/10.1109/TLT.2023.3346671","url":null,"abstract":"Knowledge tracing (KT), the task of tracking the knowledge state of a student over time, has been assessed actively by artificial intelligence researchers. Recent reports have described that Deep-IRT, which combines item response theory (IRT) with a deep learning method, provides superior performance. It can express the abilities of each student and the difficulty of each item such as IRT. Nevertheless, its interpretability is inadequate compared to that of IRT because the ability parameter depends on each item. Deep-IRT implicitly assumes that items with the same skills are equivalent, which does not hold when item difficulties for the same skills differ greatly. For identical skills, items that are not equivalent hinder the interpretation of a student's ability estimate. To overcome those difficulties, this study proposes a novel Deep-IRT that models a student response to an item using two independent networks: 1) a student network and 2) an item network. The proposed Deep-IRT method learns student parameters and item parameters independently to avoid impairing the predictive accuracy. Moreover, we propose a novel hypernetwork architecture for the proposed Deep-IRT to balance both the current and the past data in the latent variable storing student's knowledge states. Results of experiments with six benchmark datasets demonstrate that the proposed method improves the prediction accuracy by about 2.0%, on average. In addition, experiments for the simulation dataset demonstrated that the proposed method provides a stronger correlation with true parameters than the earlier Deep-IRT method does at the \u0000<inline-formula><tex-math>$p< 0.5$</tex-math></inline-formula>\u0000 significance level.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"951-965"},"PeriodicalIF":3.7,"publicationDate":"2023-12-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10373110","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139504424","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-19DOI: 10.1109/TLT.2023.3343690
Alberto Corbi;Daniel Burgos;Antonio María Pérez
Literate programming and cloud-operated open literate educational resources (COOLERs) have been catching the attention of the education community in recent years. This set of learning materials mainly comprises digital notebook-like documents, which are stored, backed, and delivered from cloud services and eventually displayed in students' web browsers. As we demonstrate in this article, the advent of cloud architectures and the COVID-19 pandemic (which forced worldwide long-term distant academic environments) fortuitously teamed up with this learning and methodological trend by easing its use and fostering its adoption. With more detail, we have quantitatively measured the impact that the COOLER paradigm has had on the teaching realm by analyzing five years of logged data gathered by its current major player in the ecosystem: MyBinder. Among other results, we show how this growth in the production and delivery of notebooks made an important leap during the second SARS-CoV-2 wave (July–September 2020). However, the general usage trend seems to have strongly decreased after the end of the most recent seventh wave (September 2022), coinciding with the official end of the global health crisis and all the lockdown episodes. From these examined data, we conclude that COOLER and recent massive online learning scenarios have been very intimately linked. This fact may represent a flaw in the adoption of these exciting and useful learning materials.
{"title":"Cloud-Operated Open Literate Educational Resources: The Case of the MyBinder","authors":"Alberto Corbi;Daniel Burgos;Antonio María Pérez","doi":"10.1109/TLT.2023.3343690","DOIUrl":"https://doi.org/10.1109/TLT.2023.3343690","url":null,"abstract":"Literate programming and cloud-operated open literate educational resources (COOLERs) have been catching the attention of the education community in recent years. This set of learning materials mainly comprises digital notebook-like documents, which are stored, backed, and delivered from cloud services and eventually displayed in students' web browsers. As we demonstrate in this article, the advent of cloud architectures and the COVID-19 pandemic (which forced worldwide long-term distant academic environments) fortuitously teamed up with this learning and methodological trend by easing its use and fostering its adoption. With more detail, we have quantitatively measured the impact that the COOLER paradigm has had on the teaching realm by analyzing five years of logged data gathered by its current major player in the ecosystem: MyBinder. Among other results, we show how this growth in the production and delivery of notebooks made an important leap during the second SARS-CoV-2 wave (July–September 2020). However, the general usage trend seems to have strongly decreased after the end of the most recent seventh wave (September 2022), coinciding with the official end of the global health crisis and all the lockdown episodes. From these examined data, we conclude that COOLER and recent massive online learning scenarios have been very intimately linked. This fact may represent a flaw in the adoption of these exciting and useful learning materials.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"893-902"},"PeriodicalIF":3.7,"publicationDate":"2023-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139419503","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-18DOI: 10.1109/TLT.2023.3343525
Dapeng Qu;Ruiduo Li;Tianqi Yang;Songlin Wu;Yan Pan;Xingwei Wang;Keqin Li
There are many important and interesting academic competitions that attract an increasing number of students. However, traditional student team building methods usually have strong randomness or involve only some first-class students. To choose more suitable students to compose a team and improve students' abilities overall, a competition-oriented student team building method is proposed. This would not only lead to better competition results by choosing more suitable students and teams but also improve the overall involvement of students in considering education fairness. First, a Big Data platform is constructed to collect students' various behavior data. Based on that, a competition with a six-tuple attribute and a student with a six-tuple attribute are modeled. Then, a corresponding utility function is designed for each attribute in the student model to denote the student's utility in this attribute for attending a competition. Furthermore, a team utility function is developed for each team to denote the utilities of all involved students. A team building utility function is also developed to denote the utilities of all involved teams. Second, a multiple-objective particle swarm optimization algorithm with dimension by dimension improvement is proposed to build appropriate teams to optimize team building utility maximization and education fairness simultaneously. Finally, extensive experimental results demonstrate that the overall performance of our proposed team building method not only has better performance in terms of team utility and student ability than other current methods, but also has better performance in terms of hyper volume and inverted generational distance than other optimization algorithms.
{"title":"A Competition-Oriented Student Team Building Method","authors":"Dapeng Qu;Ruiduo Li;Tianqi Yang;Songlin Wu;Yan Pan;Xingwei Wang;Keqin Li","doi":"10.1109/TLT.2023.3343525","DOIUrl":"https://doi.org/10.1109/TLT.2023.3343525","url":null,"abstract":"There are many important and interesting academic competitions that attract an increasing number of students. However, traditional student team building methods usually have strong randomness or involve only some first-class students. To choose more suitable students to compose a team and improve students' abilities overall, a competition-oriented student team building method is proposed. This would not only lead to better competition results by choosing more suitable students and teams but also improve the overall involvement of students in considering education fairness. First, a Big Data platform is constructed to collect students' various behavior data. Based on that, a competition with a six-tuple attribute and a student with a six-tuple attribute are modeled. Then, a corresponding utility function is designed for each attribute in the student model to denote the student's utility in this attribute for attending a competition. Furthermore, a team utility function is developed for each team to denote the utilities of all involved students. A team building utility function is also developed to denote the utilities of all involved teams. Second, a multiple-objective particle swarm optimization algorithm with dimension by dimension improvement is proposed to build appropriate teams to optimize team building utility maximization and education fairness simultaneously. Finally, extensive experimental results demonstrate that the overall performance of our proposed team building method not only has better performance in terms of team utility and student ability than other current methods, but also has better performance in terms of hyper volume and inverted generational distance than other optimization algorithms.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"2020-2033"},"PeriodicalIF":2.9,"publicationDate":"2023-12-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142091070","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：