Pub Date : 2024-10-30DOI: 10.1109/TLT.2024.3486630
Jiwon Kim;Jack Miller;Kexin Wang;Michael C. Dorneich;Eliot Winer;Lori J. Brown
This study introduces an augmented reality (AR) authoring tool tailored for flight instructors without technical expertise. While AR offers potential in aviation weather education and instructors desire to use it in the classroom, they face challenges due to limited digital proficiency and complexity of authoring tools. Many existing AR authoring tools prioritize technical aspects over user experience. To address these challenges, a no-programming-required AR authoring tool was developed based on instructor-informed requirements, such as incorporating features of flight waypoints and weather phenomena. A total of 41 participants tested the tool by crafting three AR learning modules. After using the tool, there was a significant increase in participants’ confidence in AR content creation (+30%), AR authoring process (+51%), and interactive AR development (+50%). In addition, there was a significant decrease in their concerns about technical complexity (–19%), mental effort (–30%), and time consumption (–30%). Participants rated the incorporated functions highly preferable and indicated the tool has high usability. Participants completed the most challenging task quickly and with a low cognitive load. The findings demonstrate the tool's effectiveness in enabling participants to competently and efficiently author AR content, reducing technical concerns. Such tools can facilitate the integration of AR technology into the classroom, offering students improved access to interactive 3-D visualizations of dynamic subjects, such as aviation weather, which require students to mentally visualize weather conditions and understand their manifestations.
本研究介绍了一种为没有专业技术知识的飞行教员量身定制的增强现实（AR）创作工具。虽然增强现实技术在航空气象教育方面具有潜力，教员们也希望在课堂上使用这种技术，但由于数字技术能力有限和制作工具的复杂性，他们面临着挑战。许多现有的 AR 制作工具都将技术方面的问题置于用户体验之上。为了应对这些挑战，我们根据教员提出的要求，开发了一种无需编程的 AR 创作工具，例如将飞行航点和天气现象的特征融入其中。共有 41 名学员通过制作三个 AR 学习模块对该工具进行了测试。使用该工具后，学员在 AR 内容创建（+30%）、AR 创作过程（+51%）和交互式 AR 开发（+50%）方面的信心有了显著提高。此外，他们对技术复杂性（-19%）、脑力劳动（-30%）和时间消耗（-30%）的担忧也明显减少。参与者对纳入的功能给予了很高的评价，并表示该工具具有很高的可用性。参与者以较低的认知负荷快速完成了最具挑战性的任务。研究结果表明，该工具能够有效地帮助参与者胜任并高效地编写 AR 内容，减少了技术方面的顾虑。这种工具可以促进 AR 技术与课堂的整合，为学生提供更好的机会，使他们能够获得动态主题的交互式三维可视化内容，例如航空天气，这需要学生在头脑中将天气状况可视化并理解其表现形式。
{"title":"Empowering Instructors: Augmented Reality Authoring Toolkit for Aviation Weather Education","authors":"Jiwon Kim;Jack Miller;Kexin Wang;Michael C. Dorneich;Eliot Winer;Lori J. Brown","doi":"10.1109/TLT.2024.3486630","DOIUrl":"https://doi.org/10.1109/TLT.2024.3486630","url":null,"abstract":"This study introduces an augmented reality (AR) authoring tool tailored for flight instructors without technical expertise. While AR offers potential in aviation weather education and instructors desire to use it in the classroom, they face challenges due to limited digital proficiency and complexity of authoring tools. Many existing AR authoring tools prioritize technical aspects over user experience. To address these challenges, a no-programming-required AR authoring tool was developed based on instructor-informed requirements, such as incorporating features of flight waypoints and weather phenomena. A total of 41 participants tested the tool by crafting three AR learning modules. After using the tool, there was a significant increase in participants’ confidence in AR content creation (+30%), AR authoring process (+51%), and interactive AR development (+50%). In addition, there was a significant decrease in their concerns about technical complexity (–19%), mental effort (–30%), and time consumption (–30%). Participants rated the incorporated functions highly preferable and indicated the tool has high usability. Participants completed the most challenging task quickly and with a low cognitive load. The findings demonstrate the tool's effectiveness in enabling participants to competently and efficiently author AR content, reducing technical concerns. Such tools can facilitate the integration of AR technology into the classroom, offering students improved access to interactive 3-D visualizations of dynamic subjects, such as aviation weather, which require students to mentally visualize weather conditions and understand their manifestations.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"2195-2206"},"PeriodicalIF":2.9,"publicationDate":"2024-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636251","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-09DOI: 10.1109/TLT.2024.3351352
Oscar Blessed Deho;Lin Liu;Jiuyong Li;Jixue Liu;Chen Zhan;Srecko Joksimovic
Learning analytics (LA), like much of machine learning, assumes the training and test datasets come from the same distribution. Therefore, LA models built on past observations are (implicitly) expected to work well for future observations. However, this assumption does not always hold in practice because the dataset may drift. Recently, algorithmic fairness has gained significant attention. Nevertheless, algorithmic fairness research has paid little attention to dataset drift. Majority of the existing fairness algorithms are “statically” designed. Put another way, LA models tuned to be “fair” on past data are expected to still be “fair” when dealing with current/future data. However, it is counter-intuitive to deploy a statically fair algorithm to a nonstationary world. There is, therefore, a need to assess the impact of dataset drift on the unfairness of LA models. For this reason, we investigate the relationship between dataset drift and unfairness of LA models. Specifically, we first measure the degree of drift in the features (i.e., covariates) and target label of our dataset. After that, we train predictive models on the dataset and evaluate the relationship between the dataset drift and the unfairness of the predictive models. Our findings suggest a directly proportional relationship between dataset drift and unfairness. Further, we find covariate drift to have the most impact on unfairness of models as compared to target drift, and there are no guarantees that a once fair model would consistently remain fair. Our findings imply that “robustness” of fair LA models to dataset drift is necessary before deployment.
学习分析（LA）与大部分机器学习一样，都假定训练数据集和测试数据集来自相同的分布。因此，基于过去观察结果建立的学习分析模型（隐含地）有望在未来的观察结果中发挥良好的作用。然而，这一假设在实践中并不总是成立的，因为数据集可能会漂移。最近，算法公平性受到了广泛关注。然而，算法公平性研究很少关注数据集漂移问题。现有的大多数公平性算法都是 "静态 "设计的。换句话说，在过去的数据上调整为 "公平 "的洛杉矶模型，在处理当前/未来的数据时预计仍然是 "公平 "的。然而，将静态公平算法应用于非稳态世界是违背直觉的。因此，有必要评估数据集漂移对 LA 模型公平性的影响。为此，我们研究了数据集漂移与 LA 模型不公平性之间的关系。具体来说，我们首先测量数据集的特征（即协变量）和目标标签的漂移程度。然后，我们在数据集上训练预测模型，并评估数据集漂移与预测模型不公平程度之间的关系。我们的研究结果表明，数据集漂移与不公平之间存在正比关系。此外，我们发现与目标漂移相比，协变量漂移对模型不公平程度的影响最大，而且无法保证曾经公平的模型会一直保持公平。我们的研究结果表明，在部署公平的洛杉矶模型之前，必须使其对数据集漂移具有 "稳健性"。
{"title":"When the Past != The Future: Assessing the Impact of Dataset Drift on the Fairness of Learning Analytics Models","authors":"Oscar Blessed Deho;Lin Liu;Jiuyong Li;Jixue Liu;Chen Zhan;Srecko Joksimovic","doi":"10.1109/TLT.2024.3351352","DOIUrl":"https://doi.org/10.1109/TLT.2024.3351352","url":null,"abstract":"Learning analytics (LA), like much of machine learning, assumes the training and test datasets come from the same distribution. Therefore, LA models built on past observations are (implicitly) expected to work well for future observations. However, this assumption does not always hold in practice because the dataset may drift. Recently, algorithmic fairness has gained significant attention. Nevertheless, algorithmic fairness research has paid little attention to dataset drift. Majority of the existing fairness algorithms are “statically” designed. Put another way, LA models \u0000<italic>tuned</i>\u0000 to be “fair” on past data are expected to still be “fair” when dealing with current/future data. However, it is counter-intuitive to deploy a \u0000<italic>statically</i>\u0000 fair algorithm to a \u0000<italic>nonstationary</i>\u0000 world. There is, therefore, a need to assess the impact of dataset drift on the unfairness of LA models. For this reason, we investigate the relationship between dataset drift and unfairness of LA models. Specifically, we first measure the degree of drift in the features (i.e., covariates) and target label of our dataset. After that, we train predictive models on the dataset and evaluate the relationship between the dataset drift and the unfairness of the predictive models. Our findings suggest a directly proportional relationship between dataset drift and unfairness. Further, we find covariate drift to have the most impact on unfairness of models as compared to target drift, and there are no guarantees that a once fair model would consistently remain fair. Our findings imply that “robustness” of fair LA models to dataset drift is necessary before deployment.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1007-1020"},"PeriodicalIF":3.7,"publicationDate":"2024-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139572651","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Knowledge tracing (KT), the task of tracking the knowledge state of a student over time, has been assessed actively by artificial intelligence researchers. Recent reports have described that Deep-IRT, which combines item response theory (IRT) with a deep learning method, provides superior performance. It can express the abilities of each student and the difficulty of each item such as IRT. Nevertheless, its interpretability is inadequate compared to that of IRT because the ability parameter depends on each item. Deep-IRT implicitly assumes that items with the same skills are equivalent, which does not hold when item difficulties for the same skills differ greatly. For identical skills, items that are not equivalent hinder the interpretation of a student's ability estimate. To overcome those difficulties, this study proposes a novel Deep-IRT that models a student response to an item using two independent networks: 1) a student network and 2) an item network. The proposed Deep-IRT method learns student parameters and item parameters independently to avoid impairing the predictive accuracy. Moreover, we propose a novel hypernetwork architecture for the proposed Deep-IRT to balance both the current and the past data in the latent variable storing student's knowledge states. Results of experiments with six benchmark datasets demonstrate that the proposed method improves the prediction accuracy by about 2.0%, on average. In addition, experiments for the simulation dataset demonstrated that the proposed method provides a stronger correlation with true parameters than the earlier Deep-IRT method does at the $p< 0.5$ significance level.
{"title":"Deep Knowledge Tracing Incorporating a Hypernetwork With Independent Student and Item Networks","authors":"Emiko Tsutsumi;Yiming Guo;Ryo Kinoshita;Maomi Ueno","doi":"10.1109/TLT.2023.3346671","DOIUrl":"https://doi.org/10.1109/TLT.2023.3346671","url":null,"abstract":"Knowledge tracing (KT), the task of tracking the knowledge state of a student over time, has been assessed actively by artificial intelligence researchers. Recent reports have described that Deep-IRT, which combines item response theory (IRT) with a deep learning method, provides superior performance. It can express the abilities of each student and the difficulty of each item such as IRT. Nevertheless, its interpretability is inadequate compared to that of IRT because the ability parameter depends on each item. Deep-IRT implicitly assumes that items with the same skills are equivalent, which does not hold when item difficulties for the same skills differ greatly. For identical skills, items that are not equivalent hinder the interpretation of a student's ability estimate. To overcome those difficulties, this study proposes a novel Deep-IRT that models a student response to an item using two independent networks: 1) a student network and 2) an item network. The proposed Deep-IRT method learns student parameters and item parameters independently to avoid impairing the predictive accuracy. Moreover, we propose a novel hypernetwork architecture for the proposed Deep-IRT to balance both the current and the past data in the latent variable storing student's knowledge states. Results of experiments with six benchmark datasets demonstrate that the proposed method improves the prediction accuracy by about 2.0%, on average. In addition, experiments for the simulation dataset demonstrated that the proposed method provides a stronger correlation with true parameters than the earlier Deep-IRT method does at the \u0000<inline-formula><tex-math>$p< 0.5$</tex-math></inline-formula>\u0000 significance level.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"951-965"},"PeriodicalIF":3.7,"publicationDate":"2023-12-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10373110","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139504424","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-19DOI: 10.1109/TLT.2023.3343690
Alberto Corbi;Daniel Burgos;Antonio María Pérez
Literate programming and cloud-operated open literate educational resources (COOLERs) have been catching the attention of the education community in recent years. This set of learning materials mainly comprises digital notebook-like documents, which are stored, backed, and delivered from cloud services and eventually displayed in students' web browsers. As we demonstrate in this article, the advent of cloud architectures and the COVID-19 pandemic (which forced worldwide long-term distant academic environments) fortuitously teamed up with this learning and methodological trend by easing its use and fostering its adoption. With more detail, we have quantitatively measured the impact that the COOLER paradigm has had on the teaching realm by analyzing five years of logged data gathered by its current major player in the ecosystem: MyBinder. Among other results, we show how this growth in the production and delivery of notebooks made an important leap during the second SARS-CoV-2 wave (July–September 2020). However, the general usage trend seems to have strongly decreased after the end of the most recent seventh wave (September 2022), coinciding with the official end of the global health crisis and all the lockdown episodes. From these examined data, we conclude that COOLER and recent massive online learning scenarios have been very intimately linked. This fact may represent a flaw in the adoption of these exciting and useful learning materials.
{"title":"Cloud-Operated Open Literate Educational Resources: The Case of the MyBinder","authors":"Alberto Corbi;Daniel Burgos;Antonio María Pérez","doi":"10.1109/TLT.2023.3343690","DOIUrl":"https://doi.org/10.1109/TLT.2023.3343690","url":null,"abstract":"Literate programming and cloud-operated open literate educational resources (COOLERs) have been catching the attention of the education community in recent years. This set of learning materials mainly comprises digital notebook-like documents, which are stored, backed, and delivered from cloud services and eventually displayed in students' web browsers. As we demonstrate in this article, the advent of cloud architectures and the COVID-19 pandemic (which forced worldwide long-term distant academic environments) fortuitously teamed up with this learning and methodological trend by easing its use and fostering its adoption. With more detail, we have quantitatively measured the impact that the COOLER paradigm has had on the teaching realm by analyzing five years of logged data gathered by its current major player in the ecosystem: MyBinder. Among other results, we show how this growth in the production and delivery of notebooks made an important leap during the second SARS-CoV-2 wave (July–September 2020). However, the general usage trend seems to have strongly decreased after the end of the most recent seventh wave (September 2022), coinciding with the official end of the global health crisis and all the lockdown episodes. From these examined data, we conclude that COOLER and recent massive online learning scenarios have been very intimately linked. This fact may represent a flaw in the adoption of these exciting and useful learning materials.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"893-902"},"PeriodicalIF":3.7,"publicationDate":"2023-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139419503","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-15DOI: 10.1109/TLT.2023.3324843
Anasol Peña-Rios;Junjie Gavin Wu
The metaverse is seen as an evolution paradigm of the next-generation Internet, able to support a diverse range of persistent and always-on interconnected synchronous multiuser virtual environments where people can engage with others in real time, merging the physical and virtual world [1], [2], [3]. The concept was first mentioned in 1992s Neal Stephenson novel “ Snow Crash ” [4], and it follows the web and mobile Internet revolutions, allowing users to experience virtual environments in an immersive and hyperspatiotemporal manner [1]. Thus, it represents a paradigm shift in digital interaction, enabling real-time, multidimensional experiences that transcend the boundaries of physical space with the promise of bringing new levels of social connection and collaboration. The metaverse exists within the Internet, but not in the traditional way of seeing the world through a screen [1]. Instead, the metaverse aims to provide immersive experiences based on the convergence of spatial computing technologies that enable multisensory user interactions [e.g., virtual reality (VR), augmented reality (AR), and mixed reality (MR)] [2], [3] combined with 3-D data and artificial intelligence. The metaverse is also related to the concept of digital twins (DTs), which are digital replicas of elements in the real world (e.g., assets and processes) that mirror and synchronize in real time with their source, creating a bidirectional connection between them. While DTs focus more on the bidirectional connection between real and virtual and the accuracy of the representation toward better decision-making, the metaverse looks at sociotechnical challenges of seamless embodied communication between users and the dynamic interactions with the virtual spaces.
{"title":"Guest Editorial The Metaverse and the Future of Education","authors":"Anasol Peña-Rios;Junjie Gavin Wu","doi":"10.1109/TLT.2023.3324843","DOIUrl":"https://doi.org/10.1109/TLT.2023.3324843","url":null,"abstract":"The metaverse is seen as an evolution paradigm of the next-generation Internet, able to support a diverse range of persistent and always-on interconnected synchronous multiuser virtual environments where people can engage with others in real time, merging the physical and virtual world [1], [2], [3]. The concept was first mentioned in 1992s Neal Stephenson novel “\u0000<italic>Snow Crash</i>\u0000” [4], and it follows the web and mobile Internet revolutions, allowing users to experience virtual environments in an immersive and \u0000<italic>hyperspatiotemporal</i>\u0000 manner [1]. Thus, it represents a paradigm shift in digital interaction, enabling real-time, multidimensional experiences that transcend the boundaries of physical space with the promise of bringing new levels of social connection and collaboration. The metaverse exists within the Internet, but not in the traditional way of seeing the world through a screen [1]. Instead, the metaverse aims to provide immersive experiences based on the convergence of spatial computing technologies that enable multisensory user interactions [e.g., virtual reality (VR), augmented reality (AR), and mixed reality (MR)] [2], [3] combined with 3-D data and artificial intelligence. The metaverse is also related to the concept of digital twins (DTs), which are digital replicas of elements in the real world (e.g., assets and processes) that mirror and synchronize in real time with their source, creating a bidirectional connection between them. While DTs focus more on the bidirectional connection between real and virtual and the accuracy of the representation toward better decision-making, the metaverse looks at sociotechnical challenges of seamless embodied communication between users and the dynamic interactions with the virtual spaces.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"16 6","pages":"887-891"},"PeriodicalIF":3.7,"publicationDate":"2023-12-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10361565","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138678658","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-15DOI: 10.1109/TLT.2023.3329612
{"title":"IEEE Education Society Information","authors":"","doi":"10.1109/TLT.2023.3329612","DOIUrl":"https://doi.org/10.1109/TLT.2023.3329612","url":null,"abstract":"","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"16 6","pages":"C2-C2"},"PeriodicalIF":3.7,"publicationDate":"2023-12-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10361583","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138678714","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-23DOI: 10.1109/TLT.2023.3336240
Liqing Qiu;Menglin Zhu;Jingcheng Zhou
Knowledge tracing (KT) is essential in intelligent tutoring systems for tracking learners' knowledge states and predicting their future performance. Numerous prevailing KT methods prioritize modeling learners' behavioral patterns in acquiring knowledge and the relationship among interactions. However, due to the sparsity problem, they frequently encounter challenges in effectively uncovering latent contextual features embedded within the learning sequences. This limitation may impose certain constraints on the predictive performance. In light of this concern, this article focuses on extracting latent features from learning sequences to enhance the assessment of knowledge states. Consequently, we design optimized pretraining mechanisms and introduce an enhanced deep KT method, optimized pretraining deep KT (OPKT). In the pretraining phase, the self-supervised learning approach is effectively employed to train comprehensive contextual encodings of the learning sequences. During fine-tuning, the contextual encodings are transferred to the downstream KT model, which then generates the knowledge states and makes predictions. Through our experiments, the superiority of our method over six existing KT models on five publicly available datasets is demonstrated. Furthermore, extensive ablation studies and visualized analysis validate the rationality and effectiveness of every component of the OPKT architecture.
{"title":"OPKT: Enhancing Knowledge Tracing With Optimized Pretraining Mechanisms in Intelligent Tutoring","authors":"Liqing Qiu;Menglin Zhu;Jingcheng Zhou","doi":"10.1109/TLT.2023.3336240","DOIUrl":"https://doi.org/10.1109/TLT.2023.3336240","url":null,"abstract":"Knowledge tracing (KT) is essential in intelligent tutoring systems for tracking learners' knowledge states and predicting their future performance. Numerous prevailing KT methods prioritize modeling learners' behavioral patterns in acquiring knowledge and the relationship among interactions. However, due to the sparsity problem, they frequently encounter challenges in effectively uncovering latent contextual features embedded within the learning sequences. This limitation may impose certain constraints on the predictive performance. In light of this concern, this article focuses on extracting latent features from learning sequences to enhance the assessment of knowledge states. Consequently, we design optimized pretraining mechanisms and introduce an enhanced deep KT method, optimized pretraining deep KT (OPKT). In the pretraining phase, the self-supervised learning approach is effectively employed to train comprehensive contextual encodings of the learning sequences. During fine-tuning, the contextual encodings are transferred to the downstream KT model, which then generates the knowledge states and makes predictions. Through our experiments, the superiority of our method over six existing KT models on five publicly available datasets is demonstrated. Furthermore, extensive ablation studies and visualized analysis validate the rationality and effectiveness of every component of the OPKT architecture.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"841-855"},"PeriodicalIF":3.7,"publicationDate":"2023-11-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139419505","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-16DOI: 10.1109/TLT.2023.3333669
Shi Dong;Xueyun Tao;Rui Zhong;Zhifeng Wang;Mingzhang Zuo;Jianwen Sun
Higher education is rapidly growing in the online learning landscape. However, current personalized recommendation techniques struggle with the precise extraction of complex mathematical semantics, hindering accurate perception of learners' cognitive states and relevance of recommendations. This article proposes a framework for extracting complex mathematical semantics and providing personalized exercise recommendations. We design a tree-based position encoding method to enhance the accuracy of positional representation for mathematical expressions in the pretrained model, aiming to improve the performance of downstream tasks. We propose an automatic method for extracting knowledge attributes based on expert annotations, enabling interpretable cognitive diagnosis. Furthermore, we employ sequential pattern mining to discover the knowledge usage patterns in exercises, generate learning paths using a multilayer knowledge graph, and leverage cognitive diagnostic results to enhance the relevance of recommendations. Experimental results show a 2.0% improvement in mathematical symbol embedding on mathematical formula retrieval tasks and knowledge attribute extraction accuracy ranging from 66.5% to 81.7%. Learners' posttest scores significantly improve during group testing with good consistency between online cognitive diagnosis and self-diagnosis.
{"title":"Advanced Mathematics Exercise Recommendation Based on Automatic Knowledge Extraction and Multilayer Knowledge Graph","authors":"Shi Dong;Xueyun Tao;Rui Zhong;Zhifeng Wang;Mingzhang Zuo;Jianwen Sun","doi":"10.1109/TLT.2023.3333669","DOIUrl":"https://doi.org/10.1109/TLT.2023.3333669","url":null,"abstract":"Higher education is rapidly growing in the online learning landscape. However, current personalized recommendation techniques struggle with the precise extraction of complex mathematical semantics, hindering accurate perception of learners' cognitive states and relevance of recommendations. This article proposes a framework for extracting complex mathematical semantics and providing personalized exercise recommendations. We design a tree-based position encoding method to enhance the accuracy of positional representation for mathematical expressions in the pretrained model, aiming to improve the performance of downstream tasks. We propose an automatic method for extracting knowledge attributes based on expert annotations, enabling interpretable cognitive diagnosis. Furthermore, we employ sequential pattern mining to discover the knowledge usage patterns in exercises, generate learning paths using a multilayer knowledge graph, and leverage cognitive diagnostic results to enhance the relevance of recommendations. Experimental results show a 2.0% improvement in mathematical symbol embedding on mathematical formula retrieval tasks and knowledge attribute extraction accuracy ranging from 66.5% to 81.7%. Learners' posttest scores significantly improve during group testing with good consistency between online cognitive diagnosis and self-diagnosis.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"776-793"},"PeriodicalIF":3.7,"publicationDate":"2023-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139419504","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-15DOI: 10.1109/TLT.2023.3333029
Wei Qiu;Andy W. H. Khong;S. Supraja;Wenyin Tang
Predicting student performance in an academic institution is important for detecting at-risk students and to administer early intervention strategies. In this article, we develop a new architecture that achieves grade prediction based only on grades achieved over past semesters. Our proposed architecture involves two stages—weighted loss function incorporated to the long short-term memory (LSTM) model in the first stage, followed by a short-term gated LSTM in the second. The weighted loss function in the first stage ensures low prediction error by weighing loss associated with the minority class label (in our case the at-risk label). The short-term gated LSTM in the second stage, on the other hand, models short-term variations in academic performance to suppress any residual false alarms. Experiment results using three datasets obtained from over 20 000 students across 17 undergraduate courses show that the proposed model achieves a 28.8% improvement in F1 score compared to the LSTM model for at-risk detection. Students identified as at-risk have also been presented and validated by counselors via a dashboard.
{"title":"A Dual-Mode Grade Prediction Architecture for Identifying At-Risk Students","authors":"Wei Qiu;Andy W. H. Khong;S. Supraja;Wenyin Tang","doi":"10.1109/TLT.2023.3333029","DOIUrl":"10.1109/TLT.2023.3333029","url":null,"abstract":"Predicting student performance in an academic institution is important for detecting at-risk students and to administer early intervention strategies. In this article, we develop a new architecture that achieves grade prediction based only on grades achieved over past semesters. Our proposed architecture involves two stages—weighted loss function incorporated to the long short-term memory (LSTM) model in the first stage, followed by a short-term gated LSTM in the second. The weighted loss function in the first stage ensures low prediction error by weighing loss associated with the minority class label (in our case the at-risk label). The short-term gated LSTM in the second stage, on the other hand, models short-term variations in academic performance to suppress any residual false alarms. Experiment results using three datasets obtained from over 20 000 students across 17 undergraduate courses show that the proposed model achieves a 28.8% improvement in F1 score compared to the LSTM model for at-risk detection. Students identified as at-risk have also been presented and validated by counselors via a dashboard.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"803-814"},"PeriodicalIF":3.7,"publicationDate":"2023-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135709792","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-15DOI: 10.1109/TLT.2023.3332500
Rotem Israel-Fishelson;Dan Kohen-Vacs
Educational dashboards enable students to monitor and reflect on academic performance and administrative aspects of the learning processes. Occasionally, educational institutions integrate dashboards using the information found in their learning management systems or their students' information desks. Learning analytics offers ways to enrich these dashboards and expose students to analyzed information beyond the monitored data provided such as smart recommendations. Despite the large variety of dashboards, the students’ centric perspective and the ability to adapt the dashboard to their personal needs is not a common practice. To identify and support the needs of students who wish to track aspects of their learning routine, it is very important to position the students at the core of the design process of these dashboards. This article presents a new phase in our research to expand our understanding of the students' needs in monitoring their educational routines and preferences while using an advanced form of a learning analytics dashboard. We propose an optimized approach for designing educational dashboards. In this sense, we examine and seek to integrate the components that are prominently required by students. Hence, we address both the type of components as well as their arrangement within the customized dashboard. The outcomes of our efforts reveal findings concerning students’ trends and habits when exploiting these dashboards. It also offers pivotal insights and recommendations for the optimized implementation of learning analytics dashboards that are aligned with the students’ authentic requirements.
{"title":"Towards Optimization of Learning Analytics Dashboards That are Customized for the Students’ Requirements","authors":"Rotem Israel-Fishelson;Dan Kohen-Vacs","doi":"10.1109/TLT.2023.3332500","DOIUrl":"10.1109/TLT.2023.3332500","url":null,"abstract":"Educational dashboards enable students to monitor and reflect on academic performance and administrative aspects of the learning processes. Occasionally, educational institutions integrate dashboards using the information found in their learning management systems or their students' information desks. Learning analytics offers ways to enrich these dashboards and expose students to analyzed information beyond the monitored data provided such as smart recommendations. Despite the large variety of dashboards, the students’ centric perspective and the ability to adapt the dashboard to their personal needs is not a common practice. To identify and support the needs of students who wish to track aspects of their learning routine, it is very important to position the students at the core of the design process of these dashboards. This article presents a new phase in our research to expand our understanding of the students' needs in monitoring their educational routines and preferences while using an advanced form of a learning analytics dashboard. We propose an optimized approach for designing educational dashboards. In this sense, we examine and seek to integrate the components that are prominently required by students. Hence, we address both the type of components as well as their arrangement within the customized dashboard. The outcomes of our efforts reveal findings concerning students’ trends and habits when exploiting these dashboards. It also offers pivotal insights and recommendations for the optimized implementation of learning analytics dashboards that are aligned with the students’ authentic requirements.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"794-802"},"PeriodicalIF":3.7,"publicationDate":"2023-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135709637","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：