Modern online education has the capacity to provide intelligent educational services by automatically analyzing substantial amounts of student behavioral data. Knowledge tracing (KT) is one of the fundamental tasks for student behavioral data analysis, aiming to monitor students' evolving knowledge state during their problem-solving process. In recent years, a substantial number of studies have concentrated on this rapidly growing field, significantly contributing to its advancements. In this survey, we will conduct a thorough investigation of these progressions. First, we present three types of fundamental KT models with distinct technical routes. Subsequently, we review extensive variants of the fundamental KT models that consider more stringent learning assumptions. Moreover, the development of KT cannot be separated from its applications, so we present typical KT applications in various scenarios. To facilitate the work of researchers and practitioners in this field, we have developed two open-source algorithm libraries: EduData that enables the downloading and preprocessing of KT-related datasets, and EduKTM that provides an extensible and unified implementation of existing mainstream KT models. Finally, we discuss potential directions for future research in this rapidly growing field. We hope that the current survey will assist both researchers and practitioners in fostering the development of KT, thereby benefiting a broader range of students.
{"title":"A Survey of Knowledge Tracing: Models, Variants, and Applications","authors":"Shuanghong Shen;Qi Liu;Zhenya Huang;Yonghe Zheng;Minghao Yin;Minjuan Wang;Enhong Chen","doi":"10.1109/TLT.2024.3383325","DOIUrl":"10.1109/TLT.2024.3383325","url":null,"abstract":"Modern online education has the capacity to provide intelligent educational services by automatically analyzing substantial amounts of student behavioral data. Knowledge tracing (KT) is one of the fundamental tasks for student behavioral data analysis, aiming to monitor students' evolving knowledge state during their problem-solving process. In recent years, a substantial number of studies have concentrated on this rapidly growing field, significantly contributing to its advancements. In this survey, we will conduct a thorough investigation of these progressions. First, we present three types of fundamental KT models with distinct technical routes. Subsequently, we review extensive variants of the fundamental KT models that consider more stringent learning assumptions. Moreover, the development of KT cannot be separated from its applications, so we present typical KT applications in various scenarios. To facilitate the work of researchers and practitioners in this field, we have developed two open-source algorithm libraries: EduData that enables the downloading and preprocessing of KT-related datasets, and EduKTM that provides an extensible and unified implementation of existing mainstream KT models. Finally, we discuss potential directions for future research in this rapidly growing field. We hope that the current survey will assist both researchers and practitioners in fostering the development of KT, thereby benefiting a broader range of students.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1898-1919"},"PeriodicalIF":2.9,"publicationDate":"2024-04-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140570861","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-08DOI: 10.1109/TLT.2024.3386464
Hashem A. Almusawi;Christopher M. Durugbo
Digital technologies, such as wearables, offer immense potential for active and enhanced interactive, collaborative, and immersive learning. Wearable technologies are digital devices that can be worn on or near the human body as accessories or clothing. Interest and innovativeness in the educational use of such technologies depend on teacher attitudes, and a significant challenge for education and technology research is examining the factors that relate to these attitudes. The aim of this study is to examine the determinants of teacher attitudes toward wearables and their influence on personal innovativeness in the use of wearable technology. The study involves a cross-sectional survey of 346 physical education teachers. Using partial least squared structural equation modeling, the study provides new analytical insights into affectiveness, teacher beliefs, and perceived mattering, as a triad of determining factors for teacher attitudes and personal innovativeness in wearable technology use. Results reflect the need for institutional initiatives to foster positive attitudes and perceptions toward wearables and for teacher upskilling through training and development in innovative use of digital technologies.
{"title":"Determinants of Teacher Attitudes and Innovative Use of Wearable Technology","authors":"Hashem A. Almusawi;Christopher M. Durugbo","doi":"10.1109/TLT.2024.3386464","DOIUrl":"10.1109/TLT.2024.3386464","url":null,"abstract":"Digital technologies, such as wearables, offer immense potential for active and enhanced interactive, collaborative, and immersive learning. Wearable technologies are digital devices that can be worn on or near the human body as accessories or clothing. Interest and innovativeness in the educational use of such technologies depend on teacher attitudes, and a significant challenge for education and technology research is examining the factors that relate to these attitudes. The aim of this study is to examine the determinants of teacher attitudes toward wearables and their influence on personal innovativeness in the use of wearable technology. The study involves a cross-sectional survey of 346 physical education teachers. Using partial least squared structural equation modeling, the study provides new analytical insights into affectiveness, teacher beliefs, and perceived mattering, as a triad of determining factors for teacher attitudes and personal innovativeness in wearable technology use. Results reflect the need for institutional initiatives to foster positive attitudes and perceptions toward wearables and for teacher upskilling through training and development in innovative use of digital technologies.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1428-1439"},"PeriodicalIF":3.7,"publicationDate":"2024-04-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140570612","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-08DOI: 10.1109/TLT.2024.3386095
Jose Barambones;Cristian Moral;Angélica de Antonio;Ricardo Imbert;Loïc Martínez-Normand;Elena Villalba-Mora
Before interacting with real users, developers must be proficient in human–computer interaction (HCI) so as not to exhaust user patience and availability. For that, substantial training and practice are required, but it is costly to create a variety of high-quality HCI training materials. In this context, chat generative pretrained transformer (ChatGPT) and other chatbots based on large language models (LLMs) offer an opportunity to generate training materials of acceptable quality without foregoing specific human characteristics present in real-world scenarios. Personas is a user-centered design method that encompasses fictitious but believable user archetypes to help designers understand and empathize with their target audience during product design. We conducted an exploratory study on the Personas technique, addressing the validity and believability of interviews designed by HCI trainers and answered by ChatGPT-simulated users, which can be used as training material for persona creation. Specifically, we employed ChatGPT to respond to interviews designed by user experience (UX) experts. Two groups, HCI professors and professionals, then evaluated the validity of the generated materials considering quality, usefulness, UX, and ethics. The results show that both groups rated the interviews as believable and helpful for Personas training. However, some concerns about response repetition and low response variability suggested the need for further research on improved prompt design in order to generate more diverse and well-developed responses. The findings of this study provide insight into how HCI trainers can use ChatGPT to help their students master persona creation skills before working with real users in real-world scenarios for the first time.
在与真实用户互动之前,开发人员必须精通人机交互(HCI),以免耗尽用户的耐心和可用性。为此,需要大量的培训和练习,但制作各种高质量的人机交互培训材料成本高昂。在这种情况下,聊天生成预训练转换器(ChatGPT)和其他基于大型语言模型(LLMs)的聊天机器人提供了一个机会,既能生成质量可接受的培训材料,又不会放弃现实世界场景中存在的特定人类特征。角色是一种以用户为中心的设计方法,它包含虚构但可信的用户原型,帮助设计师在产品设计过程中了解目标受众并与之产生共鸣。我们对 Personas 技术进行了一项探索性研究,探讨了由人机交互培训师设计、由 ChatGPT 模拟用户回答的访谈的有效性和可信度,这些访谈可用作创建角色的培训材料。具体来说,我们使用 ChatGPT 来回答用户体验(UX)专家设计的访谈。然后,人机交互教授和专业人士组成的两组人分别从质量、实用性、用户体验和道德等方面对生成材料的有效性进行了评估。结果显示,两组人都认为访谈是可信的,对 Personas 培训有帮助。然而,一些人对回答重复和回答可变性低表示担忧,这表明有必要进一步研究如何改进提示设计,以生成更多样、更完善的回答。本研究的结果为人机交互培训师如何使用 ChatGPT 帮助学生掌握角色创建技能提供了启示,然后再与真实用户在真实场景中进行首次合作。
{"title":"ChatGPT for Learning HCI Techniques: A Case Study on Interviews for Personas","authors":"Jose Barambones;Cristian Moral;Angélica de Antonio;Ricardo Imbert;Loïc Martínez-Normand;Elena Villalba-Mora","doi":"10.1109/TLT.2024.3386095","DOIUrl":"10.1109/TLT.2024.3386095","url":null,"abstract":"Before interacting with real users, developers must be proficient in human–computer interaction (HCI) so as not to exhaust user patience and availability. For that, substantial training and practice are required, but it is costly to create a variety of high-quality HCI training materials. In this context, chat generative pretrained transformer (ChatGPT) and other chatbots based on large language models (LLMs) offer an opportunity to generate training materials of acceptable quality without foregoing specific human characteristics present in real-world scenarios. Personas is a user-centered design method that encompasses fictitious but believable user archetypes to help designers understand and empathize with their target audience during product design. We conducted an exploratory study on the Personas technique, addressing the validity and believability of interviews designed by HCI trainers and answered by ChatGPT-simulated users, which can be used as training material for persona creation. Specifically, we employed ChatGPT to respond to interviews designed by user experience (UX) experts. Two groups, HCI professors and professionals, then evaluated the validity of the generated materials considering quality, usefulness, UX, and ethics. The results show that both groups rated the interviews as believable and helpful for Personas training. However, some concerns about response repetition and low response variability suggested the need for further research on improved prompt design in order to generate more diverse and well-developed responses. The findings of this study provide insight into how HCI trainers can use ChatGPT to help their students master persona creation skills before working with real users in real-world scenarios for the first time.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1486-1501"},"PeriodicalIF":3.7,"publicationDate":"2024-04-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10494570","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140570856","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
3D-printing (3DP) is a rapidly evolving sector and is advancing at an unmatched pace. Integrating digital-based approaches to disseminate knowledge within institutional curricula is crucial for mainstreaming knowledge. This work explores the development of a virtual lab (VL) for additive manufacturing (AM), using an interactive VL simulator. The VL's experiments encompass anatomy, assembly, techniques, preprocessing, and postprocessing modules on AM. The aim of this dedicated AM VL is to facilitate digital education on 3DP among educational institutions. A case study compares the efficacy of VL to physical experimentation. Two groups of 30 students each provided feedback before and after conducting experiments in the physical laboratory and on the VL, respectively. The comparison elucidates how integrating the VL with traditional teaching pedagogy can be a progressive step in teaching AM in educational institutions. The teaching pedagogy approach is both cost-effective and time efficient. The developed VL offers unlimited access and availability for students to learn at their own pace. The design of the 3DP virtual simulation lab incorporates an inclusive align–act–assess approach, wherein each module includes related theory, aim, procedure, simulator interface, pretest, and posttest to align with this approach.
三维打印(3DP)是一个快速发展的行业,其发展速度无与伦比。在机构课程中整合基于数字的方法来传播知识对于知识主流化至关重要。这项工作利用交互式虚拟实验室模拟器,探索了增材制造(AM)虚拟实验室(VL)的开发。虚拟实验室的实验内容包括 AM 的解剖、装配、技术、预处理和后处理模块。该专用 AM VL 的目的是促进教育机构的 3DP 数字化教育。一项案例研究比较了 VL 与物理实验的功效。两组各 30 名学生分别在物理实验室和 VL 上进行实验前后提供了反馈意见。比较结果阐明了将 VL 与传统教学法相结合如何能成为教育机构 AM 教学的一个进步步骤。这种教学法既经济又省时。开发的虚拟实验室为学生提供了无限的访问和可用性,让他们按照自己的节奏学习。3DP 虚拟仿真实验室的设计采用了 "统一-行动-评估 "的方法,其中每个模块都包括相关理论、目的、程序、仿真器界面、前测和后测,以符合这一方法。
{"title":"3D-Printing Virtual Simulation Lab","authors":"Ishant Singhal;Guru Ratan Satsangee;Lakshya Bhardwaj;Gaurang S. Sharma;Anand Swarup Chandrakar;Hritav Gupta;Gargi Malik;Bobby Tyagi;Ankit Sahai;Rahul Swarup Sharma","doi":"10.1109/TLT.2024.3384556","DOIUrl":"10.1109/TLT.2024.3384556","url":null,"abstract":"3D-printing (3DP) is a rapidly evolving sector and is advancing at an unmatched pace. Integrating digital-based approaches to disseminate knowledge within institutional curricula is crucial for mainstreaming knowledge. This work explores the development of a virtual lab (VL) for additive manufacturing (AM), using an interactive VL simulator. The VL's experiments encompass anatomy, assembly, techniques, preprocessing, and postprocessing modules on AM. The aim of this dedicated AM VL is to facilitate digital education on 3DP among educational institutions. A case study compares the efficacy of VL to physical experimentation. Two groups of 30 students each provided feedback before and after conducting experiments in the physical laboratory and on the VL, respectively. The comparison elucidates how integrating the VL with traditional teaching pedagogy can be a progressive step in teaching AM in educational institutions. The teaching pedagogy approach is both cost-effective and time efficient. The developed VL offers unlimited access and availability for students to learn at their own pace. The design of the 3DP virtual simulation lab incorporates an inclusive align–act–assess approach, wherein each module includes related theory, aim, procedure, simulator interface, pretest, and posttest to align with this approach.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1530-1543"},"PeriodicalIF":3.7,"publicationDate":"2024-04-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140570321","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Blended learning (BL) has become increasingly popular in higher education institutions. Despite its popularity and the advances in methodologies for the detection of learning tactics and strategies from trace data, little is known about how they apply to BL settings and, therefore, how students use them to plan, organize, monitor, and regulate their learning in these settings. To address this gap, we analyzed the manifestations of learning tactics and strategies of 267 students across three undergraduate-level BL courses with different course designs, instructional activities, and learning contexts. We employed a data-driven method that incorporates hidden Markov models to determine students’ learning tactics. Then, we employed optimal matching to identify the students’ strategies based on the sequences of tactics they deployed and how they relate to their self-reported self-regulated learning (SRL) skills. Our results indicate that students’ tactics and strategies varied significantly depending on the course design and learning context. Tactics with regard to the use of time management resources were common across courses. In contrast, tactics deployed when revisiting old material and interacting with an SRL support tool were course-specific. We identified strategies related to surface and deep learning and found that surface-level strategies manifested consistently across all courses. These findings contribute to a better understanding of student learning mechanisms in BL environments and have implications for instructional design and SRL support.
{"title":"Exploring Manifestations of Learners’ Self-Regulated Tactics and Strategies Across Blended Learning Courses","authors":"Esteban Villalobos;Mar Pérez-Sanagustín;Roger Azevedo;Cédric Sanza;Julien Broisin","doi":"10.1109/TLT.2024.3385641","DOIUrl":"10.1109/TLT.2024.3385641","url":null,"abstract":"Blended learning (BL) has become increasingly popular in higher education institutions. Despite its popularity and the advances in methodologies for the detection of learning tactics and strategies from trace data, little is known about how they apply to BL settings and, therefore, how students use them to plan, organize, monitor, and regulate their learning in these settings. To address this gap, we analyzed the manifestations of learning tactics and strategies of 267 students across three undergraduate-level BL courses with different course designs, instructional activities, and learning contexts. We employed a data-driven method that incorporates hidden Markov models to determine students’ learning tactics. Then, we employed optimal matching to identify the students’ strategies based on the sequences of tactics they deployed and how they relate to their self-reported self-regulated learning (SRL) skills. Our results indicate that students’ tactics and strategies varied significantly depending on the course design and learning context. Tactics with regard to the use of time management resources were common across courses. In contrast, tactics deployed when revisiting old material and interacting with an SRL support tool were course-specific. We identified strategies related to surface and deep learning and found that surface-level strategies manifested consistently across all courses. These findings contribute to a better understanding of student learning mechanisms in BL environments and have implications for instructional design and SRL support.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1544-1557"},"PeriodicalIF":3.7,"publicationDate":"2024-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140570615","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-05DOI: 10.1109/TLT.2024.3385505
Shujia Fan;Brian Yecies;Zeyang Ivy Zhou;Jun Shen
The metaverse, along with its various Web 3.0 subdomains, represents a ground-breaking extension of both the physical and digital worlds. In this emerging landscape, the real and virtual worlds are integrating in a way that allows for seamless interactions, creating an immersive experience. This integration has significant implications for diverse fields, particularly in reshaping both online and traditional education methods. By analyzing 417 comprehensive white papers released in 2022 and 2023 from leading consulting firms and think tanks, and incorporating insights from academic articles, we have extracted key information about how metaverse technologies are influencing education over time. Our investigation unveils the key impacts of metaverse technologies on the educational landscape, contributing to a more profound understanding of the transformative effects of the metaverse on the educational terrain in flux. Moreover, our study provides a holistic perspective on the advantages and disadvantages associated with metaverse education, offering in-depth insights into the challenges involved in seamlessly integrating the metaverse into educational practices. Furthermore, our research also highlights the challenges and opportunities presented by the metaverse and its impact on new educational paradigms.
元宇宙及其各种 Web 3.0 子域代表了物理世界和数字世界的突破性延伸。在这一新兴领域,现实世界和虚拟世界正在以一种无缝互动的方式融为一体,创造出一种身临其境的体验。这种融合对各个领域都有重大影响,尤其是在重塑在线教育和传统教育方法方面。通过分析领先咨询公司和智库在 2022 年和 2023 年发布的 417 份综合白皮书,并结合学术文章中的见解,我们提取了有关元虚拟技术如何随着时间的推移影响教育的关键信息。我们的调查揭示了元世界技术对教育领域的关键影响,有助于人们更深刻地理解元世界对不断变化的教育领域的变革性影响。此外,我们的研究还从整体角度探讨了与元数据教育相关的优势和劣势,深入揭示了将元数据无缝融入教育实践所面临的挑战。此外,我们的研究还强调了元海外带来的挑战和机遇及其对新教育范式的影响。
{"title":"Challenges and Opportunities for the Web 3.0 Metaverse Turn in Education","authors":"Shujia Fan;Brian Yecies;Zeyang Ivy Zhou;Jun Shen","doi":"10.1109/TLT.2024.3385505","DOIUrl":"10.1109/TLT.2024.3385505","url":null,"abstract":"The metaverse, along with its various Web 3.0 subdomains, represents a ground-breaking extension of both the physical and digital worlds. In this emerging landscape, the real and virtual worlds are integrating in a way that allows for seamless interactions, creating an immersive experience. This integration has significant implications for diverse fields, particularly in reshaping both online and traditional education methods. By analyzing 417 comprehensive white papers released in 2022 and 2023 from leading consulting firms and think tanks, and incorporating insights from academic articles, we have extracted key information about how metaverse technologies are influencing education over time. Our investigation unveils the key impacts of metaverse technologies on the educational landscape, contributing to a more profound understanding of the transformative effects of the metaverse on the educational terrain in flux. Moreover, our study provides a holistic perspective on the advantages and disadvantages associated with metaverse education, offering in-depth insights into the challenges involved in seamlessly integrating the metaverse into educational practices. Furthermore, our research also highlights the challenges and opportunities presented by the metaverse and its impact on new educational paradigms.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1989-2004"},"PeriodicalIF":2.9,"publicationDate":"2024-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140570865","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-04DOI: 10.1109/TLT.2024.3385009
Elizabeth Koh;Lishan Zhang;Alwyn Vwen Yen Lee;Hongye Wang
Generative artificial intelligence (AI) has the potential to revolutionize teaching and learning applications. This article examines the word cloud, a toolkit often used to scaffold teaching and learning for reflection, critical thinking, and content learning. Addressing the issues in traditional word clouds, semantic word clouds have been developed but they are technically challenging to develop and still problematic. However, generative AI has the potential to develop efficient, accurate, creative, and accessible word clouds. Three different methods representing three major approaches of word cloud generation were developed, implemented, and user evaluated—traditional (baseline), semantic (natural language processing enhanced), and generative AI (generative pretrained transformer based)—in two different language contexts—Chinese (China case) and English (Singapore case). The findings of the study show the technical robustness of the methods, as well as provide key pedagogical insights from the user perspective of instructors of higher education courses in China and Singapore. Implications to the design of word clouds and their application in teaching and learning are discussed.
{"title":"Revolutionizing Word Clouds for Teaching and Learning With Generative Artificial Intelligence: Cases From China and Singapore","authors":"Elizabeth Koh;Lishan Zhang;Alwyn Vwen Yen Lee;Hongye Wang","doi":"10.1109/TLT.2024.3385009","DOIUrl":"10.1109/TLT.2024.3385009","url":null,"abstract":"Generative artificial intelligence (AI) has the potential to revolutionize teaching and learning applications. This article examines the word cloud, a toolkit often used to scaffold teaching and learning for reflection, critical thinking, and content learning. Addressing the issues in traditional word clouds, semantic word clouds have been developed but they are technically challenging to develop and still problematic. However, generative AI has the potential to develop efficient, accurate, creative, and accessible word clouds. Three different methods representing three major approaches of word cloud generation were developed, implemented, and user evaluated—traditional (baseline), semantic (natural language processing enhanced), and generative AI (generative pretrained transformer based)—in two different language contexts—Chinese (China case) and English (Singapore case). The findings of the study show the technical robustness of the methods, as well as provide key pedagogical insights from the user perspective of instructors of higher education courses in China and Singapore. Implications to the design of word clouds and their application in teaching and learning are discussed.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1416-1427"},"PeriodicalIF":3.7,"publicationDate":"2024-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140570307","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This study explores and analyzes the specific performance of large language models (LLMs) in instructional design, aiming to unveil their potential strengths and possible weaknesses. Recently, the influence of LLMs has gradually increased in multiple fields, yet exploratory research on their application in education remains relatively scarce. In response to this situation, our research, grounded in pedagogical content knowledge theory, initially formulated an instructional design framework based on mathematical problem chains and corresponding prompt instructions. Subsequently, a comprehensive tool for assessing LLM's instructional design capabilities was developed. Utilizing Generative Pretrained Transformer 4, a high school mathematics teaching plan dataset was generated. Finally, the performance of LLMs in instructional design was evaluated. The evaluation results revealed that the teaching plans generated by LLMs excel in setting instructional objectives, identifying teaching priorities, organizing problem chains and teaching activities, articulating subject content, and selecting methods and strategies. Particularly commendable performance was noted in the modules of statistics and functions. However, there is room for improvement in aspects related to mathematical culture and interdisciplinary assessment, as well as in the geometry and algebra modules. Lastly, this study proposes initiatives, such as LLM prompt-based teacher training and the integration of mathematics-focused LLMs. These suggestions aim to advance personalized instructional design and professional development of teachers, offering educators new insights into the in-depth application of LLMs.
{"title":"Teaching Plan Generation and Evaluation With GPT-4: Unleashing the Potential of LLM in Instructional Design","authors":"Bihao Hu;Longwei Zheng;Jiayi Zhu;Lishan Ding;Yilei Wang;Xiaoqing Gu","doi":"10.1109/TLT.2024.3384765","DOIUrl":"10.1109/TLT.2024.3384765","url":null,"abstract":"This study explores and analyzes the specific performance of large language models (LLMs) in instructional design, aiming to unveil their potential strengths and possible weaknesses. Recently, the influence of LLMs has gradually increased in multiple fields, yet exploratory research on their application in education remains relatively scarce. In response to this situation, our research, grounded in pedagogical content knowledge theory, initially formulated an instructional design framework based on mathematical problem chains and corresponding prompt instructions. Subsequently, a comprehensive tool for assessing LLM's instructional design capabilities was developed. Utilizing Generative Pretrained Transformer 4, a high school mathematics teaching plan dataset was generated. Finally, the performance of LLMs in instructional design was evaluated. The evaluation results revealed that the teaching plans generated by LLMs excel in setting instructional objectives, identifying teaching priorities, organizing problem chains and teaching activities, articulating subject content, and selecting methods and strategies. Particularly commendable performance was noted in the modules of statistics and functions. However, there is room for improvement in aspects related to mathematical culture and interdisciplinary assessment, as well as in the geometry and algebra modules. Lastly, this study proposes initiatives, such as LLM prompt-based teacher training and the integration of mathematics-focused LLMs. These suggestions aim to advance personalized instructional design and professional development of teachers, offering educators new insights into the in-depth application of LLMs.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1471-1485"},"PeriodicalIF":3.7,"publicationDate":"2024-04-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140570320","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-02DOI: 10.1109/TLT.2024.3384290
Ka-Yan Fung;Kit-Yi Tang;Tze Leung Rick Lui;Kuen-Fung Sin;Lik-Hang Lee;Huamin Qu;Shenghui Song
Prescreening children for specific learning disabilities, e.g., dyslexia, is essential for effective intervention. With a quick and reliable prescreening result, special education coordinators (SENCOs) can provide students with early intervention and relieve their learning pressure. Unfortunately, due to the limited resources, many students in Hong Kong receive dyslexia assessments beyond the golden period, i.e., under the age of six. To this end, information technology could establish automatic prescreening tools to address this issue. However, dyslexia prescreening for children learning Chinese is challenging due to the lack of sound–script correlation in Chinese. In this article, an automatic dyslexia prescreening system (ADPS) is developed to provide a quick test to identify at-risk children. Through a two-stage approach, we first develop a gamified tool based on linguistic characteristics and then evaluate the result by a comparison study. Results from a pilot test on 30 students with dyslexia and 32 students without dyslexia indicate that the ADPS can effectively distinguish between two groups of students. Furthermore, the interactive design elements can motivate students to conduct the prescreening independently.
{"title":"ADPS—A Prescreening Tool for Students With Dyslexia in Learning Traditional Chinese","authors":"Ka-Yan Fung;Kit-Yi Tang;Tze Leung Rick Lui;Kuen-Fung Sin;Lik-Hang Lee;Huamin Qu;Shenghui Song","doi":"10.1109/TLT.2024.3384290","DOIUrl":"10.1109/TLT.2024.3384290","url":null,"abstract":"Prescreening children for specific learning disabilities, e.g., dyslexia, is essential for effective intervention. With a quick and reliable prescreening result, special education coordinators (SENCOs) can provide students with early intervention and relieve their learning pressure. Unfortunately, due to the limited resources, many students in Hong Kong receive dyslexia assessments beyond the golden period, i.e., under the age of six. To this end, information technology could establish automatic prescreening tools to address this issue. However, dyslexia prescreening for children learning Chinese is challenging due to the lack of sound–script correlation in Chinese. In this article, an automatic dyslexia prescreening system (ADPS) is developed to provide a quick test to identify at-risk children. Through a two-stage approach, we first develop a gamified tool based on linguistic characteristics and then evaluate the result by a comparison study. Results from a pilot test on 30 students with dyslexia and 32 students without dyslexia indicate that the ADPS can effectively distinguish between two groups of students. Furthermore, the interactive design elements can motivate students to conduct the prescreening independently.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1454-1470"},"PeriodicalIF":3.7,"publicationDate":"2024-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140570310","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-01DOI: 10.1109/TLT.2024.3383773
Thi Thuy An Ngo;Thanh Tu Tran;Gia Khuong An;Phuong Thy Nguyen
The growing prevalence of advanced generative artificial intelligence chatbots, such as ChatGPT, in the educational sector has raised considerable interest in understanding their impact on student knowledge and exploring effective and sustainable implementation strategies. This research investigates the influence of knowledge management factors on the continuous usage of ChatGPT for educational purposes while concurrently evaluating student satisfaction with its use in learning. Using a quantitative approach, a structured questionnaire was administered to 513 Vietnamese university students via Google Forms for data collection. The partial least squares structural equation modeling statistical technique was employed to examine the relationships between identified factors and evaluate the research model. The results provided strong support for several hypotheses, revealing significant positive effects of expectation confirmation on perceived usefulness and satisfaction, as well as perceived usefulness on user satisfaction and continuous usage of ChatGPT. These findings suggest that when students recognize the usefulness of ChatGPT for their learnings, they experience higher satisfaction and are more likely to continue using it. In addition, knowledge acquisition significantly impacts both satisfaction and continuous usage of ChatGPT, while knowledge sharing and application influence satisfaction exclusively. This indicates that students prioritize knowledge acquisition over sharing and applying knowledge through ChatGPT. The study has theoretical and practical implications for ChatGPT developers, educators, and future research. Theoretically, it contributes to understanding satisfaction and continuous usage in educational settings, utilizing the expectation confirmation model and integrating knowledge management factors. Practically, it provides insights into comprehension and suggestions for enhancing user satisfaction and continuous usage of ChatGPT in education.
{"title":"ChatGPT for Educational Purposes: Investigating the Impact of Knowledge Management Factors on Student Satisfaction and Continuous Usage","authors":"Thi Thuy An Ngo;Thanh Tu Tran;Gia Khuong An;Phuong Thy Nguyen","doi":"10.1109/TLT.2024.3383773","DOIUrl":"https://doi.org/10.1109/TLT.2024.3383773","url":null,"abstract":"The growing prevalence of advanced generative artificial intelligence chatbots, such as ChatGPT, in the educational sector has raised considerable interest in understanding their impact on student knowledge and exploring effective and sustainable implementation strategies. This research investigates the influence of knowledge management factors on the continuous usage of ChatGPT for educational purposes while concurrently evaluating student satisfaction with its use in learning. Using a quantitative approach, a structured questionnaire was administered to 513 Vietnamese university students via Google Forms for data collection. The partial least squares structural equation modeling statistical technique was employed to examine the relationships between identified factors and evaluate the research model. The results provided strong support for several hypotheses, revealing significant positive effects of expectation confirmation on perceived usefulness and satisfaction, as well as perceived usefulness on user satisfaction and continuous usage of ChatGPT. These findings suggest that when students recognize the usefulness of ChatGPT for their learnings, they experience higher satisfaction and are more likely to continue using it. In addition, knowledge acquisition significantly impacts both satisfaction and continuous usage of ChatGPT, while knowledge sharing and application influence satisfaction exclusively. This indicates that students prioritize knowledge acquisition over sharing and applying knowledge through ChatGPT. The study has theoretical and practical implications for ChatGPT developers, educators, and future research. Theoretically, it contributes to understanding satisfaction and continuous usage in educational settings, utilizing the expectation confirmation model and integrating knowledge management factors. Practically, it provides insights into comprehension and suggestions for enhancing user satisfaction and continuous usage of ChatGPT in education.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1367-1378"},"PeriodicalIF":3.7,"publicationDate":"2024-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140550172","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}