Pub Date : 2024-12-19DOI: 10.1109/TLT.2024.3520329
Rafael Herrero-Álvarez;Rafael Arnay;Eduardo Segredo;Gara Miranda;Coromoto León
RoblockLLy is an educational robotics simulator designed for primary and secondary school students, whose goal is to increase their interest in science, technology, engineering, and mathematics. In the particular case of computer science, it allows developing computational thinking skills. It has been designed with ease of use in mind. This free tool is available through a web browser and does not need a complex installation or specific hardware requirements, allowing educational robotics to be introduced to a wide range of users by working on practical projects that will help them understand key concepts of robotics and programming. The effectiveness of RoblockLLy has been validated based on motivation, usability, and user experience criteria. The tool was validated with 212 secondary school students (12–16 years old). Specifically, motivation was measured with the Intrinsic Motivation Inventory, usability with the System Usability Scale, and user experience with the User Experience Questionnaire. Generally speaking, the results demonstrate that students perceived RoblockLLy as a novel and interesting tool. The ratings for usability were predominantly positive, although a few students indicated a preference for expert assistance. The overall rating of the user experience was positive as well, yet notable differences in attitudes toward motivation and usability were observed between genders.
{"title":"Using RoblockLLy in the Classroom: Bridging the Gap in Computer Science Education Through Robotics Simulation","authors":"Rafael Herrero-Álvarez;Rafael Arnay;Eduardo Segredo;Gara Miranda;Coromoto León","doi":"10.1109/TLT.2024.3520329","DOIUrl":"https://doi.org/10.1109/TLT.2024.3520329","url":null,"abstract":"RoblockLLy is an educational robotics simulator designed for primary and secondary school students, whose goal is to increase their interest in science, technology, engineering, and mathematics. In the particular case of computer science, it allows developing computational thinking skills. It has been designed with ease of use in mind. This free tool is available through a web browser and does not need a complex installation or specific hardware requirements, allowing educational robotics to be introduced to a wide range of users by working on practical projects that will help them understand key concepts of robotics and programming. The effectiveness of RoblockLLy has been validated based on motivation, usability, and user experience criteria. The tool was validated with 212 secondary school students (12–16 years old). Specifically, motivation was measured with the Intrinsic Motivation Inventory, usability with the System Usability Scale, and user experience with the User Experience Questionnaire. Generally speaking, the results demonstrate that students perceived RoblockLLy as a novel and interesting tool. The ratings for usability were predominantly positive, although a few students indicated a preference for expert assistance. The overall rating of the user experience was positive as well, yet notable differences in attitudes toward motivation and usability were observed between genders.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"18 ","pages":"39-52"},"PeriodicalIF":2.9,"publicationDate":"2024-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10807249","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142937841","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-17DOI: 10.1109/TLT.2024.3514832
Philemon Yalamu;Abdullah Al Mahmud;Caslon Chua
Cultural differences can impact how learning is delivered in higher education. Past studies have overlooked the impacts of specific cultures on learning and technology integration, and therefore, this article explores aspects of culture and resource implications for technology-enabled learning. The study collected data from students, lecturers, and university administrators at the University of Papua New Guinea via questionnaire (n = 58), focus group (n = 15), and interview (n = 2). Descriptive statistics of questionnaire responses supplemented by conventional content analysis of interview and focus group responses revealed perceptions of higher education and teaching in Papua New Guinea (PNG). Often, learning technologies are adopted without careful consideration of the users’ requirements, especially their cultural influences, which sometimes hinder the effective implementation of chosen technologies for learning. This article discusses the key findings related to technology limitations, cultural influences, communication issues, and other challenges unique to PNG. It highlights that students and lecturers are keen to embrace technology in teaching and learning; however, institutional constraints hinder the successful implementation of technology that would allow them to advance learning. This, coupled with cultural influences such as traditional customs, practices, and belief systems, also affects how Western education is delivered. The article concludes with recommendations for implementing culturally sensitive learning management systems that can accommodate infrastructure and social challenges.
{"title":"Investigating Culture and Resource-Sensitive Technology-Enabled Learning","authors":"Philemon Yalamu;Abdullah Al Mahmud;Caslon Chua","doi":"10.1109/TLT.2024.3514832","DOIUrl":"https://doi.org/10.1109/TLT.2024.3514832","url":null,"abstract":"Cultural differences can impact how learning is delivered in higher education. Past studies have overlooked the impacts of specific cultures on learning and technology integration, and therefore, this article explores aspects of culture and resource implications for technology-enabled learning. The study collected data from students, lecturers, and university administrators at the University of Papua New Guinea via questionnaire (<italic>n</i> = 58), focus group (<italic>n</i> = 15), and interview (<italic>n</i> = 2). Descriptive statistics of questionnaire responses supplemented by conventional content analysis of interview and focus group responses revealed perceptions of higher education and teaching in Papua New Guinea (PNG). Often, learning technologies are adopted without careful consideration of the users’ requirements, especially their cultural influences, which sometimes hinder the effective implementation of chosen technologies for learning. This article discusses the key findings related to technology limitations, cultural influences, communication issues, and other challenges unique to PNG. It highlights that students and lecturers are keen to embrace technology in teaching and learning; however, institutional constraints hinder the successful implementation of technology that would allow them to advance learning. This, coupled with cultural influences such as traditional customs, practices, and belief systems, also affects how Western education is delivered. The article concludes with recommendations for implementing culturally sensitive learning management systems that can accommodate infrastructure and social challenges.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"18 ","pages":"85-99"},"PeriodicalIF":2.9,"publicationDate":"2024-12-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143106035","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-15DOI: 10.1109/TLT.2024.3499751
Tian Song;Hang Zhang;Yijia Xiao
High-quality programming projects for education are critically required in teaching. However, it is hard to develop those projects efficiently and artificially constrained by the lecturers' experience and background. The recent popularity of large language models (LLMs) has led to a great number of applications in the field of education, but concerns persist that the output might be unreliable when dealing with intricate requirements. In this study, we design a customized role-based agent (CRBA), which can be configured for different roles specializing in specific areas of expertise, making the LLM yield content of higher specialization. An iterative architecture of multi-CRBAs is proposed to generate multistep projects, where CRBAs automatically criticize and optimize the LLM's intermediate outputs to enhance quality. We propose ten evaluation metrics across three aspects to assess project quality through expert grading. Further, we conduct an A/B test among 60 undergraduate students in a programming course and collect their feedback through a questionnaire. According to the students' rating results, the LLM-generated projects have comparable performance to man-made ones in terms of project description, learning step setting, assistance to students, and overall project quality. This study effectively integrates LLM into educational scenarios and enhances the efficiency of creating high-quality and practical programming exercises for lecturers.
{"title":"A High-Quality Generation Approach for Educational Programming Projects Using LLM","authors":"Tian Song;Hang Zhang;Yijia Xiao","doi":"10.1109/TLT.2024.3499751","DOIUrl":"https://doi.org/10.1109/TLT.2024.3499751","url":null,"abstract":"High-quality programming projects for education are critically required in teaching. However, it is hard to develop those projects efficiently and artificially constrained by the lecturers' experience and background. The recent popularity of large language models (LLMs) has led to a great number of applications in the field of education, but concerns persist that the output might be unreliable when dealing with intricate requirements. In this study, we design a customized role-based agent (CRBA), which can be configured for different roles specializing in specific areas of expertise, making the LLM yield content of higher specialization. An iterative architecture of multi-CRBAs is proposed to generate multistep projects, where CRBAs automatically criticize and optimize the LLM's intermediate outputs to enhance quality. We propose ten evaluation metrics across three aspects to assess project quality through expert grading. Further, we conduct an A/B test among 60 undergraduate students in a programming course and collect their feedback through a questionnaire. According to the students' rating results, the LLM-generated projects have comparable performance to man-made ones in terms of project description, learning step setting, assistance to students, and overall project quality. This study effectively integrates LLM into educational scenarios and enhances the efficiency of creating high-quality and practical programming exercises for lecturers.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"2296-2309"},"PeriodicalIF":2.9,"publicationDate":"2024-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142777578","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-06DOI: 10.1109/TLT.2024.3492352
Peter H. F. Ng;Peter Q. Chen;Astin C. H. Wu;Ken S. K. Tai;Chen Li
This study examines a practical teaching and learning cycle tailored to integrate cutting-edge technologies (artificial intelligence (AI) and machine learning (ML) game development) and social entrepreneurship within a “STEM with meaning” approach. This cycle, rooted in service learning and the 5E constructivist teaching model (engage, explore, explain, elaborate, and evaluate), seeks to move beyond traditional lecture-based methods by promoting a deeper understanding of technology's societal impacts.Through a comparative analysis involving experimental and comparison groups, we evaluate the cycle's effectiveness in enhancing students' problem-solving skills, empathy, knowledge application, and sense of social responsibility—essential qualities for successful social entrepreneurs. This article contributes to the burgeoning field of entrepreneurship education by demonstrating the value of a pedagogical approach that combines AI, ML, and game development with a strong emphasis on social entrepreneurship. Our results advocate a shift toward educational models that prepare students with technical skills and the awareness and capabilities needed to address complex social issues. Through this research, we highlight the critical role of innovative teaching methods in cultivating the next generation of socially responsible entrepreneurs, thereby enriching both the educational landscape and society at large.
{"title":"Reimagining STEM Learning: A Comparative Analysis of Traditional and Service Learning Approaches for Social Entrepreneurship","authors":"Peter H. F. Ng;Peter Q. Chen;Astin C. H. Wu;Ken S. K. Tai;Chen Li","doi":"10.1109/TLT.2024.3492352","DOIUrl":"https://doi.org/10.1109/TLT.2024.3492352","url":null,"abstract":"This study examines a practical teaching and learning cycle tailored to integrate cutting-edge technologies (artificial intelligence (AI) and machine learning (ML) game development) and social entrepreneurship within a “STEM with meaning” approach. This cycle, rooted in service learning and the 5E constructivist teaching model (engage, explore, explain, elaborate, and evaluate), seeks to move beyond traditional lecture-based methods by promoting a deeper understanding of technology's societal impacts.Through a comparative analysis involving experimental and comparison groups, we evaluate the cycle's effectiveness in enhancing students' problem-solving skills, empathy, knowledge application, and sense of social responsibility—essential qualities for successful social entrepreneurs. This article contributes to the burgeoning field of entrepreneurship education by demonstrating the value of a pedagogical approach that combines AI, ML, and game development with a strong emphasis on social entrepreneurship. Our results advocate a shift toward educational models that prepare students with technical skills and the awareness and capabilities needed to address complex social issues. Through this research, we highlight the critical role of innovative teaching methods in cultivating the next generation of socially responsible entrepreneurs, thereby enriching both the educational landscape and society at large.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"2266-2280"},"PeriodicalIF":2.9,"publicationDate":"2024-11-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10745650","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142777849","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In recent years, lifelong learning has gained prominence, necessitating a continuous commitment from learners to enhance their skills and knowledge. During the lifelong learning process, it is essential to precisely assess the cognitive states of lifelong learners, as this will provide a learning report and further support subsequent learning activities. In the literature, researchers have proposed various cognitive diagnosis models (CDMs) to estimate the cognitive states based on learners' responses. However, learners' responses are noisy for different reasons, including guessing, slipping, accidentally clicking, and network issues. Rashly fitting the CDMs with noisy responses would yield imprecise cognitive state estimation. To tackle this problem, we first unify all types of noise underlying learners' responses. Then, we propose a novel diffusion-based cognitive diagnosis framework ( DiffCog ) to extend existing CDMs and enhance their effectiveness and robustness. DiffCog does so by addressing the following two technical challenges in denoising: 1) the hard-to-track problem and high computational cost in discrete and sparse responses and 2) the unknown extent of noise underlying responses. Specifically, DiffCog tackles these technical challenges by: 1) introducing responses encoders to project responses to continuous cognitive states for case of adding easy-to-track noise and reducing computation cost and 2) incorporating a time extractor and a denoise module to trace the noisy cognitive states back to the noise-free ones in a personalized way. We conduct extensive and sufficient experiments on three real-world datasets, and the results demonstrate that our proposed DiffCog not only elevates the performance ceiling of existing CDMs but also enhances their robustness to noise.
{"title":"A Diffusion-Based Cognitive Diagnosis Framework for Robust Learner Assessment","authors":"Guanhao Zhao;Zhenya Huang;Yan Zhuang;Haoyang Bi;Yiyan Wang;Fei Wang;Zhiyuan Ma;Yixia Zhao","doi":"10.1109/TLT.2024.3492214","DOIUrl":"https://doi.org/10.1109/TLT.2024.3492214","url":null,"abstract":"In recent years, lifelong learning has gained prominence, necessitating a continuous commitment from learners to enhance their skills and knowledge. During the lifelong learning process, it is essential to precisely assess the cognitive states of lifelong learners, as this will provide a learning report and further support subsequent learning activities. In the literature, researchers have proposed various cognitive diagnosis models (CDMs) to estimate the cognitive states based on learners' responses. However, learners' responses are noisy for different reasons, including guessing, slipping, accidentally clicking, and network issues. Rashly fitting the CDMs with noisy responses would yield imprecise cognitive state estimation. To tackle this problem, we first unify all types of noise underlying learners' responses. Then, we propose a novel diffusion-based cognitive diagnosis framework (\u0000<italic>DiffCog</i>\u0000) to extend existing CDMs and enhance their effectiveness and robustness. \u0000<italic>DiffCog</i>\u0000 does so by addressing the following two technical challenges in denoising: 1) the hard-to-track problem and high computational cost in discrete and sparse responses and 2) the unknown extent of noise underlying responses. Specifically, \u0000<italic>DiffCog</i>\u0000 tackles these technical challenges by: 1) introducing responses encoders to project responses to continuous cognitive states for case of adding easy-to-track noise and reducing computation cost and 2) incorporating a time extractor and a denoise module to trace the noisy cognitive states back to the noise-free ones in a personalized way. We conduct extensive and sufficient experiments on three real-world datasets, and the results demonstrate that our proposed DiffCog not only elevates the performance ceiling of existing CDMs but also enhances their robustness to noise.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"2281-2295"},"PeriodicalIF":2.9,"publicationDate":"2024-11-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142777848","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-06DOI: 10.1109/TLT.2024.3491864
Lifang Bai;Yijia Wei
ChatGPT can promptly reformulate a text and improve its quality in content and form while preserving the original meaning. Yet, little is known about how learners respond to such reformulations. Here, we employed a three-stage writing task (composing-comparison-rewriting) to investigate how learners notice, integrate, and perceive ChatGPT's reformulations as feedback in English as a foreign language (EFL) classroom context. We collected learners’ written notes made during the comparison of their original texts and the reformulations, categorizing them based on the language-related type of noticing (vocabulary, discourse, and form) and quality of noticing (depth of processing: low, intermediate, and high). We also analyzed the instances of reformulations integrated into learners’ rewriting and their answers to a questionnaire. The results showed that: 1) the reformulations directed learners’ attention to the gaps in their original writing, especially in word choice, and prompted them to integrate ChatGPT-generated changes in their rewriting; 2) the number of instances integrated into the rewriting was directly related to the quantity, type, and quality of noticing in the comparison stage; and 3) learners generally appreciated the pedagogical value of ChatGPT in EFL writing, particularly during the revision stage, although they believe occasionally ChatGPT might misinterpret their intentions. The study suggests that ChatGPT's reformulations should be complemented with peer and teacher feedback to create a comprehensive and personalized learning environment.
{"title":"Exploring EFL Learners’ Integration and Perceptions of ChatGPT's Text Revisions: A Three-Stage Writing Task Study","authors":"Lifang Bai;Yijia Wei","doi":"10.1109/TLT.2024.3491864","DOIUrl":"https://doi.org/10.1109/TLT.2024.3491864","url":null,"abstract":"ChatGPT can promptly reformulate a text and improve its quality in content and form while preserving the original meaning. Yet, little is known about how learners respond to such reformulations. Here, we employed a three-stage writing task (composing-comparison-rewriting) to investigate how learners notice, integrate, and perceive ChatGPT's reformulations as feedback in English as a foreign language (EFL) classroom context. We collected learners’ written notes made during the comparison of their original texts and the reformulations, categorizing them based on the language-related type of noticing (vocabulary, discourse, and form) and quality of noticing (depth of processing: low, intermediate, and high). We also analyzed the instances of reformulations integrated into learners’ rewriting and their answers to a questionnaire. The results showed that: 1) the reformulations directed learners’ attention to the gaps in their original writing, especially in word choice, and prompted them to integrate ChatGPT-generated changes in their rewriting; 2) the number of instances integrated into the rewriting was directly related to the quantity, type, and quality of noticing in the comparison stage; and 3) learners generally appreciated the pedagogical value of ChatGPT in EFL writing, particularly during the revision stage, although they believe occasionally ChatGPT might misinterpret their intentions. The study suggests that ChatGPT's reformulations should be complemented with peer and teacher feedback to create a comprehensive and personalized learning environment.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"2215-2226"},"PeriodicalIF":2.9,"publicationDate":"2024-11-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142713849","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-05DOI: 10.1109/TLT.2024.3492073
Clemens Drieschner;Ferdinand Xiong;Florian Bajraktari;Matthias C. Utesch;Jörg Weking;Helmut Krcmar
The growing demand for sustainable products and services has led to the emergence of a range of entrepreneurial opportunities for innovative business solutions. These new opportunities require novel entrepreneurial skills to design, grow, and maintain sustainable businesses. However, existing educational games do not address the specific skills needed for sustainable entrepreneurship. This work introduces an interactive business game that enables learning in an engaging and practical way to prepare learners to become the entrepreneurs of tomorrow. Our business game is intended to increase learners’ self-assessed skills and self-confidence by at least one point on the five-point Likert scale. The game is embedded into a didactic framework accompanied by a lecture and discussion part where learners manage their bike rental business and self-reflect on their decisions. The game scenario allows the players to guide their business through three growth stages: survival, growth, and expansion. In this study, 151 high school students participated in the evaluation across all stages. They assessed their knowledge in a pretest and a posttest and judged the developed game on its ease of use. The learners’ sustainable entrepreneurship knowledge and skills improved significantly. This study thus confirms the mixed approach as a viable teaching tool for this area of expertise. Furthermore, a holistic didactic framework with an introduction to the game and subsequent debriefing contributes to the success of the business game too. A sign test and bootstrapping with confidence intervals constituted a simple means of statistically quantifying the learners’ improvement.
{"title":"Measuring the Effectiveness of Learning Economic Principles in a Sustainable Entrepreneurship Game-Based Setting: A Bootstrapping Approach","authors":"Clemens Drieschner;Ferdinand Xiong;Florian Bajraktari;Matthias C. Utesch;Jörg Weking;Helmut Krcmar","doi":"10.1109/TLT.2024.3492073","DOIUrl":"https://doi.org/10.1109/TLT.2024.3492073","url":null,"abstract":"The growing demand for sustainable products and services has led to the emergence of a range of entrepreneurial opportunities for innovative business solutions. These new opportunities require novel entrepreneurial skills to design, grow, and maintain sustainable businesses. However, existing educational games do not address the specific skills needed for sustainable entrepreneurship. This work introduces an interactive business game that enables learning in an engaging and practical way to prepare learners to become the entrepreneurs of tomorrow. Our business game is intended to increase learners’ self-assessed skills and self-confidence by at least one point on the five-point Likert scale. The game is embedded into a didactic framework accompanied by a lecture and discussion part where learners manage their bike rental business and self-reflect on their decisions. The game scenario allows the players to guide their business through three growth stages: survival, growth, and expansion. In this study, 151 high school students participated in the evaluation across all stages. They assessed their knowledge in a pretest and a posttest and judged the developed game on its ease of use. The learners’ sustainable entrepreneurship knowledge and skills improved significantly. This study thus confirms the mixed approach as a viable teaching tool for this area of expertise. Furthermore, a holistic didactic framework with an introduction to the game and subsequent debriefing contributes to the success of the business game too. A sign test and bootstrapping with confidence intervals constituted a simple means of statistically quantifying the learners’ improvement.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"18 ","pages":"1-12"},"PeriodicalIF":2.9,"publicationDate":"2024-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10745173","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142859204","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-04DOI: 10.1109/TLT.2024.3491801
Yuto Tomikawa;Ayaka Suzuki;Masaki Uto
The automatic generation of reading comprehension questions, referred to as question generation (QG), is attracting attention in the field of education. To achieve efficient educational applications of QG methods, it is desirable to generate questions with difficulty levels that are appropriate for each learner's reading ability. Therefore, in recent years, several difficulty-controllable QG methods have been proposed. However, conventional methods generate only questions and cannot produce question–answer pairs. Furthermore, such methods ignore the relationship between question difficulty and learner ability, making it challenging to ascertain the appropriate difficulty levels for each learner. To address these issues, in this article, we propose a method for generating question–answer pairs based on difficulty, defined using a statistical model known as item response theory. The proposed difficulty-controllable generation is achieved by extending two pretrained transformer models: bidirectional encoder representations from transformers and text-to-text transfer transformer. In addition, because learners' abilities are generally not knowable in advance, we propose an adaptive QG framework that efficiently estimates the learners' abilities while generating and presenting questions with difficulty levels suitable for their abilities. Through experiments involving real data, we confirmed that the proposed method can generate question–answer pairs with difficulty levels that align with the learners' abilities while efficiently estimating their abilities.
{"title":"Adaptive Question–Answer Generation With Difficulty Control Using Item Response Theory and Pretrained Transformer Models","authors":"Yuto Tomikawa;Ayaka Suzuki;Masaki Uto","doi":"10.1109/TLT.2024.3491801","DOIUrl":"https://doi.org/10.1109/TLT.2024.3491801","url":null,"abstract":"The automatic generation of reading comprehension questions, referred to as question generation (QG), is attracting attention in the field of education. To achieve efficient educational applications of QG methods, it is desirable to generate questions with difficulty levels that are appropriate for each learner's reading ability. Therefore, in recent years, several difficulty-controllable QG methods have been proposed. However, conventional methods generate only questions and cannot produce question–answer pairs. Furthermore, such methods ignore the relationship between question difficulty and learner ability, making it challenging to ascertain the appropriate difficulty levels for each learner. To address these issues, in this article, we propose a method for generating question–answer pairs based on difficulty, defined using a statistical model known as item response theory. The proposed difficulty-controllable generation is achieved by extending two pretrained transformer models: bidirectional encoder representations from transformers and text-to-text transfer transformer. In addition, because learners' abilities are generally not knowable in advance, we propose an adaptive QG framework that efficiently estimates the learners' abilities while generating and presenting questions with difficulty levels suitable for their abilities. Through experiments involving real data, we confirmed that the proposed method can generate question–answer pairs with difficulty levels that align with the learners' abilities while efficiently estimating their abilities.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"2240-2252"},"PeriodicalIF":2.9,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10742557","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142713980","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-30DOI: 10.1109/TLT.2024.3486630
Jiwon Kim;Jack Miller;Kexin Wang;Michael C. Dorneich;Eliot Winer;Lori J. Brown
This study introduces an augmented reality (AR) authoring tool tailored for flight instructors without technical expertise. While AR offers potential in aviation weather education and instructors desire to use it in the classroom, they face challenges due to limited digital proficiency and complexity of authoring tools. Many existing AR authoring tools prioritize technical aspects over user experience. To address these challenges, a no-programming-required AR authoring tool was developed based on instructor-informed requirements, such as incorporating features of flight waypoints and weather phenomena. A total of 41 participants tested the tool by crafting three AR learning modules. After using the tool, there was a significant increase in participants’ confidence in AR content creation (+30%), AR authoring process (+51%), and interactive AR development (+50%). In addition, there was a significant decrease in their concerns about technical complexity (–19%), mental effort (–30%), and time consumption (–30%). Participants rated the incorporated functions highly preferable and indicated the tool has high usability. Participants completed the most challenging task quickly and with a low cognitive load. The findings demonstrate the tool's effectiveness in enabling participants to competently and efficiently author AR content, reducing technical concerns. Such tools can facilitate the integration of AR technology into the classroom, offering students improved access to interactive 3-D visualizations of dynamic subjects, such as aviation weather, which require students to mentally visualize weather conditions and understand their manifestations.
本研究介绍了一种为没有专业技术知识的飞行教员量身定制的增强现实（AR）创作工具。虽然增强现实技术在航空气象教育方面具有潜力，教员们也希望在课堂上使用这种技术，但由于数字技术能力有限和制作工具的复杂性，他们面临着挑战。许多现有的 AR 制作工具都将技术方面的问题置于用户体验之上。为了应对这些挑战，我们根据教员提出的要求，开发了一种无需编程的 AR 创作工具，例如将飞行航点和天气现象的特征融入其中。共有 41 名学员通过制作三个 AR 学习模块对该工具进行了测试。使用该工具后，学员在 AR 内容创建（+30%）、AR 创作过程（+51%）和交互式 AR 开发（+50%）方面的信心有了显著提高。此外，他们对技术复杂性（-19%）、脑力劳动（-30%）和时间消耗（-30%）的担忧也明显减少。参与者对纳入的功能给予了很高的评价，并表示该工具具有很高的可用性。参与者以较低的认知负荷快速完成了最具挑战性的任务。研究结果表明，该工具能够有效地帮助参与者胜任并高效地编写 AR 内容，减少了技术方面的顾虑。这种工具可以促进 AR 技术与课堂的整合，为学生提供更好的机会，使他们能够获得动态主题的交互式三维可视化内容，例如航空天气，这需要学生在头脑中将天气状况可视化并理解其表现形式。
{"title":"Empowering Instructors: Augmented Reality Authoring Toolkit for Aviation Weather Education","authors":"Jiwon Kim;Jack Miller;Kexin Wang;Michael C. Dorneich;Eliot Winer;Lori J. Brown","doi":"10.1109/TLT.2024.3486630","DOIUrl":"https://doi.org/10.1109/TLT.2024.3486630","url":null,"abstract":"This study introduces an augmented reality (AR) authoring tool tailored for flight instructors without technical expertise. While AR offers potential in aviation weather education and instructors desire to use it in the classroom, they face challenges due to limited digital proficiency and complexity of authoring tools. Many existing AR authoring tools prioritize technical aspects over user experience. To address these challenges, a no-programming-required AR authoring tool was developed based on instructor-informed requirements, such as incorporating features of flight waypoints and weather phenomena. A total of 41 participants tested the tool by crafting three AR learning modules. After using the tool, there was a significant increase in participants’ confidence in AR content creation (+30%), AR authoring process (+51%), and interactive AR development (+50%). In addition, there was a significant decrease in their concerns about technical complexity (–19%), mental effort (–30%), and time consumption (–30%). Participants rated the incorporated functions highly preferable and indicated the tool has high usability. Participants completed the most challenging task quickly and with a low cognitive load. The findings demonstrate the tool's effectiveness in enabling participants to competently and efficiently author AR content, reducing technical concerns. Such tools can facilitate the integration of AR technology into the classroom, offering students improved access to interactive 3-D visualizations of dynamic subjects, such as aviation weather, which require students to mentally visualize weather conditions and understand their manifestations.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"2195-2206"},"PeriodicalIF":2.9,"publicationDate":"2024-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636251","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：