Pub Date : 2024-09-19DOI: 10.1109/TLT.2024.3464560
Yu Bai;Jun Li;Jun Shen;Liang Zhao
The potential of artificial intelligence (AI) in transforming education has received considerable attention. This study aims to explore the potential of large language models (LLMs) in assisting students with studying and passing standardized exams, while many people think it is a hype situation. Using primary education as an example, this research investigates whether ChatGPT-3.5 can achieve satisfactory performance on the Chinese Primary School Exams and whether it can be used as a teaching aid or tutor. We designed an experimental framework and constructed a benchmark that comprises 4800 questions collected from 48 tasks in Chinese elementary education settings. Through automatic and manual evaluations, we observed that ChatGPT-3.5’s pass rate was below the required level of accuracy for most tasks, and the correctness of ChatGPT-3.5’s answer interpretation was unsatisfactory. These results revealed a discrepancy between the findings and our initial expectations. However, the comparative experiments between ChatGPT-3.5 and ChatGPT-4 indicated significant improvements in model performance, demonstrating the potential of using LLMs as a teaching aid. This article also investigates the use of the trans-prompting strategy to reduce the impact of language bias and enhance question understanding. We present a comparison of the models' performance and the improvement under the trans-lingual problem decomposition prompting mechanism. Finally, we discuss the challenges associated with the appropriate application of AI-driven language models, along with future directions and limitations in the field of AI for education.
{"title":"Investigating the Efficacy of ChatGPT-3.5 for Tutoring in Chinese Elementary Education Settings","authors":"Yu Bai;Jun Li;Jun Shen;Liang Zhao","doi":"10.1109/TLT.2024.3464560","DOIUrl":"https://doi.org/10.1109/TLT.2024.3464560","url":null,"abstract":"The potential of artificial intelligence (AI) in transforming education has received considerable attention. This study aims to explore the potential of large language models (LLMs) in assisting students with studying and passing standardized exams, while many people think it is a hype situation. Using primary education as an example, this research investigates whether ChatGPT-3.5 can achieve satisfactory performance on the Chinese Primary School Exams and whether it can be used as a teaching aid or tutor. We designed an experimental framework and constructed a benchmark that comprises 4800 questions collected from 48 tasks in Chinese elementary education settings. Through automatic and manual evaluations, we observed that ChatGPT-3.5’s pass rate was below the required level of accuracy for most tasks, and the correctness of ChatGPT-3.5’s answer interpretation was unsatisfactory. These results revealed a discrepancy between the findings and our initial expectations. However, the comparative experiments between ChatGPT-3.5 and ChatGPT-4 indicated significant improvements in model performance, demonstrating the potential of using LLMs as a teaching aid. This article also investigates the use of the trans-prompting strategy to reduce the impact of language bias and enhance question understanding. We present a comparison of the models' performance and the improvement under the trans-lingual problem decomposition prompting mechanism. Finally, we discuss the challenges associated with the appropriate application of AI-driven language models, along with future directions and limitations in the field of AI for education.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"2156-2171"},"PeriodicalIF":2.9,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142517999","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Research shows that gamified learning experiences can effectively improve the outstanding issues of students in online learning, such as lack of continuous motivation and easy burnout, thereby improving the effectiveness of online learning. However, how to enhance the gamified learning experience in online learning, and what impact there is between the gamified learning experience and the effectiveness of online learning, remain to be further explored. This research article is based on the theory of gamified learning experience and uses structural equation modeling methodology to explore the relationship among the three dimensions of situation-based cognitive experience, collaboration-based social experience, and motivation-based subjectivity experience and the effectiveness of online learning. The results indicate that there is a significant positive correlation among the three dimensions, and all three dimensions have a significant positive impact on the online learning effectiveness. The subjective experience based on motivation has the greatest impact on the online learning effectiveness, and the other two dimensions have a significant positive impact on the online learning effectiveness. The impact on online learning effectiveness is similar. Finally, the article makes recommendations based on the research conclusions, expecting to provide a research foundation for enhancing the gamified learning experience and improving the effectiveness of online learning.
{"title":"Impact of Gamified Learning Experience on Online Learning Effectiveness","authors":"Xiangping Cui;Chen Du;Jun Shen;Susan Zhang;Juan Xu","doi":"10.1109/TLT.2024.3462892","DOIUrl":"10.1109/TLT.2024.3462892","url":null,"abstract":"Research shows that gamified learning experiences can effectively improve the outstanding issues of students in online learning, such as lack of continuous motivation and easy burnout, thereby improving the effectiveness of online learning. However, how to enhance the gamified learning experience in online learning, and what impact there is between the gamified learning experience and the effectiveness of online learning, remain to be further explored. This research article is based on the theory of gamified learning experience and uses structural equation modeling methodology to explore the relationship among the three dimensions of situation-based cognitive experience, collaboration-based social experience, and motivation-based subjectivity experience and the effectiveness of online learning. The results indicate that there is a significant positive correlation among the three dimensions, and all three dimensions have a significant positive impact on the online learning effectiveness. The subjective experience based on motivation has the greatest impact on the online learning effectiveness, and the other two dimensions have a significant positive impact on the online learning effectiveness. The impact on online learning effectiveness is similar. Finally, the article makes recommendations based on the research conclusions, expecting to provide a research foundation for enhancing the gamified learning experience and improving the effectiveness of online learning.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"2130-2139"},"PeriodicalIF":2.9,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142266332","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-10DOI: 10.1109/TLT.2024.3451050
Seng Chee Tan;Kay Wijekumar;Huaqing Hong;Justin Olmanson;Robert Twomey;Tanmay Sinha
{"title":"Guest Editorial Education in the World of ChatGPT and Generative AI","authors":"Seng Chee Tan;Kay Wijekumar;Huaqing Hong;Justin Olmanson;Robert Twomey;Tanmay Sinha","doi":"10.1109/TLT.2024.3451050","DOIUrl":"https://doi.org/10.1109/TLT.2024.3451050","url":null,"abstract":"","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"2062-2064"},"PeriodicalIF":2.9,"publicationDate":"2024-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10673879","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142174017","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-09DOI: 10.1109/TLT.2024.3456447
Alejandra J. Magana;Syed Tanzim Mubarrat;Dominic Kao;Bedrich Benes
Fostering productive engagement within teams has been found to improve student learning outcomes. Consequently, characterizing productive and unproductive time during teamwork sessions is a critical preliminary step to increase engagement in teamwork meetings. However, research from the cognitive sciences has mainly focused on characterizing levels of productive engagement. Thus, the theoretical contribution of this study focuses on characterizing active and passive forms of engagement, as well as negative and positive forms of engagement. In tandem, researchers have used computer-based methods to supplement quantitative and qualitative analyses to investigate teamwork engagement. Yet, these studies have been limited to information extracted primarily from one data stream. For instance, text data from discussion forums or video data from recordings. We developed an artificial intelligence (AI)-based automatic system that detects productive and unproductive engagement during live teamwork sessions. The technical contribution of this study focuses on the use of three data streams from an interactive session: audio, video, and text. We automatically analyze them and determine each team's level of engagement, such as productive engagement, unproductive engagement, disengagement, and idle. The AI-based system was validated based on hand-coded data. We used the system to characterize productive and unproductive engagement patterns in teams using deep learning methods. Results showed that there were $>$ 91% prediction accuracy and $< $ 7% mismatches between predictions for the three engagement detectors. Moreover, Pearson's $r$ values between the predictions of the three detectors were $>$ 0.844. On a scale of $-$ 1 (unproductive engagement) to 1 (productive engagement), the scores for all teams were 0.94 $pm$ 0.04, suggesting high productive engagement. In addition, teams tended to mostly be in productive engagement before transitioning to disengagement ( $>$ 90.34% of the time) and to idle ( $>$ 93.69% of the time). Before transitioning to productive engagement, we noticed almost equal fractions of teams being in idle and disengagement modes. These results show that the system effectively detects engagement and can be a viable tool for characterizing productive and unproductive engagement patterns in teamwork sessions.
{"title":"AI-Based Automatic Detection of Online Teamwork Engagement in Higher Education","authors":"Alejandra J. Magana;Syed Tanzim Mubarrat;Dominic Kao;Bedrich Benes","doi":"10.1109/TLT.2024.3456447","DOIUrl":"10.1109/TLT.2024.3456447","url":null,"abstract":"Fostering productive engagement within teams has been found to improve student learning outcomes. Consequently, characterizing productive and unproductive time during teamwork sessions is a critical preliminary step to increase engagement in teamwork meetings. However, research from the cognitive sciences has mainly focused on characterizing levels of productive engagement. Thus, the theoretical contribution of this study focuses on characterizing active and passive forms of engagement, as well as negative and positive forms of engagement. In tandem, researchers have used computer-based methods to supplement quantitative and qualitative analyses to investigate teamwork engagement. Yet, these studies have been limited to information extracted primarily from one data stream. For instance, text data from discussion forums or video data from recordings. We developed an artificial intelligence (AI)-based automatic system that detects productive and unproductive engagement during live teamwork sessions. The technical contribution of this study focuses on the use of three data streams from an interactive session: audio, video, and text. We automatically analyze them and determine each team's level of engagement, such as productive engagement, unproductive engagement, disengagement, and idle. The AI-based system was validated based on hand-coded data. We used the system to characterize productive and unproductive engagement patterns in teams using deep learning methods. Results showed that there were \u0000<inline-formula><tex-math>$>$</tex-math></inline-formula>\u000091% prediction accuracy and \u0000<inline-formula><tex-math>$< $</tex-math></inline-formula>\u00007% mismatches between predictions for the three engagement detectors. Moreover, Pearson's \u0000<inline-formula><tex-math>$r$</tex-math></inline-formula>\u0000 values between the predictions of the three detectors were \u0000<inline-formula><tex-math>$>$</tex-math></inline-formula>\u00000.844. On a scale of \u0000<inline-formula><tex-math>$-$</tex-math></inline-formula>\u00001 (unproductive engagement) to 1 (productive engagement), the scores for all teams were 0.94 \u0000<inline-formula><tex-math>$pm$</tex-math></inline-formula>\u0000 0.04, suggesting high productive engagement. In addition, teams tended to mostly be in productive engagement before transitioning to disengagement (\u0000<inline-formula><tex-math>$>$</tex-math></inline-formula>\u000090.34% of the time) and to idle (\u0000<inline-formula><tex-math>$>$</tex-math></inline-formula>\u000093.69% of the time). Before transitioning to productive engagement, we noticed almost equal fractions of teams being in idle and disengagement modes. These results show that the system effectively detects engagement and can be a viable tool for characterizing productive and unproductive engagement patterns in teamwork sessions.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"2091-2106"},"PeriodicalIF":2.9,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10669806","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142225550","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-09DOI: 10.1109/TLT.2024.3456128
Qi Lang;Shengjing Tian;Mo Wang;Jianan Wang
Entrepreneurship education is critical in encouraging students' innovation, creativity, and entrepreneurial spirit. It provides essential skills and knowledge, enabling them to open their creative potential and apply innovative thinking across diverse professional fields. With the widespread application of large language models in education, intelligent-assisted teaching in entrepreneurship education is stepping into a new learning phase anytime and anywhere. Entrepreneurship education extends across interdisciplinary knowledge fields, incorporating subjects like finance and risk management, which require advanced mathematical computational skills. This complexity presents new challenges for artificial-intelligence-assisted question-and-answer models. The study explores how students can maximize the knowledge repository of current large language models to improve learning efficiency and experimentally validates the performance differences between large language models and graph convolutional reasoning models regarding the complex semantic reasoning and mathematical computational demands in entrepreneurship education questions. Based on case studies, it is found that despite the broad prospects of large language models in entrepreneurship education, they still need to improve in practical applications. Especially in tasks within entrepreneurship education that demand precision, such as mathematical computations and risk assessment, the accuracy and efficiency of existing models still need improvement. Therefore, further exploration into algorithm optimization, model fusion, and other technical enhancements can improve the processing capabilities of intelligent question-and-answer systems for specific domain issues, aiming to meet the practical needs of entrepreneurship education.
{"title":"Exploring the Answering Capability of Large Language Models in Addressing Complex Knowledge in Entrepreneurship Education","authors":"Qi Lang;Shengjing Tian;Mo Wang;Jianan Wang","doi":"10.1109/TLT.2024.3456128","DOIUrl":"10.1109/TLT.2024.3456128","url":null,"abstract":"Entrepreneurship education is critical in encouraging students' innovation, creativity, and entrepreneurial spirit. It provides essential skills and knowledge, enabling them to open their creative potential and apply innovative thinking across diverse professional fields. With the widespread application of large language models in education, intelligent-assisted teaching in entrepreneurship education is stepping into a new learning phase anytime and anywhere. Entrepreneurship education extends across interdisciplinary knowledge fields, incorporating subjects like finance and risk management, which require advanced mathematical computational skills. This complexity presents new challenges for artificial-intelligence-assisted question-and-answer models. The study explores how students can maximize the knowledge repository of current large language models to improve learning efficiency and experimentally validates the performance differences between large language models and graph convolutional reasoning models regarding the complex semantic reasoning and mathematical computational demands in entrepreneurship education questions. Based on case studies, it is found that despite the broad prospects of large language models in entrepreneurship education, they still need to improve in practical applications. Especially in tasks within entrepreneurship education that demand precision, such as mathematical computations and risk assessment, the accuracy and efficiency of existing models still need improvement. Therefore, further exploration into algorithm optimization, model fusion, and other technical enhancements can improve the processing capabilities of intelligent question-and-answer systems for specific domain issues, aiming to meet the practical needs of entrepreneurship education.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"2107-2116"},"PeriodicalIF":2.9,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142199105","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-29DOI: 10.1109/tlt.2024.3428366
Xinyue Zhang, Fangqing Zhu, Kun Wang, Guitao Cao, Yaofeng Xue, Mingzhuo Liu
{"title":"Bring the Intelligent Tutoring Robots to Education: A Systematic Literature Review","authors":"Xinyue Zhang, Fangqing Zhu, Kun Wang, Guitao Cao, Yaofeng Xue, Mingzhuo Liu","doi":"10.1109/tlt.2024.3428366","DOIUrl":"https://doi.org/10.1109/tlt.2024.3428366","url":null,"abstract":"","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"44 1","pages":""},"PeriodicalIF":3.7,"publicationDate":"2024-08-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142225551","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-29DOI: 10.1109/TLT.2024.3451462
Chengyan Yu;Shihuan Wang;Dong Zhang;Yingying Zhang;Chaoqun Cen;Zhixiang You;Xiaobing Zou;Hongzhu Deng;Ming Li
Numerous children diagnosed with autism spectrum disorder (ASD) exhibit abnormal eye gaze pattern in communication and social interaction. In this study, we aim to investigate the effectiveness of the hide-and-seek virtual reality system (HSVRS) in improving gaze fixation abilities in children with ASD. Our hypothesis is that engaging in a hide-and-seek game within a virtual environment, particularly with a customized avatar resembling a familiar figure, would significantly enhance gaze fixation skills compared to traditional interventions without supplementary virtual reality (VR) intervention. In total, 36 children with ASD were involved in this pilot study in three groups: the avatar customized group, the avatar uncustomized group, and the control group. The control group only received human intervention, while the avatar group received additional VR-assisted interventions. The effect of HSVRS was measured by a six-point Likert subjective questionnaire and demonstrated significant improvements in gaze fixation abilities in the VR-assisted intervention groups compared to the control group ( $P$ = 0.006, 0.001). Moreover, the avatar customized group, which interacted with a familiar-looking avatar, obtained noticeable increments in gaze fixation metrics ( $P$ = 0.036, 0.005, 0.001). Experimental results show the effectiveness of utilizing VR technology to complement regular interventions in terms of improving gaze fixation abilities for young children with ASD.
{"title":"HSVRS: A Virtual Reality System of the Hide-and-Seek Game to Enhance Gaze Fixation Ability for Autistic Children","authors":"Chengyan Yu;Shihuan Wang;Dong Zhang;Yingying Zhang;Chaoqun Cen;Zhixiang You;Xiaobing Zou;Hongzhu Deng;Ming Li","doi":"10.1109/TLT.2024.3451462","DOIUrl":"https://doi.org/10.1109/TLT.2024.3451462","url":null,"abstract":"Numerous children diagnosed with autism spectrum disorder (ASD) exhibit abnormal eye gaze pattern in communication and social interaction. In this study, we aim to investigate the effectiveness of the hide-and-seek virtual reality system (HSVRS) in improving gaze fixation abilities in children with ASD. Our hypothesis is that engaging in a hide-and-seek game within a virtual environment, particularly with a customized avatar resembling a familiar figure, would significantly enhance gaze fixation skills compared to traditional interventions without supplementary virtual reality (VR) intervention. In total, 36 children with ASD were involved in this pilot study in three groups: the avatar customized group, the avatar uncustomized group, and the control group. The control group only received human intervention, while the avatar group received additional VR-assisted interventions. The effect of HSVRS was measured by a six-point Likert subjective questionnaire and demonstrated significant improvements in gaze fixation abilities in the VR-assisted intervention groups compared to the control group (\u0000<inline-formula><tex-math>$P$</tex-math></inline-formula>\u0000 = 0.006, 0.001). Moreover, the avatar customized group, which interacted with a familiar-looking avatar, obtained noticeable increments in gaze fixation metrics (\u0000<inline-formula><tex-math>$P$</tex-math></inline-formula>\u0000 = 0.036, 0.005, 0.001). Experimental results show the effectiveness of utilizing VR technology to complement regular interventions in terms of improving gaze fixation abilities for young children with ASD.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"2065-2078"},"PeriodicalIF":2.9,"publicationDate":"2024-08-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142173973","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This article presents a controlled case study focused on implementing and using generative artificial intelligence, specifically large language models (LLMs), in physiotherapy education to assist instructors with formulating effective technology-mediated feedback for students. It outlines how these advanced technologies have been integrated into an existing feedback-oriented platform to guide instructors in providing feedback inputs and establish a reference framework for future innovations in practical skills training for health professions education. Specifically, the proposed solution uses LLMs to automatically evaluate feedback inputs made by instructors based on predefined and literature-based quality criteria and generates actionable textual explanations for reformulation. In addition, if the instructor requires, the tool supports summary generation for large sets of text inputs to achieve better student reception and understanding. The case study describes how these features were integrated into the feedback-oriented platform, how their effectiveness was evaluated in a controlled setting with documented feedback inputs, and the results of its implementation with real users through cognitive walkthroughs. Initial results indicate that this innovative implementation holds great potential to enhance learning and performance in physiotherapy education and has the potential to expand to other health disciplines where the development of procedural skills is critical, offering a valuable tool to assess and improve feedback based on quality standards for effective feedback processes. The cognitive walkthroughs allowed us to determine participants' usability decisions in the face of these new features and to evaluate the perceived usefulness, how this would integrate into their workload, and their opinion regarding the potential for the future within this teaching strategy. This article concludes with a discussion of the implications of these findings for practice and future research directions in this developing field.
{"title":"Implementing Artificial Intelligence in Physiotherapy Education: A Case Study on the Use of Large Language Models (LLM) to Enhance Feedback","authors":"Ignacio Villagrán;Rocío Hernández;Gregory Schuit;Andrés Neyem;Javiera Fuentes-Cimma;Constanza Miranda;Isabel Hilliger;Valentina Durán;Gabriel Escalona;Julián Varas","doi":"10.1109/TLT.2024.3450210","DOIUrl":"10.1109/TLT.2024.3450210","url":null,"abstract":"This article presents a controlled case study focused on implementing and using generative artificial intelligence, specifically large language models (LLMs), in physiotherapy education to assist instructors with formulating effective technology-mediated feedback for students. It outlines how these advanced technologies have been integrated into an existing feedback-oriented platform to guide instructors in providing feedback inputs and establish a reference framework for future innovations in practical skills training for health professions education. Specifically, the proposed solution uses LLMs to automatically evaluate feedback inputs made by instructors based on predefined and literature-based quality criteria and generates actionable textual explanations for reformulation. In addition, if the instructor requires, the tool supports summary generation for large sets of text inputs to achieve better student reception and understanding. The case study describes how these features were integrated into the feedback-oriented platform, how their effectiveness was evaluated in a controlled setting with documented feedback inputs, and the results of its implementation with real users through cognitive walkthroughs. Initial results indicate that this innovative implementation holds great potential to enhance learning and performance in physiotherapy education and has the potential to expand to other health disciplines where the development of procedural skills is critical, offering a valuable tool to assess and improve feedback based on quality standards for effective feedback processes. The cognitive walkthroughs allowed us to determine participants' usability decisions in the face of these new features and to evaluate the perceived usefulness, how this would integrate into their workload, and their opinion regarding the potential for the future within this teaching strategy. This article concludes with a discussion of the implications of these findings for practice and future research directions in this developing field.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"2079-2090"},"PeriodicalIF":2.9,"publicationDate":"2024-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142225554","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-13DOI: 10.1109/TLT.2024.3443079
Sonja Kleter;Uwe Matzat;Rianne Conijn
Much of learning analytics research has focused on factors influencing model generalizability of predictive models for academic performance. The degree of model generalizability across courses may depend on aspects, such as the similarity of the course setup, course material, the student cohort, or the teacher. Which of these contextual factors affect generalizability and to what extent is yet unclear. The current study explicitly compares model generalizability within course versus within cohort of predictive models. This study considered 66 behavioral indicators, which are commonly used in the literature. Indicators regarding frequency and duration of online study time, accessing study material, time management, assignments and quizzes, and weekly measures, were extracted from the university's learning management system. Numerical and binary predictive models were generated via recursive feature selection. Model generalizability was evaluated in terms of both model stability and model performance. The results showed that model stability was better for numerical models generalized within course compared to models generalized within cohort or across course and across cohort. Nevertheless, model stability was low for the binary models and only moderate for numerical models under all the conditions. Concerning model performance, the increase in estimation error after model generalizability depends on the initial model performance for models generalized within course and within cohort. Contrary to previous research, with respect to performance, we found no difference between model generalizability within cohort and within course. We suspect that performance reduction after any form of model generalizability depends on initial performance.
{"title":"Boundary Conditions of Generalizing Predictive Models for Academic Performance: Within Cohort Versus Within Course","authors":"Sonja Kleter;Uwe Matzat;Rianne Conijn","doi":"10.1109/TLT.2024.3443079","DOIUrl":"10.1109/TLT.2024.3443079","url":null,"abstract":"Much of learning analytics research has focused on factors influencing model generalizability of predictive models for academic performance. The degree of model generalizability across courses may depend on aspects, such as the similarity of the course setup, course material, the student cohort, or the teacher. Which of these contextual factors affect generalizability and to what extent is yet unclear. The current study explicitly compares model generalizability within course versus within cohort of predictive models. This study considered 66 behavioral indicators, which are commonly used in the literature. Indicators regarding frequency and duration of online study time, accessing study material, time management, assignments and quizzes, and weekly measures, were extracted from the university's learning management system. Numerical and binary predictive models were generated via recursive feature selection. Model generalizability was evaluated in terms of both model stability and model performance. The results showed that model stability was better for numerical models generalized within course compared to models generalized within cohort or across course and across cohort. Nevertheless, model stability was low for the binary models and only moderate for numerical models under all the conditions. Concerning model performance, the increase in estimation error after model generalizability depends on the initial model performance for models generalized within course and within cohort. Contrary to previous research, with respect to performance, we found no difference between model generalizability within cohort and within course. We suspect that performance reduction after any form of model generalizability depends on initial performance.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"2183-2194"},"PeriodicalIF":2.9,"publicationDate":"2024-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142225552","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ray tracing is a computer graphics technique used to produce realistic visuals by physically simulating the behavior of light. Although this technique can be described straightforwardly, fully comprehending it might be challenging. It is typically taught in the classroom using the 2-D formats, such as paper or a blackboard. We propose using immersive environments for incrementing the understanding of ray tracing. We focus on improving the knowledge of the technique in experienced users, particularly Master of Computer Science students minoring in a computer-graphics-related area. We argue that exploring the ray tracing process in an immersive visualization environment can further improve the understanding of ray tracing acquired using conventional means. With that objective, this study starts by presenting RayTracerVR , a virtual reality prototype tool for learning the mechanisms of ray tracing. This tool can be used to visually explore and interact with the different aspects of the technique. It allows users to observe the progression of the rays throughout the sequential stages of the ray tracing process and analyze its corresponding computer pseudocode. The study includes user evaluation where RayTracerVR is employed to assess improvements in ray tracing understanding. The prototype's usability is also assessed. The findings indicate that using the ray tracing immersive learning environment results in a supplemental increase in understanding in users who have previously learned ray tracing using conventional means.
{"title":"Improving Ray Tracing Understanding With Immersive Environments","authors":"Nuno Verdelho Trindade;Lídia Custódio;Alfredo Ferreira;João Madeiras Pereira","doi":"10.1109/TLT.2024.3436656","DOIUrl":"https://doi.org/10.1109/TLT.2024.3436656","url":null,"abstract":"Ray tracing is a computer graphics technique used to produce realistic visuals by physically simulating the behavior of light. Although this technique can be described straightforwardly, fully comprehending it might be challenging. It is typically taught in the classroom using the 2-D formats, such as paper or a blackboard. We propose using immersive environments for incrementing the understanding of ray tracing. We focus on improving the knowledge of the technique in experienced users, particularly Master of Computer Science students minoring in a computer-graphics-related area. We argue that exploring the ray tracing process in an immersive visualization environment can further improve the understanding of ray tracing acquired using conventional means. With that objective, this study starts by presenting \u0000<italic>RayTracerVR</i>\u0000, a virtual reality prototype tool for learning the mechanisms of ray tracing. This tool can be used to visually explore and interact with the different aspects of the technique. It allows users to observe the progression of the rays throughout the sequential stages of the ray tracing process and analyze its corresponding computer pseudocode. The study includes user evaluation where \u0000<italic>RayTracerVR</i>\u0000 is employed to assess improvements in ray tracing understanding. The prototype's usability is also assessed. The findings indicate that using the ray tracing immersive learning environment results in a supplemental increase in understanding in users who have previously learned ray tracing using conventional means.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1975-1988"},"PeriodicalIF":2.9,"publicationDate":"2024-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10634035","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141998662","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：