Pub Date : 2025-12-01DOI: 10.1016/j.caeai.2025.100520
King Woon Yau , Tianle Dong , Ching Sing Chai , Thomas K.F. Chiu , Helen Meng , Irwin King , Savio W.H. Wong , Yeung Yam
Teachers play a vital role in driving successful artificial intelligence (AI) education. Research on teachers' competency to teach AI (TCAI) is still limited. This study investigated the progression of in-service teachers' AI competency with the Technological Pedagogical Content Knowledge (TPACK) framework using Epistemic Network Analysis (ENA). Seven secondary school teachers who engaged in an AI education project were interviewed over a three-year period of curriculum development and implementation. The differences in ENA patterns in various stages indicated an evolution of teachers’ TPACK over the years. The ENA results also revealed different patterns between experienced and less experienced teachers. Experienced teachers tend to integrate their TPACK components with pedagogical considerations, whereas less experienced teachers focus more on content-related elements. The differences in ENA patterns indicate distinct progression paths with different focuses, highlighting the need to tailor professional development activities for different groups of teachers at various stages. These findings underscore the importance of continuous support and targeted training to enhance teachers' AI competency in AI education.
{"title":"Epistemic network analysis of in-service teachers’ competency to teach artificial intelligence for secondary education","authors":"King Woon Yau , Tianle Dong , Ching Sing Chai , Thomas K.F. Chiu , Helen Meng , Irwin King , Savio W.H. Wong , Yeung Yam","doi":"10.1016/j.caeai.2025.100520","DOIUrl":"10.1016/j.caeai.2025.100520","url":null,"abstract":"<div><div>Teachers play a vital role in driving successful artificial intelligence (AI) education. Research on teachers' competency to teach AI (TCAI) is still limited. This study investigated the progression of in-service teachers' AI competency with the Technological Pedagogical Content Knowledge (TPACK) framework using Epistemic Network Analysis (ENA). Seven secondary school teachers who engaged in an AI education project were interviewed over a three-year period of curriculum development and implementation. The differences in ENA patterns in various stages indicated an evolution of teachers’ TPACK over the years. The ENA results also revealed different patterns between experienced and less experienced teachers. Experienced teachers tend to integrate their TPACK components with pedagogical considerations, whereas less experienced teachers focus more on content-related elements. The differences in ENA patterns indicate distinct progression paths with different focuses, highlighting the need to tailor professional development activities for different groups of teachers at various stages. These findings underscore the importance of continuous support and targeted training to enhance teachers' AI competency in AI education.</div></div>","PeriodicalId":34469,"journal":{"name":"Computers and Education Artificial Intelligence","volume":"9 ","pages":"Article 100520"},"PeriodicalIF":0.0,"publicationDate":"2025-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145681342","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-12-01DOI: 10.1016/j.caeai.2025.100474
Matthew Nyaaba , Xiaoming Zhai
As educators begin using Generative AI (GenAI) for lesson planning, they often encounter generated content that fails to consider the classroom's cultural context. In this study, we address this issue by adopting a design science research approach to develop a theory-based prompt grounded in culturally responsive pedagogy (CRP) and using it to customize a Culturally Responsive Lesson Planner (CRLP) GPT. Guided by the CRP framework, the CRLP uses an InteractiveSemi-Automated (ISA) prompt architecture that engages teachers in dialogue to collect cultural and contextual details before generating a lesson plan. To evaluate the CRLP's effectiveness, we asked two expert reviewers to compare Grade 7 “States of Matter” lesson plans for Ghana's Ashanti Region, generated with both the CRLP and the base GPT-4o using a standard prompt. The expert reviewers rated the CRLP-generated lesson plan higher in cultural elements identified (36 vs. 21 elements), accuracy (1.8 vs. 1.2), and curriculum relevance (2.0 vs. 1.3) than that created by the standard prompt within the base GPT-4o. The CRLP-generated lesson plan also included more Asante Twi examples such as “Solid” (ɛpono [wooden furniture], dadeɛ [metal objects], aboɔ [stones], and ntadeɛ [clothing]), recommended local teaching resources, and allowed teachers to make final revisions before generating the complete lesson plan. Additionally, the CRLP included the developer's contact details to encourage ongoing feedback and improvement. However, cultural hallucinations were slightly higher (0.75 vs. 0.5) in the CRLP-generated lesson plan compared with the standard GPT-4o prompt. These findings suggest that a GenAI tool grounded in educational theory is more effective in supporting the goals of education than the standard version. Furthermore, the CRLP and its ISA prompt strategy represent Human-in-the-loop system that has the potential to enhance teachers' AI literacy and culturally responsive pedagogy as they engage with the tool. We recommend future studies comparing CRLP and human-generated lesson plans, as well as empirical research that tests CRLP lesson plans in classroom settings.
{"title":"Developing a theory-grounded AI tool for the generation of culturally responsive lesson plans","authors":"Matthew Nyaaba , Xiaoming Zhai","doi":"10.1016/j.caeai.2025.100474","DOIUrl":"10.1016/j.caeai.2025.100474","url":null,"abstract":"<div><div>As educators begin using Generative AI (GenAI) for lesson planning, they often encounter generated content that fails to consider the classroom's cultural context. In this study, we address this issue by adopting a design science research approach to develop a theory-based prompt grounded in culturally responsive pedagogy (CRP) and using it to customize a <em>Culturally Responsive Lesson Planner (CRLP) GPT.</em> Guided by the CRP framework, the CRLP uses an <em>Interactive</em> <em>S</em><em>emi-</em><em>Automated</em> (ISA) prompt architecture that engages teachers in dialogue to collect cultural and contextual details before generating a lesson plan. To evaluate the CRLP's effectiveness, we asked two expert reviewers to compare Grade 7 “States of Matter” lesson plans for Ghana's Ashanti Region, generated with both the CRLP and the base GPT-4o using a standard prompt. The expert reviewers rated the CRLP-generated lesson plan higher in cultural elements identified (36 vs. 21 elements), accuracy (1.8 vs. 1.2), and curriculum relevance (2.0 vs. 1.3) than that created by the standard prompt within the base GPT-4o. The CRLP-generated lesson plan also included more Asante Twi examples such as “Solid” (<em>ɛpono</em> [wooden furniture], <em>dadeɛ</em> [metal objects], <em>aboɔ</em> [stones], and <em>ntadeɛ</em> [clothing]), recommended local teaching resources, and allowed teachers to make final revisions before generating the complete lesson plan. Additionally, the CRLP included the developer's contact details to encourage ongoing feedback and improvement. However, cultural hallucinations were slightly higher (0.75 vs. 0.5) in the CRLP-generated lesson plan compared with the standard GPT-4o prompt. These findings suggest that a GenAI tool grounded in educational theory is more effective in supporting the goals of education than the standard version. Furthermore, the CRLP and its ISA prompt strategy represent Human-in-the-loop system that has the potential to enhance teachers' AI literacy and culturally responsive pedagogy as they engage with the tool. We recommend future studies comparing CRLP and human-generated lesson plans, as well as empirical research that tests CRLP lesson plans in classroom settings.</div></div>","PeriodicalId":34469,"journal":{"name":"Computers and Education Artificial Intelligence","volume":"9 ","pages":"Article 100474"},"PeriodicalIF":0.0,"publicationDate":"2025-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145681348","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-12-01DOI: 10.1016/j.caeai.2025.100524
Danial Hooshyar , Gustav Šír , Yeongwook Yang , Eve Kikas , Raija Hämäläinen , Tommi Kärkkäinen , Dragan Gašević , Roger Azevedo
Despite significant advancements in AI-driven educational systems and ongoing calls for responsible AI for education, several critical issues remain unresolved—acting as elephant in the room within AI in education, learning analytics, educational data mining, learning sciences, and educational psychology communities. This critical analysis identifies and examines nine persistent challenges across the conceptual, methodological, and ethical dimensions that continue to undermine the fairness, transparency, and effectiveness of current AI methods and applications in education. These include: 1) the lack of clarity around what AI for education truly means—often ignoring the distinct purposes, strengths, and limitations of different AI families—and the trend of equating it with domain-agnostic, company-driven large language models; 2) the widespread neglect of essential learning processes such as motivation, emotion, and (meta)cognition in AI-driven learner modelling and their contextual nature; 3) limited integration of domain knowledge and lack of stakeholder involvement in AI design and development; 4) continued use of non-sequential machine learning models on temporal educational data; 5) misuse of non-sequential metrics to evaluate sequential models; 6) using unreliable explainable AI methods to provide explanations for black-box models; 7) ignoring ethical guidelines in addressing data inconsistencies during model training; 8) use of mainstream AI methods for pattern discovery and learning analytics without systematic benchmarking; and 9) overemphasis on global prescriptions while overlooking localized, student-specific recommendations. Supported by theoretical and empirical research, we demonstrate how hybrid AI methods—specifically neural-symbolic AI—can address the elephant in the room and serve as the foundation for responsible, trustworthy AI systems in education.
{"title":"Towards responsible AI for education: Hybrid human-AI to confront the elephant in the room","authors":"Danial Hooshyar , Gustav Šír , Yeongwook Yang , Eve Kikas , Raija Hämäläinen , Tommi Kärkkäinen , Dragan Gašević , Roger Azevedo","doi":"10.1016/j.caeai.2025.100524","DOIUrl":"10.1016/j.caeai.2025.100524","url":null,"abstract":"<div><div>Despite significant advancements in AI-driven educational systems and ongoing calls for responsible AI for education, several critical issues remain unresolved—acting as <em>elephant in the room</em> within AI in education, learning analytics, educational data mining, learning sciences, and educational psychology communities. This critical analysis identifies and examines nine persistent challenges across the conceptual, methodological, and ethical dimensions that continue to undermine the fairness, transparency, and effectiveness of current AI methods and applications in education. These include: 1) the lack of clarity around what AI for education truly means—often ignoring the distinct purposes, strengths, and limitations of different AI families—and the trend of equating it with domain-agnostic, company-driven large language models; 2) the widespread neglect of essential learning processes such as motivation, emotion, and (meta)cognition in AI-driven learner modelling and their contextual nature; 3) limited integration of domain knowledge and lack of stakeholder involvement in AI design and development; 4) continued use of non-sequential machine learning models on temporal educational data; 5) misuse of non-sequential metrics to evaluate sequential models; 6) using unreliable explainable AI methods to provide explanations for black-box models; 7) ignoring ethical guidelines in addressing data inconsistencies during model training; 8) use of mainstream AI methods for pattern discovery and learning analytics without systematic benchmarking; and 9) overemphasis on global prescriptions while overlooking localized, student-specific recommendations. Supported by theoretical and empirical research, we demonstrate how hybrid AI methods—specifically neural-symbolic AI—can address the <em>elephant in the room</em> and serve as the foundation for responsible, trustworthy AI systems in education.</div></div>","PeriodicalId":34469,"journal":{"name":"Computers and Education Artificial Intelligence","volume":"9 ","pages":"Article 100524"},"PeriodicalIF":0.0,"publicationDate":"2025-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145681351","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-12-01DOI: 10.1016/j.caeai.2025.100512
Ayşe Candan Şimşek , Gerrit Anders , Jonathan Göth , Luisa Specht , Markus Huff
Online videos have become a central tool in modern education. Alongside this shift, Artificial Intelligence (AI) is reshaping personalized learning experiences, with generative large language models like ChatGPT offering new ways to tailor information to individual learners. Based on the Cognitive Theory of Multimedia Learning (CTML), which proposes two principles that relate to the interaction with the learning material (segmenting and generative activity), we conducted two experiments in which participants were asked to pause an educational video at times of comprehension difficulty. In Experiment 1 (N = 101), we examined whether GPT-generated summaries -introduced at self-paced pause points-result in better learning compared to video transcripts. In Experiment 2 (N = 215), we compared the role of GPT-generated summaries and GPT-generated reflective prompts. Those elicited open-ended answers from the participants. We measured retention and transfer learning, as well as mental effort, and perceived task difficulty. Contrary to our expectations, we observed no differences between AI summaries and transcripts in terms of retention and transfer outcomes. Participants showed a learning effect indicating more correct answers after watching the video, but this effect did not differ between conditions. We can especially note that the motivation to engage in the material, as well as the difficulty and length of the video, may have affected the results. As research investigating the role of AI in educational settings is still new, future research can delve into finding the optimal conditions under which AI can benefit learning outcomes.
{"title":"Is ChatGPT a good study companion? The role of AI-generated summaries and reflective prompts in learning from educational videos","authors":"Ayşe Candan Şimşek , Gerrit Anders , Jonathan Göth , Luisa Specht , Markus Huff","doi":"10.1016/j.caeai.2025.100512","DOIUrl":"10.1016/j.caeai.2025.100512","url":null,"abstract":"<div><div>Online videos have become a central tool in modern education. Alongside this shift, Artificial Intelligence (AI) is reshaping personalized learning experiences, with generative large language models like ChatGPT offering new ways to tailor information to individual learners. Based on the Cognitive Theory of Multimedia Learning (CTML), which proposes two principles that relate to the interaction with the learning material (segmenting and generative activity), we conducted two experiments in which participants were asked to pause an educational video at times of comprehension difficulty. In Experiment 1 (<em>N</em> = 101), we examined whether GPT-generated summaries -introduced at self-paced pause points-result in better learning compared to video transcripts. In Experiment 2 (<em>N</em> = 215), we compared the role of GPT-generated summaries and GPT-generated reflective prompts. Those elicited open-ended answers from the participants. We measured retention and transfer learning, as well as mental effort, and perceived task difficulty. Contrary to our expectations, we observed no differences between AI summaries and transcripts in terms of retention and transfer outcomes. Participants showed a learning effect indicating more correct answers after watching the video, but this effect did not differ between conditions. We can especially note that the motivation to engage in the material, as well as the difficulty and length of the video, may have affected the results. As research investigating the role of AI in educational settings is still new, future research can delve into finding the optimal conditions under which AI can benefit learning outcomes.</div></div>","PeriodicalId":34469,"journal":{"name":"Computers and Education Artificial Intelligence","volume":"9 ","pages":"Article 100512"},"PeriodicalIF":0.0,"publicationDate":"2025-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145614497","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-12-01DOI: 10.1016/j.caeai.2025.100515
Haitang Wan
With the rapid development of educational informatization, the IntelliAssessment is increasingly widely used in the formative evaluation of course teaching. Scalability testing (Python Locust framework) showed 100 % response rate under 500 concurrent requests (consistent with typical university course sizes), while 92 % response rate was observed at 1000 concurrent requests (an extreme stress test scenario). Security validation included a 94 % attack-blocking rate in penetration testing and 91.0 % F1-score for AI-driven phishing detection. The <2-s real-time feedback window (p50 = 1.2 s, p90 = 1.8 s, p99 = 2.3 s) is maintained for 90 % of interactions under typical loads, with latency degrading only at very high concurrency—pedagogically, this ensures timely formative feedback for most classroom scenarios. A supplementary analysis discussing current security limitations and the evolving nature of security threats has been added, along with potential development ideas for enhancing system security. These improvements aim to strengthen the comprehensiveness and scientific reliability of our manuscript. Statistics show that 27.65 % of the students who participated in the evaluation were very satisfied with the feedback, while 17.4 % thought that the feedback was helpful. As for understanding the assessment content, 3.15 % of the students indicated that they needed more clarity, indicating that the clarity of the assessment questions still required improvement. Among the students’ learning performance, 67.24 % scored higher than the passing line, indicating that most students can master the course content.
{"title":"Role of online assessment system in formative evaluation of programming education","authors":"Haitang Wan","doi":"10.1016/j.caeai.2025.100515","DOIUrl":"10.1016/j.caeai.2025.100515","url":null,"abstract":"<div><div>With the rapid development of educational informatization, the IntelliAssessment is increasingly widely used in the formative evaluation of course teaching. Scalability testing (Python Locust framework) showed 100 % response rate under 500 concurrent requests (consistent with typical university course sizes), while 92 % response rate was observed at 1000 concurrent requests (an extreme stress test scenario). Security validation included a 94 % attack-blocking rate in penetration testing and 91.0 % F1-score for AI-driven phishing detection. The <2-s real-time feedback window (p50 = 1.2 s, p90 = 1.8 s, p99 = 2.3 s) is maintained for 90 % of interactions under typical loads, with latency degrading only at very high concurrency—pedagogically, this ensures timely formative feedback for most classroom scenarios. A supplementary analysis discussing current security limitations and the evolving nature of security threats has been added, along with potential development ideas for enhancing system security. These improvements aim to strengthen the comprehensiveness and scientific reliability of our manuscript. Statistics show that 27.65 % of the students who participated in the evaluation were very satisfied with the feedback, while 17.4 % thought that the feedback was helpful. As for understanding the assessment content, 3.15 % of the students indicated that they needed more clarity, indicating that the clarity of the assessment questions still required improvement. Among the students’ learning performance, 67.24 % scored higher than the passing line, indicating that most students can master the course content.</div></div>","PeriodicalId":34469,"journal":{"name":"Computers and Education Artificial Intelligence","volume":"9 ","pages":"Article 100515"},"PeriodicalIF":0.0,"publicationDate":"2025-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145681344","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-12-01DOI: 10.1016/j.caeai.2025.100503
Siu Cheung Kong , Chunyu Hou
In the artificial intelligence (AI) era, secondary and university students should be able to apply AI for problem-solving. This study designed and evaluated an AI literacy programme to enhance understanding of machine learning concepts. It also examined how the conceptual understanding from two foundational courses (Courses 1 and 2) affected students' application of these concepts in the subsequent two project-based learning courses (Courses 3 and 4). The regression analysis of data from 566, 566, 470, and 196 student participants enrolled on Courses 1, 2, 3, and 4, respectively, revealed that the post-course concept tests for Courses 1 and 2 accounted for 19.9 % of the variance in the students' problem-solving ability test before they took Course 3. This result indicates that teaching students' foundational concepts can develop their ability to solve machine learning-related problems. The post-course concept tests for Courses 1 and 2, together with the pre-course problem-solving ability test for Course 3, collectively explained 27.4 % of the variance in the students’ problem-solving ability after completing Course 3. Together with the significant improvement in the paired-samples t-test statistics for the pre- and post-course problem-solving test of Course 3, this indicates the importance of providing opportunities for students to solve machine learning-related problems. These findings provide empirical evidence to inform the design of curricula for AI literacy programmes. Project-based learning (PBL) is an approach that can provide opportunities for participants to develop problem-solving skills using foundational AI knowledge.
{"title":"Predictive capability of foundational concepts tests for problem-solving using machine learning concepts: Evaluating project-based learning courses in artificial intelligence literacy education","authors":"Siu Cheung Kong , Chunyu Hou","doi":"10.1016/j.caeai.2025.100503","DOIUrl":"10.1016/j.caeai.2025.100503","url":null,"abstract":"<div><div>In the artificial intelligence (AI) era, secondary and university students should be able to apply AI for problem-solving. This study designed and evaluated an AI literacy programme to enhance understanding of machine learning concepts. It also examined how the conceptual understanding from two foundational courses (Courses 1 and 2) affected students' application of these concepts in the subsequent two project-based learning courses (Courses 3 and 4). The regression analysis of data from 566, 566, 470, and 196 student participants enrolled on Courses 1, 2, 3, and 4, respectively, revealed that the post-course concept tests for Courses 1 and 2 accounted for 19.9 % of the variance in the students' problem-solving ability test before they took Course 3. This result indicates that teaching students' foundational concepts can develop their ability to solve machine learning-related problems. The post-course concept tests for Courses 1 and 2, together with the pre-course problem-solving ability test for Course 3, collectively explained 27.4 % of the variance in the students’ problem-solving ability after completing Course 3. Together with the significant improvement in the paired-samples <em>t</em>-test statistics for the pre- and post-course problem-solving test of Course 3, this indicates the importance of providing opportunities for students to solve machine learning-related problems. These findings provide empirical evidence to inform the design of curricula for AI literacy programmes. Project-based learning (PBL) is an approach that can provide opportunities for participants to develop problem-solving skills using foundational AI knowledge.</div></div>","PeriodicalId":34469,"journal":{"name":"Computers and Education Artificial Intelligence","volume":"9 ","pages":"Article 100503"},"PeriodicalIF":0.0,"publicationDate":"2025-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145614501","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Towards responsible AI in education: Challenges and implications for research and practice","authors":"Teresa Cerratto Pargman, Cormac McGrath, Marcelo Milrad","doi":"10.1016/j.caeai.2024.100345","DOIUrl":"10.1016/j.caeai.2024.100345","url":null,"abstract":"","PeriodicalId":34469,"journal":{"name":"Computers and Education Artificial Intelligence","volume":"9 ","pages":"Article 100345"},"PeriodicalIF":0.0,"publicationDate":"2025-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145789991","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-12-01DOI: 10.1016/j.caeai.2025.100521
Xiao Tan, Gary Cheng, Man Ho Ling
With the rapid penetration of generative artificial intelligence (AI) in higher education, university teachers' AI competency has become a critical determinant of effective technology integration in teaching. However, systematic and empirically validated intervention frameworks to support the development of this competency remain scarce. To address this gap, this study implemented a six-month professional development (PD) programme grounded in the Intelligent-TPACK framework and evaluated its effectiveness using a quasi-experimental pre-test-post-test design. A total of 64 teachers participated in the PD programme (experimental group), while pre- and post-test data were also collected from 61 teachers who did not participate (control group). Results indicate that the PD programme significantly enhanced AI competency in the experimental group, particularly in the domains of AI Technological Knowledge (AITK) and AI Technological Pedagogical Knowledge (AITPK). After controlling for baseline differences using ANCOVA, the effect size remained above the moderate threshold. A mixed-designed ANOVA further confirmed a significant interaction effect between group and time, ruling out maturation effects. Multi-level regression analysis revealed that background variables such as teaching experience, discipline, and professional title had limited predictive power for AI competency gains. Notably, self-perceived participation level did not significantly predict outcomes, whereas attendance rate emerged as a significant positive predictor. Interestingly, negative gain scores were observed in both groups. Follow-up interviews indicated that these scores did not reflect an actual decline in AI competency but rather a metacognitive recalibration, in which teachers shifted from unconscious incompetence to conscious incompetence—a pattern consistent with the Dunning–Kruger effect. This finding offers a novel theoretical perspective on the mechanism of change underlying the intervention. Overall, the PD programme based on the Intelligent-TPACK framework effectively enhanced university teachers’ AI competency and provides a systematic and evidence-based model for future PD initiatives in the AI era.
{"title":"Enhancing teachers’ AI competency: A professional development intervention study based on intelligent-TPACK framework","authors":"Xiao Tan, Gary Cheng, Man Ho Ling","doi":"10.1016/j.caeai.2025.100521","DOIUrl":"10.1016/j.caeai.2025.100521","url":null,"abstract":"<div><div>With the rapid penetration of generative artificial intelligence (AI) in higher education, university teachers' AI competency has become a critical determinant of effective technology integration in teaching. However, systematic and empirically validated intervention frameworks to support the development of this competency remain scarce. To address this gap, this study implemented a six-month professional development (PD) programme grounded in the Intelligent-TPACK framework and evaluated its effectiveness using a quasi-experimental pre-test-post-test design. A total of 64 teachers participated in the PD programme (experimental group), while pre- and post-test data were also collected from 61 teachers who did not participate (control group). Results indicate that the PD programme significantly enhanced AI competency in the experimental group, particularly in the domains of AI Technological Knowledge (AITK) and AI Technological Pedagogical Knowledge (AITPK). After controlling for baseline differences using ANCOVA, the effect size remained above the moderate threshold. A mixed-designed ANOVA further confirmed a significant interaction effect between group and time, ruling out maturation effects. Multi-level regression analysis revealed that background variables such as teaching experience, discipline, and professional title had limited predictive power for AI competency gains. Notably, self-perceived participation level did not significantly predict outcomes, whereas attendance rate emerged as a significant positive predictor. Interestingly, negative gain scores were observed in both groups. Follow-up interviews indicated that these scores did not reflect an actual decline in AI competency but rather a metacognitive recalibration, in which teachers shifted from unconscious incompetence to conscious incompetence—a pattern consistent with the Dunning–Kruger effect. This finding offers a novel theoretical perspective on the mechanism of change underlying the intervention. Overall, the PD programme based on the Intelligent-TPACK framework effectively enhanced university teachers’ AI competency and provides a systematic and evidence-based model for future PD initiatives in the AI era.</div></div>","PeriodicalId":34469,"journal":{"name":"Computers and Education Artificial Intelligence","volume":"9 ","pages":"Article 100521"},"PeriodicalIF":0.0,"publicationDate":"2025-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145736669","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-11-29DOI: 10.1016/j.caeai.2025.100514
Peiwen Huang , Yanling Hwang , Jui Ling Hsu , Chien Fand Peng , Cheng Han Tsai , Chih Yao Wang
Despite the growing importance of English oral communication skills, traditional language learning approaches show limited effectiveness in simultaneously addressing psychological barriers and speaking proficiency among college students. While previous studies have explored anxiety reduction or speaking enhancement separately, a significant gap exists in research examining integrated approaches that tackle Public Speaking Anxiety (PSA), Interview Anxiety, and English-speaking proficiency improvement simultaneously. This study investigated whether an AI-integrated VR oral training application could effectively address these interconnected challenges. A quasi-experimental design was employed with 20 English major students from a mid-central university in Taiwan. Participants completed five training sessions using Meta Quest 2 headsets and an AI-integrated VR oral training application providing tailored feedback on pronunciation, grammar, and fluency based on IELTS standards. Pre- and post-intervention assessments utilized validated instruments including the Personal Report of Public Speaking Anxiety (PRPSA) and Measure of Anxiety in Selection Interviews (MASI), alongside comprehensive speaking proficiency measures. Results demonstrated significant improvements in English speaking proficiency, including increased sentence length and word count, with grammatical errors and incomplete sentences decreasing markedly (p < .001). Concurrently, significant reductions in both PRPSA and MASI scores (p < .05) were observed, though lexical diversity showed slight decline. VR-related motion-sickness symptoms were mildly alleviated, and participants' perceived control increased significantly (p < .05), while interest and attention levels remained stable. These findings suggest that AI-integrated VR oral training applications can effectively enhance English speaking proficiency while simultaneously reducing anxiety levels and improving self-efficacy among English learners. The study addresses a critical research gap by demonstrating the potential of integrated technological approaches to tackle multiple barriers to effective English oral communication, offering promising implications for language education and anxiety management in academic contexts.
{"title":"The effectiveness of an AI-integrated VR oral training application in reducing public speaking anxiety and interview anxiety","authors":"Peiwen Huang , Yanling Hwang , Jui Ling Hsu , Chien Fand Peng , Cheng Han Tsai , Chih Yao Wang","doi":"10.1016/j.caeai.2025.100514","DOIUrl":"10.1016/j.caeai.2025.100514","url":null,"abstract":"<div><div>Despite the growing importance of English oral communication skills, traditional language learning approaches show limited effectiveness in simultaneously addressing psychological barriers and speaking proficiency among college students. While previous studies have explored anxiety reduction or speaking enhancement separately, a significant gap exists in research examining integrated approaches that tackle Public Speaking Anxiety (PSA), Interview Anxiety, and English-speaking proficiency improvement simultaneously. This study investigated whether an AI-integrated VR oral training application could effectively address these interconnected challenges. A quasi-experimental design was employed with 20 English major students from a mid-central university in Taiwan. Participants completed five training sessions using Meta Quest 2 headsets and an AI-integrated VR oral training application providing tailored feedback on pronunciation, grammar, and fluency based on IELTS standards. Pre- and post-intervention assessments utilized validated instruments including the Personal Report of Public Speaking Anxiety (PRPSA) and Measure of Anxiety in Selection Interviews (MASI), alongside comprehensive speaking proficiency measures. Results demonstrated significant improvements in English speaking proficiency, including increased sentence length and word count, with grammatical errors and incomplete sentences decreasing markedly (p < .001). Concurrently, significant reductions in both PRPSA and MASI scores (p < .05) were observed, though lexical diversity showed slight decline. VR-related motion-sickness symptoms were mildly alleviated, and participants' perceived control increased significantly (p < .05), while interest and attention levels remained stable. These findings suggest that AI-integrated VR oral training applications can effectively enhance English speaking proficiency while simultaneously reducing anxiety levels and improving self-efficacy among English learners. The study addresses a critical research gap by demonstrating the potential of integrated technological approaches to tackle multiple barriers to effective English oral communication, offering promising implications for language education and anxiety management in academic contexts.</div></div>","PeriodicalId":34469,"journal":{"name":"Computers and Education Artificial Intelligence","volume":"10 ","pages":"Article 100514"},"PeriodicalIF":0.0,"publicationDate":"2025-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145926802","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-11-28DOI: 10.1016/j.caeai.2025.100511
Jim Lo , Christy Wong , Agnes Ng , Pinna Wong , Denise Cheung , Pauli Lai
Advances in large language models (LLMs) enable timely and scalable writing evaluation. Previous research has shown that LLM-driven conversational systems, such as ChatGPT, can provide feedback on short essays. However, it is unclear whether AI can effectively evaluate more demanding genres. This study investigates a custom-built writing feedback system developed at a Hong Kong university that uses OpenAI's GPT-4 Turbo (0125-preview) to provide rubric-based feedback on a 1500-word academic report. Guided by a detailed, rubric-aligned prompt, the system generated 333 feedback items from 37 undergraduates, which were analysed for accuracy, tone, and inclusion of examples. The analysis showed that most feedback was accurate and addressed both strengths and weaknesses, but over half lacked concrete examples. Often recycling phrases from rubric descriptors, the feedback was largely generic and occasionally inaccurate. Interview data from six students revealed that the AI feedback was valued for its coverage, efficiency, and constructive tone, yet its generic nature undermined its usefulness. Despite these limitations, students expressed interest in receiving both AI and teacher feedback for the efficiency and coverage that AI offers, alongside the specificity and relevance of teacher input. These findings suggest that employing a well-crafted prompt on an AI model with a large context window does not necessarily guarantee substantive feedback. Therefore, educators using AI-driven feedback systems should thoroughly assess these systems' capacity to handle extended academic writing. Future research could explore ways to refine prompts and system design for long-form writing assignments.
{"title":"Stretching AI's reach: Assessing an AI-driven feedback system for extended academic writing","authors":"Jim Lo , Christy Wong , Agnes Ng , Pinna Wong , Denise Cheung , Pauli Lai","doi":"10.1016/j.caeai.2025.100511","DOIUrl":"10.1016/j.caeai.2025.100511","url":null,"abstract":"<div><div>Advances in large language models (LLMs) enable timely and scalable writing evaluation. Previous research has shown that LLM-driven conversational systems, such as ChatGPT, can provide feedback on short essays. However, it is unclear whether AI can effectively evaluate more demanding genres. This study investigates a custom-built writing feedback system developed at a Hong Kong university that uses OpenAI's GPT-4 Turbo (0125-preview) to provide rubric-based feedback on a 1500-word academic report. Guided by a detailed, rubric-aligned prompt, the system generated 333 feedback items from 37 undergraduates, which were analysed for accuracy, tone, and inclusion of examples. The analysis showed that most feedback was accurate and addressed both strengths and weaknesses, but over half lacked concrete examples. Often recycling phrases from rubric descriptors, the feedback was largely generic and occasionally inaccurate. Interview data from six students revealed that the AI feedback was valued for its coverage, efficiency, and constructive tone, yet its generic nature undermined its usefulness. Despite these limitations, students expressed interest in receiving both AI and teacher feedback for the efficiency and coverage that AI offers, alongside the specificity and relevance of teacher input. These findings suggest that employing a well-crafted prompt on an AI model with a large context window does not necessarily guarantee substantive feedback. Therefore, educators using AI-driven feedback systems should thoroughly assess these systems' capacity to handle extended academic writing. Future research could explore ways to refine prompts and system design for long-form writing assignments.</div></div>","PeriodicalId":34469,"journal":{"name":"Computers and Education Artificial Intelligence","volume":"10 ","pages":"Article 100511"},"PeriodicalIF":0.0,"publicationDate":"2025-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145718922","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号