Margaret Bearman, Paige Mahoney, Harsha Chandir, Christine Contessotto, M. Dunn, Brandi Fox, Fiona H. McKay, Darci Taylor
Innovative changes to online teaching practices are becoming increasingly important with the rise of e-learning across the higher education sector. Such innovations ideally become part of teaching repertoires rather than reverting to prior approaches. This study investigated the sustainability of a centralised approach to online learning design. Of the 74 survey respondents, more than 70% described changes to their teaching or work practices, uptake of learning design tools or development of student-facing materials, and these changes were mostly reported as sustained. Changes to system-level supports were less common and less likely to be sustained. Free-text comments highlight the importance of adaptation, collaboration and iteratively developed learning design templates. These provide guidance on how to promote sustained changes to online teaching practice. Implications for practice or policy: Outcomes for sustained changes to online teaching practice can be improved through innovations that are adaptable and collaborative and build educator capability. Learning design templates and resources can guide educators to develop their online teaching practice while allowing flexible and iterative implementation. Sustained changes to online teaching practices may be impeded by aging student-facing materials, withdrawal of support and unfeasible workloads.
{"title":"How to sustain a centralised approach to learning design","authors":"Margaret Bearman, Paige Mahoney, Harsha Chandir, Christine Contessotto, M. Dunn, Brandi Fox, Fiona H. McKay, Darci Taylor","doi":"10.14742/ajet.9114","DOIUrl":"https://doi.org/10.14742/ajet.9114","url":null,"abstract":"Innovative changes to online teaching practices are becoming increasingly important with the rise of e-learning across the higher education sector. Such innovations ideally become part of teaching repertoires rather than reverting to prior approaches. This study investigated the sustainability of a centralised approach to online learning design. Of the 74 survey respondents, more than 70% described changes to their teaching or work practices, uptake of learning design tools or development of student-facing materials, and these changes were mostly reported as sustained. Changes to system-level supports were less common and less likely to be sustained. Free-text comments highlight the importance of adaptation, collaboration and iteratively developed learning design templates. These provide guidance on how to promote sustained changes to online teaching practice.\u0000 \u0000Implications for practice or policy:\u0000\u0000Outcomes for sustained changes to online teaching practice can be improved through innovations that are adaptable and collaborative and build educator capability.\u0000Learning design templates and resources can guide educators to develop their online teaching practice while allowing flexible and iterative implementation.\u0000Sustained changes to online teaching practices may be impeded by aging student-facing materials, withdrawal of support and unfeasible workloads.\u0000","PeriodicalId":47812,"journal":{"name":"Australasian Journal of Educational Technology","volume":null,"pages":null},"PeriodicalIF":3.3,"publicationDate":"2024-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141812184","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The rapid recent use of learning analytics (LA) in higher education, specifically during the COVID-19 pandemic, allows the monitoring of users' behavior while learning. Using LA may promote students' learning outcomes but also intrude into their privacy. This study aimed to explore students’ behaviour and perceptions towards privacy and data protection when using LA for pedagogical needs, examine the privacy trade-off of students' willingness to share personal information in exchange for pedagogical benefits and understand the predicting variables for this privacy trade-off. A model was developed containing five groups of influencing variables (demographic data, perceptions, feelings, behaviour and awareness) on the privacy trade-off. A total of 1,014 students completed an online questionnaire. The results found that students do care about their privacy but are not aware of privacy and data protection regulations. They are willing to trade off privacy for pedagogical benefits, and they trust their academic institutions, but they want transparency. Age, a sense of security in the academic institution, behaviour, data misuse concern and institution management of students' personal information are the significant predictors for a privacy trade-off. It is important to engage students in the process as they are the main beneficiaries of LA and build trust between them and the institution. Implications for practice or policy: Academic institution should actively provide information to raise privacy awareness. Academic institutions should engage students in the process of using LA to create a high degree of trust. Universities should endorse LA policy, fostering it for pedagogical purposes. Instructors may utilise LA to enrich student learning, respecting their willingness to share pedagogical data. Academic institutions should provide a high level of transparency in order to build students’ trust in their institutions.
最近，特别是在 COVID-19 大流行期间，学习分析（LA）在高等教育中得到了迅速应用，可以监控用户在学习过程中的行为。使用学习分析可能会促进学生的学习成果，但也会侵犯他们的隐私。本研究旨在探讨学生在因教学需要使用实验室时对隐私和数据保护的行为和看法，研究学生愿意分享个人信息以换取教学利益的隐私权衡，并了解这种隐私权衡的预测变量。研究建立了一个模型，其中包含五组影响隐私权衡的变量（人口统计学数据、看法、感受、行为和意识）。共有 1 014 名学生完成了在线问卷调查。结果发现，学生确实关心自己的隐私，但并不了解隐私和数据保护法规。他们愿意用隐私换取教学上的好处，他们信任学术机构，但他们希望透明。年龄、对学术机构的安全感、行为、对数据滥用的担忧以及学术机构对学生个人信息的管理是预测隐私权取舍的重要因素。重要的是要让学生参与到这一过程中来，因为他们是 "LA "的主要受益者，并在他们和院校之间建立信任。对实践或政策的启示：学术机构应积极提供信息，以提高隐私意识。学术机构应让学生参与使用LA的过程，以建立高度信任。大学应认可LA政策，将其用于教学目的。教师可利用LA丰富学生的学习，尊重他们共享教学数据的意愿。学术机构应提供高透明度，以建立学生对其机构的信任。
{"title":"Privacy versus pedagogy – students’ perceptions of using learning analytics in higher education","authors":"T. Soffer, Anat Cohen","doi":"10.14742/ajet.9130","DOIUrl":"https://doi.org/10.14742/ajet.9130","url":null,"abstract":"The rapid recent use of learning analytics (LA) in higher education, specifically during the COVID-19 pandemic, allows the monitoring of users' behavior while learning. Using LA may promote students' learning outcomes but also intrude into their privacy. This study aimed to explore students’ behaviour and perceptions towards privacy and data protection when using LA for pedagogical needs, examine the privacy trade-off of students' willingness to share personal information in exchange for pedagogical benefits and understand the predicting variables for this privacy trade-off. A model was developed containing five groups of influencing variables (demographic data, perceptions, feelings, behaviour and awareness) on the privacy trade-off. A total of 1,014 students completed an online questionnaire. The results found that students do care about their privacy but are not aware of privacy and data protection regulations. They are willing to trade off privacy for pedagogical benefits, and they trust their academic institutions, but they want transparency. Age, a sense of security in the academic institution, behaviour, data misuse concern and institution management of students' personal information are the significant predictors for a privacy trade-off. It is important to engage students in the process as they are the main beneficiaries of LA and build trust between them and the institution.\u0000 \u0000Implications for practice or policy:\u0000\u0000Academic institution should actively provide information to raise privacy awareness.\u0000Academic institutions should engage students in the process of using LA to create a high degree of trust.\u0000Universities should endorse LA policy, fostering it for pedagogical purposes.\u0000Instructors may utilise LA to enrich student learning, respecting their willingness to share pedagogical data.\u0000Academic institutions should provide a high level of transparency in order to build students’ trust in their institutions.\u0000","PeriodicalId":47812,"journal":{"name":"Australasian Journal of Educational Technology","volume":null,"pages":null},"PeriodicalIF":3.3,"publicationDate":"2024-07-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141815444","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Spanish face-to-face universities experience constant challenges that threaten the inclusion of students with disabilities. Adopting inclusive pedagogies can support universities to develop a more inclusive approach. This paper explores the extent to which the use of technology influences the inclusive pedagogies of Spanish university lecturers. In particular, we focus on how lecturers use technologies to promote student participation and accessibility in Spanish institution. A qualitative methodology involving semi-structured in-depth interviews with 42 Spanish face-to-face universities experience constant challenges that threaten the inclusion of students with disabilities. Adopting inclusive pedagogies can support universities to develop a more inclusive approach. This paper explores the extent to which the use of technology influences the inclusive pedagogies of Spanish university lecturers. In particular, we focus on how lecturers use technologies to promote student participation and accessibility in Spanish institution. A qualitative methodology involving semi-structured in-depth interviews with 42 Spanish university lecturers regarding their inclusive practices and use of technology was employed. The results contribute four fundamental findings: (a) Lecturers who practise inclusive pedagogy can be differentiated as proactive or reactive; (b) University lecturers place greater emphasis on the use of technology as a promoter of accessibility rather than as a tool to promote participation; (c) There is a disconnect between knowledge of universal design for learning and the use of the technologies; and (d) The full transformative potential of technologies to facilitate the inclusion of learners with disabilities is not being harnessed. All these results allow us to highlight some key points about the use of technological tools in the application of inclusive pedagogy in university classrooms. Implications for practice or policy: • Spanish face-to-face universities still need to harness the full transformative potential of technologies to facilitate student inclusion. • There is a need for more professional development programmes within higher education institutions that focus on the potential of technologies to promote inclusion. • Universities have an important role to play in promoting the use of technological tools to ensure accessibility and participation of all students, especially students with disabilities.
{"title":"Access and Participation: The use of Technologies as tools for Inclusion by Spanish University Lecturers","authors":"María Nieves Sánchez Díaz, B. Morgado, Jane Seale","doi":"10.14742/ajet.8178","DOIUrl":"https://doi.org/10.14742/ajet.8178","url":null,"abstract":"Spanish face-to-face universities experience constant challenges that threaten the inclusion of students with disabilities. Adopting inclusive pedagogies can support universities to develop a more inclusive approach. This paper explores the extent to which the use of technology influences the inclusive pedagogies of Spanish university lecturers. In particular, we focus on how lecturers use technologies to promote student participation and accessibility in Spanish institution. A qualitative methodology involving semi-structured in-depth interviews with 42 Spanish face-to-face universities experience constant challenges that threaten the inclusion of students with disabilities. Adopting inclusive pedagogies can support universities to develop a more inclusive approach. This paper explores the extent to which the use of technology influences the inclusive pedagogies of Spanish university lecturers. In particular, we focus on how lecturers use technologies to promote student participation and accessibility in Spanish institution. A qualitative methodology involving semi-structured in-depth interviews with 42 Spanish university lecturers regarding their inclusive practices and use of technology was employed. The results contribute four fundamental findings: (a) Lecturers who practise inclusive pedagogy can be differentiated as proactive or reactive; (b) University lecturers place greater emphasis on the use of technology as a promoter of accessibility rather than as a tool to promote participation; (c) There is a disconnect between knowledge of universal design for learning and the use of the technologies; and (d) The full transformative potential of technologies to facilitate the inclusion of learners with disabilities is not being harnessed. All these results allow us to highlight some key points about the use of technological tools in the application of inclusive pedagogy in university classrooms.\u0000Implications for practice or policy: • Spanish face-to-face universities still need to harness the full transformative potential of technologies to facilitate student inclusion. • There is a need for more professional development programmes within higher education institutions that focus on the potential of technologies to promote inclusion. • Universities have an important role to play in promoting the use of technological tools to ensure accessibility and participation of all students, especially students with disabilities.","PeriodicalId":47812,"journal":{"name":"Australasian Journal of Educational Technology","volume":null,"pages":null},"PeriodicalIF":4.1,"publicationDate":"2024-03-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140237887","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yuchen Liu, Stanislav Pozdniakov, Roberto Martinez-Maldonado
Learning analytics (LA) dashboards are becoming increasingly available in various learning settings. However, teachers may face challenges in understanding and interpreting the data visualisations presented on those dashboards. In response to this, some LA researchers are incorporating visual cueing techniques, like data storytelling (DS), into LA dashboard design to reduce the data visualisation skills – often referred to as visualisation literacy (VL) – and cognitive effort required by teachers to effectively use dashboards. However, despite the potential of DS principles in simplifying data visualisations, there is limited evidence supporting their effectiveness in actually reducing teachers’ cognitive load. The study presented in this paper addresses this gap by investigating the potential impact of LA dashboards, with and without DS elements, on teachers with varying VL levels. Through a quasi-experimental study involving 23 teachers, we analysed changes in pupil dilation – a proxy for cognitive load – as they examined LA dashboards featuring student data captured while participating in synchronous, online collaborative learning tasks. Our findings suggest DS can reduce cognitive load, particularly for teachers with lower VL. These results provide insight into the effects of DS and VL on teachers’ cognitive load, thereby informing the design of LA dashboards. Implications for practice or policy:• Developers of LA dashboards need to pay more attention to incorporating visual and narrative elements that are easily comprehensible and target-oriented, based on users’ visualisation literacy levels.• Educational providers and LA designers can recommend dashboards with DS elements to teachers with low VL to enhance their work efficiency.
{"title":"The effects of visualisation literacy and data storytelling dashboards on teachers’ cognitive load","authors":"Yuchen Liu, Stanislav Pozdniakov, Roberto Martinez-Maldonado","doi":"10.14742/ajet.8988","DOIUrl":"https://doi.org/10.14742/ajet.8988","url":null,"abstract":"Learning analytics (LA) dashboards are becoming increasingly available in various learning settings. However, teachers may face challenges in understanding and interpreting the data visualisations presented on those dashboards. In response to this, some LA researchers are incorporating visual cueing techniques, like data storytelling (DS), into LA dashboard design to reduce the data visualisation skills – often referred to as visualisation literacy (VL) – and cognitive effort required by teachers to effectively use dashboards. However, despite the potential of DS principles in simplifying data visualisations, there is limited evidence supporting their effectiveness in actually reducing teachers’ cognitive load. The study presented in this paper addresses this gap by investigating the potential impact of LA dashboards, with and without DS elements, on teachers with varying VL levels. Through a quasi-experimental study involving 23 teachers, we analysed changes in pupil dilation – a proxy for cognitive load – as they examined LA dashboards featuring student data captured while participating in synchronous, online collaborative learning tasks. Our findings suggest DS can reduce cognitive load, particularly for teachers with lower VL. These results provide insight into the effects of DS and VL on teachers’ cognitive load, thereby informing the design of LA dashboards.\u0000Implications for practice or policy:• Developers of LA dashboards need to pay more attention to incorporating visual and narrative elements that are easily comprehensible and target-oriented, based on users’ visualisation literacy levels.• Educational providers and LA designers can recommend dashboards with DS elements to teachers with low VL to enhance their work efficiency.","PeriodicalId":47812,"journal":{"name":"Australasian Journal of Educational Technology","volume":null,"pages":null},"PeriodicalIF":4.1,"publicationDate":"2024-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140245088","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Madison Reynolds, Jonathan Tyler, James Wakefield, Raechel Wight
This paper examines the effectiveness of providing personalised email feedback via an automated email application in a large undergraduate introductory accounting course. Over 1,200 students received feedback via emails sent weekly, semi-tailored to each student based on their results in online self-test learning quizzes. We first found that students read the majority of emails distributed. Second, through tracking, using regression analysis, we found that reading emails is related to significantly higher final examination performance in some cases. However, this is moderated by factors relating to diversity in a large cohort exceeding 1,000 students. The results indicate that feedback needs to be readily actionable and aligned with assessed learning outcomes to realise significant impacts on exam performance. This study is relevant to educators who teach large and diverse cohorts and need time-efficient solutions to tailor the learning experience to each student. Implications for practice or policy: Undergraduate students read the majority of weekly emails distributed, indicating emails can be used as an instructor communication device. Emails should specifically encourage activity completion aligning with learning objectives to improve grades. Students who achieve lower grades in tertiary studies, from English-speaking backgrounds and medium to high socio-economic status, high performers at high school and younger students aged 20 or less are positively impacted. Therefore, emails should be used by instructors teaching cohorts with these characteristics.
{"title":"Students got mail: Do students read semi-tailored emails and what is the impact?","authors":"Madison Reynolds, Jonathan Tyler, James Wakefield, Raechel Wight","doi":"10.14742/ajet.8812","DOIUrl":"https://doi.org/10.14742/ajet.8812","url":null,"abstract":"This paper examines the effectiveness of providing personalised email feedback via an automated email application in a large undergraduate introductory accounting course. Over 1,200 students received feedback via emails sent weekly, semi-tailored to each student based on their results in online self-test learning quizzes. We first found that students read the majority of emails distributed. Second, through tracking, using regression analysis, we found that reading emails is related to significantly higher final examination performance in some cases. However, this is moderated by factors relating to diversity in a large cohort exceeding 1,000 students. The results indicate that feedback needs to be readily actionable and aligned with assessed learning outcomes to realise significant impacts on exam performance. This study is relevant to educators who teach large and diverse cohorts and need time-efficient solutions to tailor the learning experience to each student.\u0000 \u0000Implications for practice or policy:\u0000\u0000Undergraduate students read the majority of weekly emails distributed, indicating emails can be used as an instructor communication device.\u0000Emails should specifically encourage activity completion aligning with learning objectives to improve grades.\u0000Students who achieve lower grades in tertiary studies, from English-speaking backgrounds and medium to high socio-economic status, high performers at high school and younger students aged 20 or less are positively impacted. Therefore, emails should be used by instructors teaching cohorts with these characteristics.\u0000","PeriodicalId":47812,"journal":{"name":"Australasian Journal of Educational Technology","volume":null,"pages":null},"PeriodicalIF":4.1,"publicationDate":"2024-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140246496","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In the context of information and communication technology (ICT)-enhanced teaching, teacher well-being plays a crucial role in promoting teaching effectiveness and students’ learning achievement. Drawing on the interactionist model of teacher well-being, this study investigated university teachers’ well-being (e.g., emotional exhaustion and teacher engagement) in ICT-enhanced teaching and its associations with their self-efficacy (e.g., classroom management, instructional strategy and course design) and teaching support (e.g., autonomy support, teaching resources and peer support). The results of an online questionnaire survey conducted among 836 university teachers in China indicated that the enhanced integration of ICT into teaching practices neither impaired teacher engagement nor caused them significant emotional exhaustion. Instead, adequate teaching resources and autonomy support contributed positively to both teacher self-efficacy and engagement. Increased efficacy in course design and classroom management alleviated their emotional exhaustion. Moreover, teacher self-efficacy significantly mediated the effects of autonomy support on emotional exhaustion and teacher engagement. These results have practical implications for understanding and promoting university teachers’ well-being as well as teaching effectiveness in ICT-enhanced teaching environments. Implications for practice or policy Administrators may consider providing adequate resources geared towards enhancing university teachers’ confidence and engagement in ICT-enhanced teaching. Administrators may avoid introducing excessive and burdensome initiatives to university teachers to prevent teacher emotional exhaustion. University teachers may be granted significant autonomy in selecting their preferred teaching platforms, methods and materials to meet their specific needs and preferences in ICT-enhanced teaching.
{"title":"University teachers’ well-being in ICT-enhanced teaching: The roles of teacher self-efficacy and teaching support","authors":"Jiying Han, Chao Gao","doi":"10.14742/ajet.8868","DOIUrl":"https://doi.org/10.14742/ajet.8868","url":null,"abstract":"In the context of information and communication technology (ICT)-enhanced teaching, teacher well-being plays a crucial role in promoting teaching effectiveness and students’ learning achievement. Drawing on the interactionist model of teacher well-being, this study investigated university teachers’ well-being (e.g., emotional exhaustion and teacher engagement) in ICT-enhanced teaching and its associations with their self-efficacy (e.g., classroom management, instructional strategy and course design) and teaching support (e.g., autonomy support, teaching resources and peer support). The results of an online questionnaire survey conducted among 836 university teachers in China indicated that the enhanced integration of ICT into teaching practices neither impaired teacher engagement nor caused them significant emotional exhaustion. Instead, adequate teaching resources and autonomy support contributed positively to both teacher self-efficacy and engagement. Increased efficacy in course design and classroom management alleviated their emotional exhaustion. Moreover, teacher self-efficacy significantly mediated the effects of autonomy support on emotional exhaustion and teacher engagement. These results have practical implications for understanding and promoting university teachers’ well-being as well as teaching effectiveness in ICT-enhanced teaching environments.\u0000Implications for practice or policy\u0000\u0000Administrators may consider providing adequate resources geared towards enhancing university teachers’ confidence and engagement in ICT-enhanced teaching.\u0000Administrators may avoid introducing excessive and burdensome initiatives to university teachers to prevent teacher emotional exhaustion.\u0000University teachers may be granted significant autonomy in selecting their preferred teaching platforms, methods and materials to meet their specific needs and preferences in ICT-enhanced teaching.\u0000","PeriodicalId":47812,"journal":{"name":"Australasian Journal of Educational Technology","volume":null,"pages":null},"PeriodicalIF":4.1,"publicationDate":"2023-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138944605","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Binh Nguyen Thanh, Diem Thi-Ngoc Vo, Minh Nguyen Nhat, Thi Thu Tra Pham, Hieu Thai Trung, Son Ha Xuan
In this study, we introduce a framework designed to help educators assess the effectiveness of popular generative artificial intelligence (AI) tools in solving authentic assessments. We employed Bloom’s taxonomy as a guiding principle to create authentic assessments that evaluate the capabilities of generative AI tools. We applied this framework to assess the abilities of ChatGPT-4, ChatGPT-3.5, Google Bard and Microsoft Bing in solving authentic assessments in economics. We found that generative AI tools perform very well at the lower levels of Bloom's taxonomy while still maintaining a decent level of performance at the higher levels, with “create” being the weakest level of performance. Interestingly, these tools are better able to address numeric-based questions than text-based ones. Moreover, all the generative AI tools exhibit weaknesses in building arguments based on theoretical frameworks, maintaining the coherence of different arguments and providing appropriate references. Our study provides educators with a framework to assess the capabilities of generative AI tools, enabling them to make more informed decisions regarding assessments and learning activities. Our findings demand a strategic reimagining of educational goals and assessments, emphasising higher cognitive skills and calling for a concerted effort to enhance the capabilities of educators in preparing students for a rapidly transforming professional environment. Implications for practice or policy Our proposed framework enables educators to systematically evaluate the capabilities of widely used generative AI tools in assessments and assist them in the assessment design process. Tertiary institutions should re-evaluate and redesign programmes and course learning outcomes. The new focus on learning outcomes should address the higher levels of educational goals of Bloom’s taxonomy, specifically the “create” level.
{"title":"Race with the machines: Assessing the capability of generative AI in solving authentic assessments","authors":"Binh Nguyen Thanh, Diem Thi-Ngoc Vo, Minh Nguyen Nhat, Thi Thu Tra Pham, Hieu Thai Trung, Son Ha Xuan","doi":"10.14742/ajet.8902","DOIUrl":"https://doi.org/10.14742/ajet.8902","url":null,"abstract":"In this study, we introduce a framework designed to help educators assess the effectiveness of popular generative artificial intelligence (AI) tools in solving authentic assessments. We employed Bloom’s taxonomy as a guiding principle to create authentic assessments that evaluate the capabilities of generative AI tools. We applied this framework to assess the abilities of ChatGPT-4, ChatGPT-3.5, Google Bard and Microsoft Bing in solving authentic assessments in economics. We found that generative AI tools perform very well at the lower levels of Bloom's taxonomy while still maintaining a decent level of performance at the higher levels, with “create” being the weakest level of performance. Interestingly, these tools are better able to address numeric-based questions than text-based ones. Moreover, all the generative AI tools exhibit weaknesses in building arguments based on theoretical frameworks, maintaining the coherence of different arguments and providing appropriate references. Our study provides educators with a framework to assess the capabilities of generative AI tools, enabling them to make more informed decisions regarding assessments and learning activities. Our findings demand a strategic reimagining of educational goals and assessments, emphasising higher cognitive skills and calling for a concerted effort to enhance the capabilities of educators in preparing students for a rapidly transforming professional environment.\u0000Implications for practice or policy\u0000\u0000Our proposed framework enables educators to systematically evaluate the capabilities of widely used generative AI tools in assessments and assist them in the assessment design process.\u0000Tertiary institutions should re-evaluate and redesign programmes and course learning outcomes. The new focus on learning outcomes should address the higher levels of educational goals of Bloom’s taxonomy, specifically the “create” level.\u0000","PeriodicalId":47812,"journal":{"name":"Australasian Journal of Educational Technology","volume":null,"pages":null},"PeriodicalIF":4.1,"publicationDate":"2023-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138947823","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Generative artificial intelligence (AI) has had a significant impact in tertiary education for practitioners and researchers during 2023. We review the way in which academics have made sense of generative AI, revisit our proposed research agenda and reflect on our changing roles as academics in relation to learning, teaching, design and policy.
{"title":"AI in tertiary education: progress on research and practice","authors":"Kate Thompson, L. Corrin, J. Lodge","doi":"10.14742/ajet.9251","DOIUrl":"https://doi.org/10.14742/ajet.9251","url":null,"abstract":"Generative artificial intelligence (AI) has had a significant impact in tertiary education for practitioners and researchers during 2023. We review the way in which academics have made sense of generative AI, revisit our proposed research agenda and reflect on our changing roles as academics in relation to learning, teaching, design and policy.","PeriodicalId":47812,"journal":{"name":"Australasian Journal of Educational Technology","volume":null,"pages":null},"PeriodicalIF":4.1,"publicationDate":"2023-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138994202","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Bünyamin Kayali, Mehmet Yavuz, Şener Balat, M. Çalişan
The purpose of this study was to determine university students' experiences with the use of ChatGPT in online courses. The sample consisted of 84 associate degree students from a state university in Turkey. A multi-method approach was used in the study. Although quantitative data were collected using the Chatbot Usability Scale, qualitative data were collected using a semi-structured interview form that we developed. The data were analysed using descriptive and content analysis methods. According to the findings, ChatGPT exhibits advantages such as a user-friendly interface and fast, concise, relevant responses. Moreover, emphasizing its contribution to the learning process, the information provided was sufficient and topic-oriented. The understandability of the chatbot’s functions and the clarity of their communication were emphasized. However, there are disadvantages such as performance issues, frequency of errors and the risk of providing misleading information. Concerns have also been raised about the potential difficulties chatbots may face in ambiguous conversations and providing insufficient information on privacy issues. In conclusion, ChatGPT is recognised as a potentially valuable tool in education based on positive usability impressions; however, more research is needed for its safe use. Implications for practice or policy Based on positive usability impressions, students and instructors can use ChatGPT to support educational activities. ChatGPT can promote and enhance students' personalised learning experiences. ChatGPT can be used in all higher education courses. Users should be cautious about the accuracy and reliability of the answers provided by ChatGPT. Decision-makers should take precautions against risks such as privacy, ethics, confidentiality and security that may arise from using artificial intelligence in education.
{"title":"Investigation of student experiences with ChatGPT-supported online learning applications in higher education","authors":"Bünyamin Kayali, Mehmet Yavuz, Şener Balat, M. Çalişan","doi":"10.14742/ajet.8915","DOIUrl":"https://doi.org/10.14742/ajet.8915","url":null,"abstract":"The purpose of this study was to determine university students' experiences with the use of ChatGPT in online courses. The sample consisted of 84 associate degree students from a state university in Turkey. A multi-method approach was used in the study. Although quantitative data were collected using the Chatbot Usability Scale, qualitative data were collected using a semi-structured interview form that we developed. The data were analysed using descriptive and content analysis methods. According to the findings, ChatGPT exhibits advantages such as a user-friendly interface and fast, concise, relevant responses. Moreover, emphasizing its contribution to the learning process, the information provided was sufficient and topic-oriented. The understandability of the chatbot’s functions and the clarity of their communication were emphasized. However, there are disadvantages such as performance issues, frequency of errors and the risk of providing misleading information. Concerns have also been raised about the potential difficulties chatbots may face in ambiguous conversations and providing insufficient information on privacy issues. In conclusion, ChatGPT is recognised as a potentially valuable tool in education based on positive usability impressions; however, more research is needed for its safe use.\u0000Implications for practice or policy\u0000\u0000Based on positive usability impressions, students and instructors can use ChatGPT to support educational activities.\u0000ChatGPT can promote and enhance students' personalised learning experiences.\u0000ChatGPT can be used in all higher education courses.\u0000Users should be cautious about the accuracy and reliability of the answers provided by ChatGPT.\u0000Decision-makers should take precautions against risks such as privacy, ethics, confidentiality and security that may arise from using artificial intelligence in education.\u0000","PeriodicalId":47812,"journal":{"name":"Australasian Journal of Educational Technology","volume":null,"pages":null},"PeriodicalIF":4.1,"publicationDate":"2023-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138945617","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Simon Knight, Camille Dickson-Deane, Keith Heggart, Kirsty Kitto, Dilek Çetindamar Kozanoğlu, Damian Maher, Bhuva Narayan, Forooq Zarrabi
The launch of new tools in late 2022 heralded significant growth in attention to the impacts of generative AI (GenAI) in education. Claims of the potential impact on education are contested, but there are clear risks of inappropriate use particularly where GenAI aligns poorly with learning aims. In response, in mid-2023, the Australian Federal Government held an inquiry, calling for public submissions. This inquiry offers a lens onto the policy framing of GenAI in education and provides the object of investigation for this paper. We use the inquiry submissions, extracting structured claims from each. This extraction is provided as an open data set for further research, while this paper focuses on our analysis of the policy recommendations made. Implications for practice or policy For practitioners, policymakers, and researchers. the paper provides an overview and synthesis of submission recommendations and their themes, by source type. For respondents to the inquiry (sources), the paper supports reflection regarding synergies and gaps in recommendations, pointing to opportunity for collaboration and policy development. For stakeholders with responsibility for aspects of policy delivery and/or those applying a critical lens to the inquiry and recommendation framing(s), the paper offers actionable insight.
{"title":"Generative AI in the Australian education system: An open data set of stakeholder recommendations and emerging analysis from a public inquiry","authors":"Simon Knight, Camille Dickson-Deane, Keith Heggart, Kirsty Kitto, Dilek Çetindamar Kozanoğlu, Damian Maher, Bhuva Narayan, Forooq Zarrabi","doi":"10.14742/ajet.8922","DOIUrl":"https://doi.org/10.14742/ajet.8922","url":null,"abstract":"The launch of new tools in late 2022 heralded significant growth in attention to the impacts of generative AI (GenAI) in education. Claims of the potential impact on education are contested, but there are clear risks of inappropriate use particularly where GenAI aligns poorly with learning aims. In response, in mid-2023, the Australian Federal Government held an inquiry, calling for public submissions. This inquiry offers a lens onto the policy framing of GenAI in education and provides the object of investigation for this paper. We use the inquiry submissions, extracting structured claims from each. This extraction is provided as an open data set for further research, while this paper focuses on our analysis of the policy recommendations made.\u0000Implications for practice or policy\u0000\u0000For practitioners, policymakers, and researchers. the paper provides an overview and synthesis of submission recommendations and their themes, by source type.\u0000For respondents to the inquiry (sources), the paper supports reflection regarding synergies and gaps in recommendations, pointing to opportunity for collaboration and policy development.\u0000For stakeholders with responsibility for aspects of policy delivery and/or those applying a critical lens to the inquiry and recommendation framing(s), the paper offers actionable insight.\u0000","PeriodicalId":47812,"journal":{"name":"Australasian Journal of Educational Technology","volume":null,"pages":null},"PeriodicalIF":4.1,"publicationDate":"2023-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138947052","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：