Alice Huang,Dale Hancock,Matthew Clemson,Giselle Yeo,Dylan Harney,Paul Denny,Gareth Denyer
Production of high-quality multiple-choice questions (MCQs) for both formative and summative assessments is a time-consuming task requiring great skill, creativity and insight. The transition to online examinations, with the concomitant exposure of previously tried-and-tested MCQs, exacerbates the challenges of question production and highlights the need for innovative solutions. Several groups have shown that it is practical to leverage the student cohort to produce a very large number of syllabus-aligned MCQs for study banks. Although student-generated questions are well suited for formative feedback and practice activities, they are generally not thought to be suitable for high-stakes assessments. In this study, we aimed to demonstrate that training can be provided to students in a scalable fashion to generate questions of similar quality to those produced by experts and that identification of suitable questions can be achieved with minimal academic review and editing. Second-year biochemistry and molecular biology students were assigned a series of activities designed to coach them in the art of writing and critiquing MCQs. This training resulted in the production of over 1000 MCQs that were then gauged for potential by either expert academic judgement or via a data-driven approach in which the questions were trialled objectively in a low-stakes test. Questions selected by either method were then deployed in a high-stakes in-semester assessment alongside questions from two academically authored sources: textbook-derived MCQs and past paper questions. A total of 120 MCQs from these four sources were deployed in assessments attempted by over 600 students. Each question was subjected to rigorous performance analysis, including the calculation of standard metrics from classical test theory and more sophisticated item response theory (IRT) measures. The results showed that MCQs authored by students, and selected at low cost, performed as well as questions authored by academics, illustrating the potential of this strategy for the efficient creation of large numbers of high-quality MCQs for summative assessment.
{"title":"Selecting student-authored questions for summative assessments","authors":"Alice Huang,Dale Hancock,Matthew Clemson,Giselle Yeo,Dylan Harney,Paul Denny,Gareth Denyer","doi":"10.25304/rlt.v29.2517","DOIUrl":"https://doi.org/10.25304/rlt.v29.2517","url":null,"abstract":"Production of high-quality multiple-choice questions (MCQs) for both formative and summative assessments is a time-consuming task requiring great skill, creativity and insight. The transition to online examinations, with the concomitant exposure of previously tried-and-tested MCQs, exacerbates the challenges of question production and highlights the need for innovative solutions. Several groups have shown that it is practical to leverage the student cohort to produce a very large number of syllabus-aligned MCQs for study banks. Although student-generated questions are well suited for formative feedback and practice activities, they are generally not thought to be suitable for high-stakes assessments. In this study, we aimed to demonstrate that training can be provided to students in a scalable fashion to generate questions of similar quality to those produced by experts and that identification of suitable questions can be achieved with minimal academic review and editing. Second-year biochemistry and molecular biology students were assigned a series of activities designed to coach them in the art of writing and critiquing MCQs. This training resulted in the production of over 1000 MCQs that were then gauged for potential by either expert academic judgement or via a data-driven approach in which the questions were trialled objectively in a low-stakes test. Questions selected by either method were then deployed in a high-stakes in-semester assessment alongside questions from two academically authored sources: textbook-derived MCQs and past paper questions. A total of 120 MCQs from these four sources were deployed in assessments attempted by over 600 students. Each question was subjected to rigorous performance analysis, including the calculation of standard metrics from classical test theory and more sophisticated item response theory (IRT) measures. The results showed that MCQs authored by students, and selected at low cost, performed as well as questions authored by academics, illustrating the potential of this strategy for the efficient creation of large numbers of high-quality MCQs for summative assessment.","PeriodicalId":46691,"journal":{"name":"Research in Learning Technology","volume":"24 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2021-02-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138533291","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This research examined first year undergraduate tertiary student preferences for different online video playback options by comparing a didactic long lecture recording versus a series of topical ‘chunked’ videos of identical learning material in an information literacy unit. Student preference was determined by student unique download choice of streaming video lecture material, cumulative visits and percent completion of viewing of lecture videos. De-identified click-stream data for 1268 university students across two academic years 2016 (n = 647) and 2017 (n = 621) were pooled to examine student preference. The major findings indicated a significant preference for chunk-style videos between 3 and 17 min duration when compared to traditional long-view didactic lecture materials. Results also highlighted an increase in unique views (60%–67%), cumulative visits (54%–67%) and percentage completions (25%) of chunked videos compared to didactic lectures (60 min). Additionally, student total viewing of the unit information influenced the final grade for the unit. Student preference and success were in favour of the smaller chunk-style lectures, which may also improve student attention, assist with time management to complete the materials and increase unit engagement. The overall findings of this research re-enforce the value of student-centric learning design in university education settings.
{"title":"An examination of student preference for traditional didactic or chunking teaching strategies in an online learning environment","authors":"B. Humphries, D. Clark","doi":"10.25304/RLT.V29.2405","DOIUrl":"https://doi.org/10.25304/RLT.V29.2405","url":null,"abstract":"This research examined first year undergraduate tertiary student preferences for different online video playback options by comparing a didactic long lecture recording versus a series of topical ‘chunked’ videos of identical learning material in an information literacy unit. Student preference was determined by student unique download choice of streaming video lecture material, cumulative visits and percent completion of viewing of lecture videos. De-identified click-stream data for 1268 university students across two academic years 2016 (n = 647) and 2017 (n = 621) were pooled to examine student preference. The major findings indicated a significant preference for chunk-style videos between 3 and 17 min duration when compared to traditional long-view didactic lecture materials. Results also highlighted an increase in unique views (60%–67%), cumulative visits (54%–67%) and percentage completions (25%) of chunked videos compared to didactic lectures (60 min). Additionally, student total viewing of the unit information influenced the final grade for the unit. Student preference and success were in favour of the smaller chunk-style lectures, which may also improve student attention, assist with time management to complete the materials and increase unit engagement. The overall findings of this research re-enforce the value of student-centric learning design in university education settings.","PeriodicalId":46691,"journal":{"name":"Research in Learning Technology","volume":"43 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2021-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76610060","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Prajakt Pande, Amalie Thit, A. Sørensen, B. Mojsoska, Morten E. Moeller, P. Jepsen
Our main goal was to investigate if and how using multiple immersive virtual reality (iVR) simulations and their video playback, in a science course, affects student learning over time. We conducted a longitudinal study, in ecological settings, at an undergraduate field-course on three topics in environmental biology. Twenty-eight undergraduates were randomly assigned to either an iVR-interaction group or a video-viewing group. During the field-course, the iVR group interacted with a head-mounted device-based iVR simulation related to each topic (i.e. total three interventions), while the video group watched a pre-recorded video of the respective simulation on a laptop. Cognitive and affective data were collected through the following checkpoints: a pre-test before the first intervention, one topic-specific post-test immediately after each intervention, a final post-test towards the end of the course, and a longitudinal post-test deployed approximately 2 months after the course. Through a descriptive analysis, it was found that student performance on the knowledge tests increased considerably over time for the iVR group but remained unchanged for the video group. While no within- or between-group differences were noted for intrinsic motivation and self-efficacy measures, students in the iVR group enjoyed all the simulations, and perceived themselves to benefit from those simulations.
{"title":"Long-term effectiveness of immersive VR simulations in undergraduate science learning: lessons from a media-comparison study","authors":"Prajakt Pande, Amalie Thit, A. Sørensen, B. Mojsoska, Morten E. Moeller, P. Jepsen","doi":"10.25304/RLT.V29.2482","DOIUrl":"https://doi.org/10.25304/RLT.V29.2482","url":null,"abstract":"Our main goal was to investigate if and how using multiple immersive virtual reality (iVR) simulations and their video playback, in a science course, affects student learning over time. We conducted a longitudinal study, in ecological settings, at an undergraduate field-course on three topics in environmental biology. Twenty-eight undergraduates were randomly assigned to either an iVR-interaction group or a video-viewing group. During the field-course, the iVR group interacted with a head-mounted device-based iVR simulation related to each topic (i.e. total three interventions), while the video group watched a pre-recorded video of the respective simulation on a laptop. Cognitive and affective data were collected through the following checkpoints: a pre-test before the first intervention, one topic-specific post-test immediately after each intervention, a final post-test towards the end of the course, and a longitudinal post-test deployed approximately 2 months after the course. Through a descriptive analysis, it was found that student performance on the knowledge tests increased considerably over time for the iVR group but remained unchanged for the video group. While no within- or between-group differences were noted for intrinsic motivation and self-efficacy measures, students in the iVR group enjoyed all the simulations, and perceived themselves to benefit from those simulations.","PeriodicalId":46691,"journal":{"name":"Research in Learning Technology","volume":"7 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2021-01-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"91258718","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Laura E. Reeves,Edward Bolton,Matthew Bulpitt,Alex Scott,Ian Tomey,Micah Gates,Robert A. Baldock
In recent years, development of new technologies designed to enhance user experience have accelerated, often being used in modern media such as in films and games. Specifically, immersive experiences, such as virtual reality (VR) and augmented reality (AR), have redefined how digital media can be delivered, encouraging us to interact with and explore our environment. Reciprocally, as the power of these technologies has advanced, the associated costs to implement them have decreased, making them more cost-effective and feasible to deliver in a variety of settings. Despite the cost reduction, several issues remain with accessibility due to the knowledge base required to generate, optimise and deliver three-dimensional (3D)-digital content in both AR and VR. Here, we sought to integrate an AR-based experience into a level-4 biochemistry module in order to support the delivery of university lectures on protein structure and function. Traditionally, this topic would comprise two-dimensional still images of complex 3D structures. By combining a breadth of subject-specific and technological expertise from across the university, we developed an AR-enhanced learning experience hosted on the Zapworks AR platform. AR enabled full illustration of the complexity of these 3D structures, while promoting collaboration through a shared user experience. Assessing the impact of the AR experience via a formative test and survey revealed that despite only a modest increase in test performance, students overwhelmingly reported positively on the engaging nature and interactivity of AR. Critically, expanding our repertoire of content delivery formats will support the forward-thinking blended learning environments adopted across the higher education sector.
{"title":"Use of augmented reality (AR) to aid bioscience education and enrich student experience","authors":"Laura E. Reeves,Edward Bolton,Matthew Bulpitt,Alex Scott,Ian Tomey,Micah Gates,Robert A. Baldock","doi":"10.25304/rlt.v29.2572","DOIUrl":"https://doi.org/10.25304/rlt.v29.2572","url":null,"abstract":"In recent years, development of new technologies designed to enhance user experience have accelerated, often being used in modern media such as in films and games. Specifically, immersive experiences, such as virtual reality (VR) and augmented reality (AR), have redefined how digital media can be delivered, encouraging us to interact with and explore our environment. Reciprocally, as the power of these technologies has advanced, the associated costs to implement them have decreased, making them more cost-effective and feasible to deliver in a variety of settings. Despite the cost reduction, several issues remain with accessibility due to the knowledge base required to generate, optimise and deliver three-dimensional (3D)-digital content in both AR and VR. Here, we sought to integrate an AR-based experience into a level-4 biochemistry module in order to support the delivery of university lectures on protein structure and function. Traditionally, this topic would comprise two-dimensional still images of complex 3D structures. By combining a breadth of subject-specific and technological expertise from across the university, we developed an AR-enhanced learning experience hosted on the Zapworks AR platform. AR enabled full illustration of the complexity of these 3D structures, while promoting collaboration through a shared user experience. Assessing the impact of the AR experience via a formative test and survey revealed that despite only a modest increase in test performance, students overwhelmingly reported positively on the engaging nature and interactivity of AR. Critically, expanding our repertoire of content delivery formats will support the forward-thinking blended learning environments adopted across the higher education sector.","PeriodicalId":46691,"journal":{"name":"Research in Learning Technology","volume":"90 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2021-01-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138533223","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The purpose of our correlational, quantitative study was to determine if time spent using digital media (i.e. text messaging and social media) influences students’ media writing self-perceptions (MWSPs). We measured students’ perceived writing ability using the MWSP scale and their time spent using digital media with the social networking time use scale (SONTUS). Correlations between students’ MWSP scores and SONTUS scores were statistically insignificant, suggesting that time spent using digital media does not negatively influence their perceived writing abilities. However, results from further analyses indicated that as students’ social media use increased, so did their ability to recognise the difference between writing for social media and writing for professional publications. We also found that the more students text the more they use social media and vice versa. We present directions for future research and practice.
{"title":"Measuring the correlation between digital media usage and students’ perceived writing ability: Are they related?","authors":"J. Parrella, H. Leggette, Tobin Redwine","doi":"10.25304/RLT.V29.2506","DOIUrl":"https://doi.org/10.25304/RLT.V29.2506","url":null,"abstract":"The purpose of our correlational, quantitative study was to determine if time spent using digital media (i.e. text messaging and social media) influences students’ media writing self-perceptions (MWSPs). We measured students’ perceived writing ability using the MWSP scale and their time spent using digital media with the social networking time use scale (SONTUS). Correlations between students’ MWSP scores and SONTUS scores were statistically insignificant, suggesting that time spent using digital media does not negatively influence their perceived writing abilities. However, results from further analyses indicated that as students’ social media use increased, so did their ability to recognise the difference between writing for social media and writing for professional publications. We also found that the more students text the more they use social media and vice versa. We present directions for future research and practice.","PeriodicalId":46691,"journal":{"name":"Research in Learning Technology","volume":"6 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2021-01-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87273488","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Digital technologies have been widely used in higher education (HE) for years, and the benefits have been recognised by both students and academics. Although many universities have developed their own digital technology strategies, many do not share either their vision or implementation strategies with staff. This research explores differences and similarities in the perception of digital technology by lecturers and academic managers. The purpose of this paper is to compare and contrast motivations, barriers and support systems required for the use and adoption of digital strategies. Interviews were conducted with a group of 20 lecturers and academic managers in the HE sector. The results reveal that both groups shared a common view that the introduction of digital technology can have a clear set of benefits to students; however, their motivations for introducing new approaches differed significantly. Whilst it is important not to generalise too much given the lack of homogeneity in the two groups and also the crossover between managers and lecturers, managers tended to take a performance goal-based approach to its introduction whilst lecturers were more learning goal orientated. This difference can cause significant difficulties in the implementation of new approaches to learning.
{"title":"Two groups separated by a shared goal: how academic managers and lecturers have embraced the introduction of digital technologies in UK Higher Education","authors":"Xue-wei Zhou, Melania Milecka-Forrest","doi":"10.25304/RLT.V29.2446","DOIUrl":"https://doi.org/10.25304/RLT.V29.2446","url":null,"abstract":"Digital technologies have been widely used in higher education (HE) for years, and the benefits have been recognised by both students and academics. Although many universities have developed their own digital technology strategies, many do not share either their vision or implementation strategies with staff. \u0000This research explores differences and similarities in the perception of digital technology by lecturers and academic managers. The purpose of this paper is to compare and contrast motivations, barriers and support systems required for the use and adoption of digital strategies. Interviews were conducted with a group of 20 lecturers and academic managers in the HE sector. The results reveal that both groups shared a common view that the introduction of digital technology can have a clear set of benefits to students; however, their motivations for introducing new approaches differed significantly. Whilst it is important not to generalise too much given the lack of homogeneity in the two groups and also the crossover between managers and lecturers, managers tended to take a performance goal-based approach to its introduction whilst lecturers were more learning goal orientated. This difference can cause significant difficulties in the implementation of new approaches to learning.","PeriodicalId":46691,"journal":{"name":"Research in Learning Technology","volume":"27 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2021-01-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83765474","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The research work reported in this article is part of a wider study aimed at developing a mobile application (app) for Science Education in primary school. For that, we designed a participatory framework proposal nested within the larger framework of Educational Design Research. This framework emerged from the authors’ need to organise the different phases of the mobile apps development and to arrange the expected products that arise from them. Our framework suggests a grounded, participatory and user-centred approach, relating literature contributions with data collected among future end-users. In this study, we exemplify the implementation of the proposed framework by presenting the outcomes of a specific moment of the preliminary research: students’ stories and drawing analysis, to define the mobile app concept. For that, we (1) present and describe the participatory framework proposal, (2) identify and characterise the research method adopted to define the mobile app concept, (3) reveal and analyse data collected from the implementation of a creative writing and drawing activity performed by fourth grade primary-school students, and (4) describe the implications of data analysis in the mobile app concept definition. Our intention is to share with other educational researchers an approach that can be used to develop educational mobile apps grounded on future end-users’ perceptions and ideas. We also aim to contribute to deepen the Educational Design Research apps by proposing and exemplifying the implementation of a framework that brings together researchers, students, teachers and experts.
{"title":"A participatory framework proposal for guiding researchers through an educational mobile app development","authors":"Rita Tavares, R. Vieira, Luís Pedro","doi":"10.25304/rlt.v28.2370","DOIUrl":"https://doi.org/10.25304/rlt.v28.2370","url":null,"abstract":"The research work reported in this article is part of a wider study aimed at developing a mobile application (app) for Science Education in primary school. For that, we designed a participatory framework proposal nested within the larger framework of Educational Design Research. This framework emerged from the authors’ need to organise the different phases of the mobile apps development and to arrange the expected products that arise from them. Our framework suggests a grounded, participatory and user-centred approach, relating literature contributions with data collected among future end-users. In this study, we exemplify the implementation of the proposed framework by presenting the outcomes of a specific moment of the preliminary research: students’ stories and drawing analysis, to define the mobile app concept. For that, we (1) present and describe the participatory framework proposal, (2) identify and characterise the research method adopted to define the mobile app concept, (3) reveal and analyse data collected from the implementation of a creative writing and drawing activity performed by fourth grade primary-school students, and (4) describe the implications of data analysis in the mobile app concept definition. Our intention is to share with other educational researchers an approach that can be used to develop educational mobile apps grounded on future end-users’ perceptions and ideas. We also aim to contribute to deepen the Educational Design Research apps by proposing and exemplifying the implementation of a framework that brings together researchers, students, teachers and experts.","PeriodicalId":46691,"journal":{"name":"Research in Learning Technology","volume":"111 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2020-12-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76934885","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：