Pub Date : 2024-08-28DOI: 10.1007/s10459-024-10365-9
Adam G Gavarkovs, Rashmi A Kusurkar, Kulamakan Kulasegaram, Ryan Brydges
To design effective instruction, educators need to know what design strategies are generally effective and why these strategies work, based on the mechanisms through which they operate. Experimental comparison studies, which compare one instructional design against another, can generate much needed evidence in support of effective design strategies. However, experimental comparison studies are often not equipped to generate evidence regarding the mechanisms through which strategies operate. Therefore, simply conducting experimental comparison studies may not provide educators with all the information they need to design more effective instruction. To generate evidence for the what and the why of design strategies, we advocate for researchers to conduct experimental comparison studies that include mediation or moderation analyses, which can illuminate the mechanisms through which design strategies operate. The purpose of this article is to provide a conceptual overview of mediation and moderation analyses for researchers who conduct experimental comparison studies in instructional design. While these statistical techniques add complexity to study design and analysis, they hold great promise for providing educators with more powerful information upon which to base their instructional design decisions. Using two real-world examples from our own work, we describe the structure of mediation and moderation analyses, emphasizing the need to control for confounding even in the context of experimental studies. We also discuss the importance of using learning theories to help identify mediating or moderating variables to test.
{"title":"Going beyond the comparison: toward experimental instructional design research with impact.","authors":"Adam G Gavarkovs, Rashmi A Kusurkar, Kulamakan Kulasegaram, Ryan Brydges","doi":"10.1007/s10459-024-10365-9","DOIUrl":"https://doi.org/10.1007/s10459-024-10365-9","url":null,"abstract":"<p><p>To design effective instruction, educators need to know what design strategies are generally effective and why these strategies work, based on the mechanisms through which they operate. Experimental comparison studies, which compare one instructional design against another, can generate much needed evidence in support of effective design strategies. However, experimental comparison studies are often not equipped to generate evidence regarding the mechanisms through which strategies operate. Therefore, simply conducting experimental comparison studies may not provide educators with all the information they need to design more effective instruction. To generate evidence for the what and the why of design strategies, we advocate for researchers to conduct experimental comparison studies that include mediation or moderation analyses, which can illuminate the mechanisms through which design strategies operate. The purpose of this article is to provide a conceptual overview of mediation and moderation analyses for researchers who conduct experimental comparison studies in instructional design. While these statistical techniques add complexity to study design and analysis, they hold great promise for providing educators with more powerful information upon which to base their instructional design decisions. Using two real-world examples from our own work, we describe the structure of mediation and moderation analyses, emphasizing the need to control for confounding even in the context of experimental studies. We also discuss the importance of using learning theories to help identify mediating or moderating variables to test.</p>","PeriodicalId":50959,"journal":{"name":"Advances in Health Sciences Education","volume":" ","pages":""},"PeriodicalIF":3.0,"publicationDate":"2024-08-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142082503","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-26DOI: 10.1007/s10459-024-10362-y
Patricia O’Sullivan, Ayelet Kuper, Jennifer Cleland
This column is intended to address the kinds of knotty problems and dilemmas with which many scholars grapple in studying health professions education. In this article, the authors focus on how to help mentees take an analytic approach to improve their mixed methods work. Mixed methods research has increased in popularity and with that comes both strengths and weaknesses in these studies. We suggest key elements to look for when reading a mixed methods research paper. We also provide guidance around weaknesses we have noticed in reporting.
{"title":"Avoiding common pitfalls in mixed methods research?","authors":"Patricia O’Sullivan, Ayelet Kuper, Jennifer Cleland","doi":"10.1007/s10459-024-10362-y","DOIUrl":"10.1007/s10459-024-10362-y","url":null,"abstract":"<div><p>This column is intended to address the kinds of knotty problems and dilemmas with which many scholars grapple in studying health professions education. In this article, the authors focus on how to help mentees take an analytic approach to improve their mixed methods work. Mixed methods research has increased in popularity and with that comes both strengths and weaknesses in these studies. We suggest key elements to look for when reading a mixed methods research paper. We also provide guidance around weaknesses we have noticed in reporting.</p></div>","PeriodicalId":50959,"journal":{"name":"Advances in Health Sciences Education","volume":"29 4","pages":"1071 - 1073"},"PeriodicalIF":3.0,"publicationDate":"2024-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142057142","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-26DOI: 10.1007/s10459-024-10358-8
Simon Kitto, H L Michelle Chiang, Olivia Ng, Jennifer Cleland
There is a long-standing lack of learner satisfaction with quality and quantity of feedback in health professions education (HPE) and training. To address this, university and training programmes are increasingly using technological advancements and data analytic tools to provide feedback. One such educational technology is the Learning Analytic Dashboard (LAD), which holds the promise of a comprehensive view of student performance via partial or fully automated feedback delivered to learners in real time. The possibility of displaying performance data visually, on a single platform, so users can access and process feedback efficiently and constantly, and use this to improve their performance, is very attractive to users, educators and institutions. However, the mainstream literature tends to take an atheoretical and instrumentalist view of LADs, a view that uncritically celebrates the promise of LAD's capacity to provide a 'technical fix' to the 'wicked problem' of feedback in health professions education. This paper seeks to recast the discussion of LADs as something other than a benign material technology using the lenses of Miller and Rose's technologies of government and Barry's theory of Technological Societies, where such technical devices are also inherently agentic and political. An examination of the purpose, design and deployment of LADs from these theoretical perspectives can reveal how these educational devices shape and govern the HPE learner body in different ways, which in turn, may produce a myriad of unintended- and ironic- effects on the feedback process. In this Reflections article we wish to encourage health professions education scholars to examine the practices and consequences thereof of the ever-expanding use of LADs more deeply and with a sense of urgency.
{"title":"More, better feedback please: are learning analytics dashboards (LAD) the solution to a wicked problem?","authors":"Simon Kitto, H L Michelle Chiang, Olivia Ng, Jennifer Cleland","doi":"10.1007/s10459-024-10358-8","DOIUrl":"https://doi.org/10.1007/s10459-024-10358-8","url":null,"abstract":"<p><p>There is a long-standing lack of learner satisfaction with quality and quantity of feedback in health professions education (HPE) and training. To address this, university and training programmes are increasingly using technological advancements and data analytic tools to provide feedback. One such educational technology is the Learning Analytic Dashboard (LAD), which holds the promise of a comprehensive view of student performance via partial or fully automated feedback delivered to learners in real time. The possibility of displaying performance data visually, on a single platform, so users can access and process feedback efficiently and constantly, and use this to improve their performance, is very attractive to users, educators and institutions. However, the mainstream literature tends to take an atheoretical and instrumentalist view of LADs, a view that uncritically celebrates the promise of LAD's capacity to provide a 'technical fix' to the 'wicked problem' of feedback in health professions education. This paper seeks to recast the discussion of LADs as something other than a benign material technology using the lenses of Miller and Rose's technologies of government and Barry's theory of Technological Societies, where such technical devices are also inherently agentic and political. An examination of the purpose, design and deployment of LADs from these theoretical perspectives can reveal how these educational devices shape and govern the HPE learner body in different ways, which in turn, may produce a myriad of unintended- and ironic- effects on the feedback process. In this Reflections article we wish to encourage health professions education scholars to examine the practices and consequences thereof of the ever-expanding use of LADs more deeply and with a sense of urgency.</p>","PeriodicalId":50959,"journal":{"name":"Advances in Health Sciences Education","volume":" ","pages":""},"PeriodicalIF":3.0,"publicationDate":"2024-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142057143","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-05DOI: 10.1007/s10459-024-10361-z
Rose Hatala, Rachel H. Ellaway
In this editorial the editors consider the ideals and realities of high and low stakes assessments in clinical workplaces, the impact of these assessments on clinical workplace learning, and the clash between authenticity in assessment and authenticity in learning.
{"title":"Does authentic assessment undermine authentic learning?","authors":"Rose Hatala, Rachel H. Ellaway","doi":"10.1007/s10459-024-10361-z","DOIUrl":"10.1007/s10459-024-10361-z","url":null,"abstract":"<div><p>In this editorial the editors consider the ideals and realities of high and low stakes assessments in clinical workplaces, the impact of these assessments on clinical workplace learning, and the clash between authenticity in assessment and authenticity in learning.</p></div>","PeriodicalId":50959,"journal":{"name":"Advances in Health Sciences Education","volume":"29 4","pages":"1067 - 1070"},"PeriodicalIF":3.0,"publicationDate":"2024-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141890799","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-24DOI: 10.1007/s10459-024-10360-0
Gillian M Scanlan, Lisi Gordon, Kim Walker, Lindsey Pope
The transition into postgraduate medical training is complex, requiring an integration into the workplace, adjustment to new identities, and understanding of the social and organisational structure of healthcare. Studies suggest that social resources, including a sense of belonging, inclusivity from social groups, and having strong social identities can facilitate positive transitions. However, little is known about the role these resources play in junior doctors' transitions into the healthcare community. This study aimed to explore the implications of having access to social resources for junior doctors. This study undertook secondary analysis from a longitudinal qualitative study which followed 19 junior doctors (residents within two years of qualification) for nine months. Data were thematically analysed using an abductive approach, with the social identity resource and belongingness (SIRB) model as a conceptual lens to explore how social networks of support act as identity resources (IRs) for junior doctors as they experience transitions. The doctors narrated that having accessible IRs in the form of supportive workplace relationships enabled an integration and a sense of belonging into healthcare practice, supported the construction of new professional identities, and strengthened career intentions. Those with inaccessible IRs (i.e. poor workplace relationships) expressed a lack of belonging, and casted doubt on their identity as a doctor and their career intentions. Our study indicates that SIRB model would be beneficial for medical educators, supervisors, and managers to help them understand the importance and implications of having IRs within the workplace environment and the consequences of their accessibility for healthcare staff experiencing transitions.
医学研究生培训的过渡是复杂的,需要融入工作场所,适应新的身份,了解医疗保健的社会和组织结构。研究表明,社会资源(包括归属感、社会群体的包容性以及强烈的社会认同感)可以促进积极的过渡。然而,人们对这些资源在初级医生过渡到医疗群体中所起的作用知之甚少。本研究旨在探讨获得社会资源对初级医生的影响。本研究对一项纵向定性研究进行了二次分析,该研究对 19 名初级医生(获得资格后两年内的住院医师)进行了为期九个月的跟踪调查。研究采用归纳法对数据进行主题分析,以社会身份资源和归属感(SIRB)模型为概念视角,探讨社会支持网络如何作为初级医生的身份资源(IRs),帮助他们经历转型。医生们讲述说,以支持性工作场所关系为形式的可获得的社会认同资源能够让他们融入医疗实践并产生归属感,支持他们构建新的职业身份,并加强职业意向。而那些无法获得投资者关系(即糟糕的工作场所关系)的人则表示缺乏归属感,并对自己的医生身份和职业意向产生怀疑。我们的研究表明,SIRB 模型对医学教育者、主管和管理者很有帮助,可以帮助他们了解在工作场所环境中建立 IR 的重要性和意义,以及建立 IR 对经历转型的医护人员的影响。
{"title":"Enabling and inhibiting doctors transitions: introducing the social identity resource and belonginess model (SIRB).","authors":"Gillian M Scanlan, Lisi Gordon, Kim Walker, Lindsey Pope","doi":"10.1007/s10459-024-10360-0","DOIUrl":"https://doi.org/10.1007/s10459-024-10360-0","url":null,"abstract":"<p><p>The transition into postgraduate medical training is complex, requiring an integration into the workplace, adjustment to new identities, and understanding of the social and organisational structure of healthcare. Studies suggest that social resources, including a sense of belonging, inclusivity from social groups, and having strong social identities can facilitate positive transitions. However, little is known about the role these resources play in junior doctors' transitions into the healthcare community. This study aimed to explore the implications of having access to social resources for junior doctors. This study undertook secondary analysis from a longitudinal qualitative study which followed 19 junior doctors (residents within two years of qualification) for nine months. Data were thematically analysed using an abductive approach, with the social identity resource and belongingness (SIRB) model as a conceptual lens to explore how social networks of support act as identity resources (IRs) for junior doctors as they experience transitions. The doctors narrated that having accessible IRs in the form of supportive workplace relationships enabled an integration and a sense of belonging into healthcare practice, supported the construction of new professional identities, and strengthened career intentions. Those with inaccessible IRs (i.e. poor workplace relationships) expressed a lack of belonging, and casted doubt on their identity as a doctor and their career intentions. Our study indicates that SIRB model would be beneficial for medical educators, supervisors, and managers to help them understand the importance and implications of having IRs within the workplace environment and the consequences of their accessibility for healthcare staff experiencing transitions.</p>","PeriodicalId":50959,"journal":{"name":"Advances in Health Sciences Education","volume":" ","pages":""},"PeriodicalIF":3.0,"publicationDate":"2024-07-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141753333","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-23DOI: 10.1007/s10459-024-10357-9
Huiming Ding, Matt Homer
Summative assessments are often underused for feedback, despite them being rich with data of students' applied knowledge and clinical and professional skills. To better inform teaching and student support, this study aims to gain insights from summative assessments through profiling students' performance patterns and identify those students missing the basic knowledge and skills in medical specialities essential for their future career. We use Latent Profile Analysis to classify a senior undergraduate year group (n = 295) based on their performance in applied knowledge test (AKT) and OSCE, in which items and stations are pre-classified across five specialities (e.g. Acute and Critical Care, Paediatrics,…). Four distinct groups of students with increasing average performance levels in the AKT, and three such groups in the OSCE are identified. Overall, these two classifications are positively correlated. However, some students do well in one assessment format but not in the other. Importantly, in both the AKT and the OSCE there is a mixed group containing students who have met the required standard to pass, and those who have not. This suggests that a conception of a borderline group at the exam-level can be overly simplistic. There is little literature relating AKT and OSCE performance in this way, and the paper discusses how our analysis gives placement tutors key insights into providing tailored support for distinct student groups needing remediation. It also gives additional information to assessment writers about the performance and difficulty of their assessment items/stations, and to wider faculty about student overall performance and across specialities.
{"title":"Tailoring support following summative assessments: a latent profile analysis of student outcomes across five medical specialities.","authors":"Huiming Ding, Matt Homer","doi":"10.1007/s10459-024-10357-9","DOIUrl":"https://doi.org/10.1007/s10459-024-10357-9","url":null,"abstract":"<p><p>Summative assessments are often underused for feedback, despite them being rich with data of students' applied knowledge and clinical and professional skills. To better inform teaching and student support, this study aims to gain insights from summative assessments through profiling students' performance patterns and identify those students missing the basic knowledge and skills in medical specialities essential for their future career. We use Latent Profile Analysis to classify a senior undergraduate year group (n = 295) based on their performance in applied knowledge test (AKT) and OSCE, in which items and stations are pre-classified across five specialities (e.g. Acute and Critical Care, Paediatrics,…). Four distinct groups of students with increasing average performance levels in the AKT, and three such groups in the OSCE are identified. Overall, these two classifications are positively correlated. However, some students do well in one assessment format but not in the other. Importantly, in both the AKT and the OSCE there is a mixed group containing students who have met the required standard to pass, and those who have not. This suggests that a conception of a borderline group at the exam-level can be overly simplistic. There is little literature relating AKT and OSCE performance in this way, and the paper discusses how our analysis gives placement tutors key insights into providing tailored support for distinct student groups needing remediation. It also gives additional information to assessment writers about the performance and difficulty of their assessment items/stations, and to wider faculty about student overall performance and across specialities.</p>","PeriodicalId":50959,"journal":{"name":"Advances in Health Sciences Education","volume":" ","pages":""},"PeriodicalIF":3.0,"publicationDate":"2024-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141749609","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Increasing the proportion of culturally and linguistically diverse (CALD) health practitioners is identified as one strategy to address healthcare disparities that individuals from minority or under-represented backgrounds experience. However, professional and institutional cultures and structures are known to contribute to the challenges for CALD practitioners who work in dominant culture practice contexts. This scoping review used the theory of Legitimate Peripheral Participation to describe and interpret literature about the experiences of CALD health practitioners in view of informing strategies to increase their representation. A systematised search was conducted across four allied health, medicine and nursing databases. Following abstract and full text screening, articles which fit the inclusion criteria (n = 124) proceeded to data extraction. Categories relating to the experiences of practitioners were extracted, and three themes were identified that were subsequently theoretically interpreted: Discrimination, Consequences and Hierarchy. Discrimination functioned as a barrier to CALD practitioners being legitimised and able to participate equally in healthcare practice, retaining their position at the periphery of the practice community; Consequences reinforced this peripheral position and further impeded legitimation and participation; and Hierarchy was maintained through structures that reinforced and reproduced these barriers. The findings summarise how these barriers are reinforced through the intersections of professional and racial hierarchies, and highlight a need for strategies to address discrimination and structures that marginalise CALD practitioners' identity, practices and participation in their health professional communities.
{"title":"The experiences of culturally and linguistically diverse health practitioners in dominant culture practice: a scoping review.","authors":"Mikaela Harris, Timothea Lau-Bogaardt, Fathimath Shifaza, Stacie Attrill","doi":"10.1007/s10459-024-10359-7","DOIUrl":"https://doi.org/10.1007/s10459-024-10359-7","url":null,"abstract":"<p><p>Increasing the proportion of culturally and linguistically diverse (CALD) health practitioners is identified as one strategy to address healthcare disparities that individuals from minority or under-represented backgrounds experience. However, professional and institutional cultures and structures are known to contribute to the challenges for CALD practitioners who work in dominant culture practice contexts. This scoping review used the theory of Legitimate Peripheral Participation to describe and interpret literature about the experiences of CALD health practitioners in view of informing strategies to increase their representation. A systematised search was conducted across four allied health, medicine and nursing databases. Following abstract and full text screening, articles which fit the inclusion criteria (n = 124) proceeded to data extraction. Categories relating to the experiences of practitioners were extracted, and three themes were identified that were subsequently theoretically interpreted: Discrimination, Consequences and Hierarchy. Discrimination functioned as a barrier to CALD practitioners being legitimised and able to participate equally in healthcare practice, retaining their position at the periphery of the practice community; Consequences reinforced this peripheral position and further impeded legitimation and participation; and Hierarchy was maintained through structures that reinforced and reproduced these barriers. The findings summarise how these barriers are reinforced through the intersections of professional and racial hierarchies, and highlight a need for strategies to address discrimination and structures that marginalise CALD practitioners' identity, practices and participation in their health professional communities.</p>","PeriodicalId":50959,"journal":{"name":"Advances in Health Sciences Education","volume":" ","pages":""},"PeriodicalIF":3.0,"publicationDate":"2024-07-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141735572","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-15DOI: 10.1007/s10459-024-10348-w
Leonie Fleck, Dorothee Amelung, Anna Fuchs, Benjamin Mayer, Malvin Escher, Lena Listunova, Jobst-Hendrik Schultz, Andreas Möltner, Clara Schütte, Tim Wittenberg, Isabella Schneider, Sabine C Herpertz
Doctors' interactional competencies play a crucial role in patient satisfaction, well-being, and compliance. Accordingly, it is in medical schools' interest to select candidates with strong interactional abilities. While Multiple Mini Interviews (MMIs) provide a useful context to assess such abilities, the evaluation of candidate performance during MMIs is not always based on a solid theoretical framework. The newly developed selection procedure "Interactional Competencies - Medical Doctors (IC-MD)" uses an MMI circuit with five simulation patient scenarios and is rated based on the theoretically and empirically grounded construct of emotional availability. A first validation study with N = 70 first-semester medical students took place in 2021. In terms of convergent validity, IC-MD ratings showed strong correlations with simulation patients' satisfaction with the encounter (r =.57) but no association with emotional intelligence measures. IC-MD ratings were not related to high school performance or a cognitive student aptitude test, indicating divergent validity. Inter-rater reliability (ICC = 0.63) and generalizability (Eρ2 = 0.64) were satisfactory. The IC-MD proved to be fair regarding participants' age and gender. Participants with prior work experience in healthcare outperformed those without such experience. Participant acceptance of the procedure were good. The IC-MD is a promising selection procedure capable of assessing interactional competencies relevant to the medical setting. Measures of interactional competencies can complement the use of cognitive selection criteria in medical student admission. The predictive validity of the IC-MD needs to be addressed in future studies.
{"title":"Interactional competencies in medical student admission- what makes a \"good medical doctor\"?","authors":"Leonie Fleck, Dorothee Amelung, Anna Fuchs, Benjamin Mayer, Malvin Escher, Lena Listunova, Jobst-Hendrik Schultz, Andreas Möltner, Clara Schütte, Tim Wittenberg, Isabella Schneider, Sabine C Herpertz","doi":"10.1007/s10459-024-10348-w","DOIUrl":"https://doi.org/10.1007/s10459-024-10348-w","url":null,"abstract":"<p><p>Doctors' interactional competencies play a crucial role in patient satisfaction, well-being, and compliance. Accordingly, it is in medical schools' interest to select candidates with strong interactional abilities. While Multiple Mini Interviews (MMIs) provide a useful context to assess such abilities, the evaluation of candidate performance during MMIs is not always based on a solid theoretical framework. The newly developed selection procedure \"Interactional Competencies - Medical Doctors (IC-MD)\" uses an MMI circuit with five simulation patient scenarios and is rated based on the theoretically and empirically grounded construct of emotional availability. A first validation study with N = 70 first-semester medical students took place in 2021. In terms of convergent validity, IC-MD ratings showed strong correlations with simulation patients' satisfaction with the encounter (r =.57) but no association with emotional intelligence measures. IC-MD ratings were not related to high school performance or a cognitive student aptitude test, indicating divergent validity. Inter-rater reliability (ICC = 0.63) and generalizability (Eρ<sup>2</sup> = 0.64) were satisfactory. The IC-MD proved to be fair regarding participants' age and gender. Participants with prior work experience in healthcare outperformed those without such experience. Participant acceptance of the procedure were good. The IC-MD is a promising selection procedure capable of assessing interactional competencies relevant to the medical setting. Measures of interactional competencies can complement the use of cognitive selection criteria in medical student admission. The predictive validity of the IC-MD needs to be addressed in future studies.</p>","PeriodicalId":50959,"journal":{"name":"Advances in Health Sciences Education","volume":" ","pages":""},"PeriodicalIF":3.0,"publicationDate":"2024-07-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141617520","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-08DOI: 10.1007/s10459-024-10354-y
David Hope, David Kluth, Matthew Homer, Avril Dewar, Rikki Goddard-Fuller, Alan Jaap, Helen Cameron
Rasch modelling is a powerful tool for evaluating item performance, measuring drift in difficulty over time, and comparing students who sat assessments at different times or at different sites. Here, we use data from thirty UK medical schools to describe the benefits of Rasch modelling in quality assurance and the barriers to using it. Sixty "common content" multiple choice items were offered to all UK medical schools in 2016-17, and a further sixty in 2017-18, with five available in both years. Thirty medical schools participated, for sixty total datasets across two sessions, and 14,342 individual sittings. Schools selected items to embed in written assessment near the end of their programmes. We applied Rasch modelling to evaluate unidimensionality, model fit statistics and item quality, horizontal equating to compare performance across schools, and vertical equating to compare item performance across time. Of the sixty sittings, three provided non-unidimensional data, and eight violated goodness of fit measures. Item-level statistics identified potential improvements in item construction and provided quality assurance. Horizontal equating demonstrated large differences in scores across schools, while vertical equating showed item characteristics were stable across sessions. Rasch modelling provides significant advantages in model- and item- level reporting compared to classical approaches. However, the complexity of the analysis and the smaller number of educators familiar with Rasch must be addressed locally for a programme to benefit. Furthermore, due to the comparative novelty of Rasch modelling, there is greater ambiguity on how to proceed when a Rasch model identifies misfitting or problematic data.
{"title":"Exploring the use of Rasch modelling in \"common content\" items for multi-site and multi-year assessment.","authors":"David Hope, David Kluth, Matthew Homer, Avril Dewar, Rikki Goddard-Fuller, Alan Jaap, Helen Cameron","doi":"10.1007/s10459-024-10354-y","DOIUrl":"https://doi.org/10.1007/s10459-024-10354-y","url":null,"abstract":"<p><p>Rasch modelling is a powerful tool for evaluating item performance, measuring drift in difficulty over time, and comparing students who sat assessments at different times or at different sites. Here, we use data from thirty UK medical schools to describe the benefits of Rasch modelling in quality assurance and the barriers to using it. Sixty \"common content\" multiple choice items were offered to all UK medical schools in 2016-17, and a further sixty in 2017-18, with five available in both years. Thirty medical schools participated, for sixty total datasets across two sessions, and 14,342 individual sittings. Schools selected items to embed in written assessment near the end of their programmes. We applied Rasch modelling to evaluate unidimensionality, model fit statistics and item quality, horizontal equating to compare performance across schools, and vertical equating to compare item performance across time. Of the sixty sittings, three provided non-unidimensional data, and eight violated goodness of fit measures. Item-level statistics identified potential improvements in item construction and provided quality assurance. Horizontal equating demonstrated large differences in scores across schools, while vertical equating showed item characteristics were stable across sessions. Rasch modelling provides significant advantages in model- and item- level reporting compared to classical approaches. However, the complexity of the analysis and the smaller number of educators familiar with Rasch must be addressed locally for a programme to benefit. Furthermore, due to the comparative novelty of Rasch modelling, there is greater ambiguity on how to proceed when a Rasch model identifies misfitting or problematic data.</p>","PeriodicalId":50959,"journal":{"name":"Advances in Health Sciences Education","volume":" ","pages":""},"PeriodicalIF":3.0,"publicationDate":"2024-07-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141560287","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-07DOI: 10.1007/s10459-024-10356-w
Diego Suarez, Adam Sawatsky
One of the main goals of medical education is to facilitate the development of a professional identity. As part of this effort, trainees are exposed to the values and cultures of the profession in a process known as socialization. Learners must then negotiate incongruent aspects between their preexisting identities and nascent professional identities. Individuals from historically underrepresented ethnic groups often undergo more significant changes due to their values and culture not being as prevalent within the dominant ideology of medicine. This transformative process can lead to identity dissonance and manifest as an internal discomfort resulting from perceived contradictions between one's existing identity and the required professional identity. Identity dissonance may be traumatic and pose a threat to the academic performance and professional integration of trainees. These detrimental effects harm the medical workforce by depriving it of a group crucial in addressing health inequities. Educators tasked with facilitating the professional development of learners must consider their implicit expectations about professionalism, explore the distinct challenges experienced by individuals from underrepresented backgrounds in their professional development, and work to develop strategies to help trainees navigate identity dissonance. Subjectification, an education philosophy that focuses on compelling individuals to explore the new possibilities and responsibilities imparted to them by their education, provides a theoretical framework to help educators guide learners through identity dissonance.
{"title":"Navigating identity dissonance: subjectification to balance socialization.","authors":"Diego Suarez, Adam Sawatsky","doi":"10.1007/s10459-024-10356-w","DOIUrl":"https://doi.org/10.1007/s10459-024-10356-w","url":null,"abstract":"<p><p>One of the main goals of medical education is to facilitate the development of a professional identity. As part of this effort, trainees are exposed to the values and cultures of the profession in a process known as socialization. Learners must then negotiate incongruent aspects between their preexisting identities and nascent professional identities. Individuals from historically underrepresented ethnic groups often undergo more significant changes due to their values and culture not being as prevalent within the dominant ideology of medicine. This transformative process can lead to identity dissonance and manifest as an internal discomfort resulting from perceived contradictions between one's existing identity and the required professional identity. Identity dissonance may be traumatic and pose a threat to the academic performance and professional integration of trainees. These detrimental effects harm the medical workforce by depriving it of a group crucial in addressing health inequities. Educators tasked with facilitating the professional development of learners must consider their implicit expectations about professionalism, explore the distinct challenges experienced by individuals from underrepresented backgrounds in their professional development, and work to develop strategies to help trainees navigate identity dissonance. Subjectification, an education philosophy that focuses on compelling individuals to explore the new possibilities and responsibilities imparted to them by their education, provides a theoretical framework to help educators guide learners through identity dissonance.</p>","PeriodicalId":50959,"journal":{"name":"Advances in Health Sciences Education","volume":" ","pages":""},"PeriodicalIF":3.0,"publicationDate":"2024-07-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141545428","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}