Georgina C. Stephens, Gabrielle Brand, Sharon Yahalom
<p>The construction of validity arguments for assessments in health professions education (HPE) has been likened to a lawyer preparing for and presenting a case.<span><sup>1</sup></span> Like a lawyer curates a brief of evidence with the aim of convincing a judge or jury to make a particular decision about their client, so are health professions educators required to provide validity evidence that supports a defensible decision about a student being assessed.<span><sup>1</sup></span> Kane's argument-based validity framework,<span><sup>2</sup></span> now expanded by scholars in language testing and assessment (LTA), addresses challenges of prior conceptualisations of validity by providing a staged approach to building a validation argument according to ‘inferences’. Whereas Kane's original four inference model commences with scoring,<span><sup>2</sup></span> the expanded seven inference model as described in this issue by Dai et al.<span><sup>3</sup></span> in LTA commences with a domain description.</p><p>The goal of the domain description is to ensure that the ‘selection, design and delivery of the test tasks takes the relevant target domain into account’.<span><sup>3</sup></span> Described sources of backing for this inference include interviews or surveys of domain insiders. Starting with a domain description should provide a solid foundation for subsequent inferences made about the assessment but also begs the question of who are considered domain insiders. And whether insights from diverse groups with insider perspectives can together build a more robust and nuanced validity argument. Returning to the analogy of the lawyer's decision-making processes, there may be multiple witnesses with evidence to share, but which witnesses are called upon to provide evidence in court? Or alternatively, which witnesses are not selected out of concern that their differing perspectives may threaten the lawyer's plan for the case?</p><p>Domain insiders could be considered ‘expert witnesses’, that is, those with subject matter expertise typically built through education and professional experience, such as health professions educators with clinical and/or pedagogical expertise. While subject matter expertise is important to understanding whether assessment tasks sufficiently reflect the domain being assessed, potential differences between expert and novice (i.e., student) understandings of a domain could disrupt a validity argument. Consider assessments of uncertainty tolerance (UT): Commonly used UT scales intended to measure UT in healthcare contexts engaged expertise during scale development in the form of interviews with health professionals, reviews of construct literature and consultation with medical educator peers.<span><sup>4</sup></span> One UT scale has been used by the Association of American Medical Colleges as part of routine matriculation and graduation surveys of medical students, with the intent that the results inform medical school programma
{"title":"Whose voices are heard in health professions education validity arguments?","authors":"Georgina C. Stephens, Gabrielle Brand, Sharon Yahalom","doi":"10.1111/medu.15528","DOIUrl":"10.1111/medu.15528","url":null,"abstract":"<p>The construction of validity arguments for assessments in health professions education (HPE) has been likened to a lawyer preparing for and presenting a case.<span><sup>1</sup></span> Like a lawyer curates a brief of evidence with the aim of convincing a judge or jury to make a particular decision about their client, so are health professions educators required to provide validity evidence that supports a defensible decision about a student being assessed.<span><sup>1</sup></span> Kane's argument-based validity framework,<span><sup>2</sup></span> now expanded by scholars in language testing and assessment (LTA), addresses challenges of prior conceptualisations of validity by providing a staged approach to building a validation argument according to ‘inferences’. Whereas Kane's original four inference model commences with scoring,<span><sup>2</sup></span> the expanded seven inference model as described in this issue by Dai et al.<span><sup>3</sup></span> in LTA commences with a domain description.</p><p>The goal of the domain description is to ensure that the ‘selection, design and delivery of the test tasks takes the relevant target domain into account’.<span><sup>3</sup></span> Described sources of backing for this inference include interviews or surveys of domain insiders. Starting with a domain description should provide a solid foundation for subsequent inferences made about the assessment but also begs the question of who are considered domain insiders. And whether insights from diverse groups with insider perspectives can together build a more robust and nuanced validity argument. Returning to the analogy of the lawyer's decision-making processes, there may be multiple witnesses with evidence to share, but which witnesses are called upon to provide evidence in court? Or alternatively, which witnesses are not selected out of concern that their differing perspectives may threaten the lawyer's plan for the case?</p><p>Domain insiders could be considered ‘expert witnesses’, that is, those with subject matter expertise typically built through education and professional experience, such as health professions educators with clinical and/or pedagogical expertise. While subject matter expertise is important to understanding whether assessment tasks sufficiently reflect the domain being assessed, potential differences between expert and novice (i.e., student) understandings of a domain could disrupt a validity argument. Consider assessments of uncertainty tolerance (UT): Commonly used UT scales intended to measure UT in healthcare contexts engaged expertise during scale development in the form of interviews with health professionals, reviews of construct literature and consultation with medical educator peers.<span><sup>4</sup></span> One UT scale has been used by the Association of American Medical Colleges as part of routine matriculation and graduation surveys of medical students, with the intent that the results inform medical school programma","PeriodicalId":18370,"journal":{"name":"Medical Education","volume":"58 12","pages":"1429-1432"},"PeriodicalIF":4.9,"publicationDate":"2024-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/medu.15528","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142216837","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
<p>Improving equity, diversity and inclusion (EDI) within health profession education is a global priority. In this issue of <i>Medical Education</i>, Lam et al.<span><sup>1</sup></span> review EDI literature in postgraduate medical education (PGME) focusing on how discrimination is conceptualised and addressed. They find that while learner representation and gender inequities are recognised, systemic racism and power dynamics are often overlooked, limiting the effectiveness of current reforms. They emphasise the need for critical, intersectional approaches and re-examining educational processes to truly advance equity in learning environments for marginalised groups.</p><p>One educational process to re-examine is workplace-based assessment (WBA), a significant challenge in advancing EDI within PGME. Since the introduction of Competency-Based Medical Education (CBME), WBA has been increasingly adopted for competence assessments and workplace learning.<span><sup>2</sup></span> WBA is inherently subjective, influenced by individual judgement and existing workplace structures and hierarchies and thus susceptible to racism and inequity through implicit and explicit biases in direct observation, performance interpretations, coaching and feedback and supervisor–trainee power dynamics. The lack of diverse perspectives and inadequate supervisor training on EDI principles can exacerbate inequities in assessments, disadvantaging marginalised trainees. Yet, CBME principles can help advance EDI by centring the trainee and providing individualised resources to navigate barriers and ensure fair learning and assessment opportunities.<span><sup>3</sup></span></p><p>One approach that has gained significant attention over recent years in aligning WBA with CBME is the use of Entrustable Professional Activities (EPAs). EPAs are units of professional practice, defined as tasks entrusted to trainees for unsupervised execution once they demonstrate sufficient competence.<span><sup>4</sup></span> Assessment through EPAs involves entrustment decision-making, which requires evaluating a trainee's competence and determining their readiness to take on more responsibility or autonomy with less supervision. Whether EPAs can reduce bias in WBA is a complex and multifaceted question. While entrustment decision-making offers a new rating approach, it is not immune to bias and could potentially introduce new biases related to how supervisors conceptualise or experience trust. For instance, studies comparing traditional proficiency scales with entrustment–supervision scales have shown that the latter offer more reliable performance estimates with less inter-rater variability, suggesting that entrustment could be less influenced by performance-irrelevant trainee characteristics.<span><sup>5, 6</sup></span> However, considerable variability in supervisors' willingness to grant trust has been reported.<span><sup>7</sup></span></p><p>One advantage offered by the EPA model is its expli
{"title":"Equity, diversity, and inclusion in entrustable professional activities based assessment","authors":"Marije P. Hennus, H. Carrie Chen","doi":"10.1111/medu.15526","DOIUrl":"10.1111/medu.15526","url":null,"abstract":"<p>Improving equity, diversity and inclusion (EDI) within health profession education is a global priority. In this issue of <i>Medical Education</i>, Lam et al.<span><sup>1</sup></span> review EDI literature in postgraduate medical education (PGME) focusing on how discrimination is conceptualised and addressed. They find that while learner representation and gender inequities are recognised, systemic racism and power dynamics are often overlooked, limiting the effectiveness of current reforms. They emphasise the need for critical, intersectional approaches and re-examining educational processes to truly advance equity in learning environments for marginalised groups.</p><p>One educational process to re-examine is workplace-based assessment (WBA), a significant challenge in advancing EDI within PGME. Since the introduction of Competency-Based Medical Education (CBME), WBA has been increasingly adopted for competence assessments and workplace learning.<span><sup>2</sup></span> WBA is inherently subjective, influenced by individual judgement and existing workplace structures and hierarchies and thus susceptible to racism and inequity through implicit and explicit biases in direct observation, performance interpretations, coaching and feedback and supervisor–trainee power dynamics. The lack of diverse perspectives and inadequate supervisor training on EDI principles can exacerbate inequities in assessments, disadvantaging marginalised trainees. Yet, CBME principles can help advance EDI by centring the trainee and providing individualised resources to navigate barriers and ensure fair learning and assessment opportunities.<span><sup>3</sup></span></p><p>One approach that has gained significant attention over recent years in aligning WBA with CBME is the use of Entrustable Professional Activities (EPAs). EPAs are units of professional practice, defined as tasks entrusted to trainees for unsupervised execution once they demonstrate sufficient competence.<span><sup>4</sup></span> Assessment through EPAs involves entrustment decision-making, which requires evaluating a trainee's competence and determining their readiness to take on more responsibility or autonomy with less supervision. Whether EPAs can reduce bias in WBA is a complex and multifaceted question. While entrustment decision-making offers a new rating approach, it is not immune to bias and could potentially introduce new biases related to how supervisors conceptualise or experience trust. For instance, studies comparing traditional proficiency scales with entrustment–supervision scales have shown that the latter offer more reliable performance estimates with less inter-rater variability, suggesting that entrustment could be less influenced by performance-irrelevant trainee characteristics.<span><sup>5, 6</sup></span> However, considerable variability in supervisors' willingness to grant trust has been reported.<span><sup>7</sup></span></p><p>One advantage offered by the EPA model is its expli","PeriodicalId":18370,"journal":{"name":"Medical Education","volume":"58 12","pages":"1426-1428"},"PeriodicalIF":4.9,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/medu.15526","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142120214","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
<p>‘<i>Dr. [A] was late one minute, one time. But to be fair, it didn't negatively impact my learning</i>’. This is a comment from a learner on my colleague's anonymised student evaluation of teaching form. The consideration of whether this might be a ‘fair’ comment is reflective of a larger conversation about the utility and value of online, anonymised feedback forms. As such tensions garner a heightened focus in health professions education (HPE) specifically,<span><sup>1</sup></span> the article by Jenq et al.<span><sup>2</sup></span> is particularly timely.</p><p>Their study offers an important contribution to this area of scholarly work as it compares both givers and receivers of feedback, offering a useful juxtaposition of challenges while most studies focus on one group or the other. Viewing both sides of the same coin in one context helps to illuminate the perspectives of multiple players in the clinical learning environment on the topic of upward feedback. Another distinctive aspect of their work is that the authors examine upward feedback in a Taiwanese culture, with speculation that learners and faculty in Asian cultures are more sensitive to power dynamics than those in Westernised cultures, thereby making in-person feedback conversations more difficult to hold successfully.</p><p>The authors found that the influence of medical hierarchy was prevalent and posed a barrier to giving and receiving feedback across all participant groups which included medical students, residents, nurses and clinical educators. For example, nurses were often prevented by their departments from giving feedback to clinical educators and felt as though their feedback would be ignored by clinicians. Learners were reluctant to provide constructive feedback for fear of retribution and concern that their feedback would also be dismissed due to their low status. Educators were more likely to accept feedback from more senior clinicians than from their learners or peers. Further, clinicians took constructive feedback particularly hard because they felt reputational consequences in front of others. Because of these strong and persistent power and social dynamics, the authors leave us with three implications for their work, each of which are deserving of their own respective ‘unpacking’.</p><p>The authors first recommend a shift to anonymous upward feedback processes with the hope that learners can give, and teachers can receive, constructive feedback without the threat of hierarchical issues that come with face to face conversations. It is important to keep in mind, however, that anonymous feedback comes with its own challenges. For example, feedback should be timely,<span><sup>3</sup></span> but to maintain learner anonymity, feedback has to be held until long after the teaching encounter. Anonymous feedback mechanisms violate other foundational feedback tenets as well, by not being situated in an educational alliance, and with no opportunities for discussion and re
{"title":"The pitfalls and perils of anonymous learner feedback","authors":"Katherine M. Wisener","doi":"10.1111/medu.15487","DOIUrl":"10.1111/medu.15487","url":null,"abstract":"<p>‘<i>Dr. [A] was late one minute, one time. But to be fair, it didn't negatively impact my learning</i>’. This is a comment from a learner on my colleague's anonymised student evaluation of teaching form. The consideration of whether this might be a ‘fair’ comment is reflective of a larger conversation about the utility and value of online, anonymised feedback forms. As such tensions garner a heightened focus in health professions education (HPE) specifically,<span><sup>1</sup></span> the article by Jenq et al.<span><sup>2</sup></span> is particularly timely.</p><p>Their study offers an important contribution to this area of scholarly work as it compares both givers and receivers of feedback, offering a useful juxtaposition of challenges while most studies focus on one group or the other. Viewing both sides of the same coin in one context helps to illuminate the perspectives of multiple players in the clinical learning environment on the topic of upward feedback. Another distinctive aspect of their work is that the authors examine upward feedback in a Taiwanese culture, with speculation that learners and faculty in Asian cultures are more sensitive to power dynamics than those in Westernised cultures, thereby making in-person feedback conversations more difficult to hold successfully.</p><p>The authors found that the influence of medical hierarchy was prevalent and posed a barrier to giving and receiving feedback across all participant groups which included medical students, residents, nurses and clinical educators. For example, nurses were often prevented by their departments from giving feedback to clinical educators and felt as though their feedback would be ignored by clinicians. Learners were reluctant to provide constructive feedback for fear of retribution and concern that their feedback would also be dismissed due to their low status. Educators were more likely to accept feedback from more senior clinicians than from their learners or peers. Further, clinicians took constructive feedback particularly hard because they felt reputational consequences in front of others. Because of these strong and persistent power and social dynamics, the authors leave us with three implications for their work, each of which are deserving of their own respective ‘unpacking’.</p><p>The authors first recommend a shift to anonymous upward feedback processes with the hope that learners can give, and teachers can receive, constructive feedback without the threat of hierarchical issues that come with face to face conversations. It is important to keep in mind, however, that anonymous feedback comes with its own challenges. For example, feedback should be timely,<span><sup>3</sup></span> but to maintain learner anonymity, feedback has to be held until long after the teaching encounter. Anonymous feedback mechanisms violate other foundational feedback tenets as well, by not being situated in an educational alliance, and with no opportunities for discussion and re","PeriodicalId":18370,"journal":{"name":"Medical Education","volume":"58 12","pages":"1436-1438"},"PeriodicalIF":4.9,"publicationDate":"2024-08-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/medu.15487","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141897770","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
<p>Immortalised as a foundation of medical ethics, beneficence is often described as not taking advantage of another's vulnerability, particularly in hierarchical scenarios, or leader–follower relationships.<span><sup>1</sup></span> It is commonly referred to in relation to the doctor–patient duty of care and the inherent power imbalance of this relationship. However, to understand beneficence, one must also understand the concept of vulnerability; the shared human experience of our susceptibilities<span><sup>2</sup></span> and the likelihood that others will be less sensitive to our interests.<span><sup>3</sup></span> In this issue of Medical Education, Nichol et al. explore the paradox of vulnerability through the experiences of resident doctors along the continuum of intrapersonal, interpersonal, and institutional dynamics.<span><sup>4</sup></span> In doing so, they reveal resident doctors who are themselves vulnerable in the hierarchy and leader–follower relationships that construct the cornerstone of the medical institution.</p><p>Historically, Western medicine embodied the valorised, paternalistic, and infallible medical doctor, a portrayal which lingers today and eradicates the innate human quality of vulnerability. In modern-day experiences of resident doctors, this manifests a mutual exclusion between showing vulnerability and upholding professional competency.<span><sup>4</sup></span> The authors reveal that residents expressing vulnerability were seen as undesirable by superiors and often met with humiliation, mistreatment or an erosion of professional relationships that stifled learning opportunities and career development.<span><sup>4</sup></span> Thus, resident doctors 'eviscerated' themselves for ‘the culture of medicine’.<span><sup>4</sup></span> Furthermore, residents discussed a conflation between vulnerability and weakness, which imposed a subsequent barrier to leadership positions.<span><sup>4</sup></span> This is deeply disappointing, given that decades of evidence necessitates vulnerability in leader–follower trust,<span><sup>1</sup></span> and implicates it within the four primary characteristics of authentic leadership itself: self-awareness, balanced processing, relational transparency and an internal moral perspective.<span><sup>5</sup></span> For an evidence-based profession, here lies the irony that core human attributes are negatively stigmatised among overwhelming evidence that both leaders and clinical preceptors who themselves model vulnerability create a positive symbiosis that strengthens trusting relationships,<span><sup>1</sup></span> institutional culture, learning and the quality of patient care.<span><sup>4, 6-8</sup></span></p><p>The stigma on vulnerability in medicine is met with well-known systemic failings through the form of pervasive hierarchies, bullying, untenable working hours<span><sup>9</sup></span> and constant evaluation by superiors.<span><sup>4</sup></span> In such environments, to even consid
{"title":"Epistemic injustice: The hidden vulnerability of medicine","authors":"Joelle Winderbaum","doi":"10.1111/medu.15478","DOIUrl":"10.1111/medu.15478","url":null,"abstract":"<p>Immortalised as a foundation of medical ethics, beneficence is often described as not taking advantage of another's vulnerability, particularly in hierarchical scenarios, or leader–follower relationships.<span><sup>1</sup></span> It is commonly referred to in relation to the doctor–patient duty of care and the inherent power imbalance of this relationship. However, to understand beneficence, one must also understand the concept of vulnerability; the shared human experience of our susceptibilities<span><sup>2</sup></span> and the likelihood that others will be less sensitive to our interests.<span><sup>3</sup></span> In this issue of Medical Education, Nichol et al. explore the paradox of vulnerability through the experiences of resident doctors along the continuum of intrapersonal, interpersonal, and institutional dynamics.<span><sup>4</sup></span> In doing so, they reveal resident doctors who are themselves vulnerable in the hierarchy and leader–follower relationships that construct the cornerstone of the medical institution.</p><p>Historically, Western medicine embodied the valorised, paternalistic, and infallible medical doctor, a portrayal which lingers today and eradicates the innate human quality of vulnerability. In modern-day experiences of resident doctors, this manifests a mutual exclusion between showing vulnerability and upholding professional competency.<span><sup>4</sup></span> The authors reveal that residents expressing vulnerability were seen as undesirable by superiors and often met with humiliation, mistreatment or an erosion of professional relationships that stifled learning opportunities and career development.<span><sup>4</sup></span> Thus, resident doctors 'eviscerated' themselves for ‘the culture of medicine’.<span><sup>4</sup></span> Furthermore, residents discussed a conflation between vulnerability and weakness, which imposed a subsequent barrier to leadership positions.<span><sup>4</sup></span> This is deeply disappointing, given that decades of evidence necessitates vulnerability in leader–follower trust,<span><sup>1</sup></span> and implicates it within the four primary characteristics of authentic leadership itself: self-awareness, balanced processing, relational transparency and an internal moral perspective.<span><sup>5</sup></span> For an evidence-based profession, here lies the irony that core human attributes are negatively stigmatised among overwhelming evidence that both leaders and clinical preceptors who themselves model vulnerability create a positive symbiosis that strengthens trusting relationships,<span><sup>1</sup></span> institutional culture, learning and the quality of patient care.<span><sup>4, 6-8</sup></span></p><p>The stigma on vulnerability in medicine is met with well-known systemic failings through the form of pervasive hierarchies, bullying, untenable working hours<span><sup>9</sup></span> and constant evaluation by superiors.<span><sup>4</sup></span> In such environments, to even consid","PeriodicalId":18370,"journal":{"name":"Medical Education","volume":"58 12","pages":"1433-1435"},"PeriodicalIF":4.9,"publicationDate":"2024-07-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/medu.15478","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141792808","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}