Influence of pairing in examiner leniency and stringency ('hawk-dove effect') in part II of the European Diploma of Anaesthesiology and Intensive Care: A cohort study.

IF 4.2 2区 医学 Q1 ANESTHESIOLOGY European Journal of Anaesthesiology Pub Date : 2024-12-01 Epub Date: 2024-08-28 DOI:10.1097/EJA.0000000000002052
Stephen Sciberras, Markus Klimek, Bazil Ateleanu, Hugues Scipioni, Rodolphe Di Loreto, Joana Berger-Estilita
{"title":"Influence of pairing in examiner leniency and stringency ('hawk-dove effect') in part II of the European Diploma of Anaesthesiology and Intensive Care: A cohort study.","authors":"Stephen Sciberras, Markus Klimek, Bazil Ateleanu, Hugues Scipioni, Rodolphe Di Loreto, Joana Berger-Estilita","doi":"10.1097/EJA.0000000000002052","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>The European Diploma of Anaesthesiology and Intensive Care (EDAIC) Part II examination is a supranational examination for anaesthesiologists.</p><p><strong>Objectives: </strong>We explore the impact of examiner pairing on leniency and stringency, commonly referred to as the 'hawk-dove effect'. We investigate the potential variations in grading approaches, resulting from different examiner pairs and their implications for candidate performance.</p><p><strong>Design: </strong>Retrospective cohort, observational design.</p><p><strong>Setting: </strong>EDAIC Part II examination data from 2021 to 2023.</p><p><strong>Participants: </strong>Three hundred and twenty-five examiners across 122 EDAIC Part II single-day examination sessions.</p><p><strong>Interventions: </strong>We analysed the influence of examiner leniency and examiner pairing on candidate performance in the EDAIC Part II using many-facet Rasch modelling.</p><p><strong>Main outcome measures: </strong>The study's main outcome measure was determining a leniency score among the examiner population. The study also aimed to assess how examiner pairing influenced candidate performance, as measured by their scores in the examination.</p><p><strong>Results: </strong>During the study period, the number of examiners who participated in 2021, 2022 and 2023 were 253, 242 and 247, respectively. The median [IQR] single-day sessions attended were 7.0 [3 to 10]. The examination data revealed a mean leniency score of 0 (95% confidence interval (CI) -0.046 to 0.046), with the standard deviation being one-third that of the candidates' ability scores. There were 1424 different pairs of examiners, with most pairs (97%) having only a one-point difference in marking. The mean leniency score for the pair of examiners was -0.053 (95% CI -0.069 to -0.037).</p><p><strong>Conclusion: </strong>The variations in grading approaches associated with different pairings emphasise the potential for the 'hawk-dove effect' to influence candidate performance and outcomes. Understanding these variations can guide curriculum development, examiner training and coupling, ensuring a balanced and equitable assessment process.</p><p><strong>Trial registration: </strong>None.</p>","PeriodicalId":11920,"journal":{"name":"European Journal of Anaesthesiology","volume":" ","pages":"921-931"},"PeriodicalIF":4.2000,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11556864/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"European Journal of Anaesthesiology","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1097/EJA.0000000000002052","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/8/28 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"ANESTHESIOLOGY","Score":null,"Total":0}
引用次数: 0

Abstract

Background: The European Diploma of Anaesthesiology and Intensive Care (EDAIC) Part II examination is a supranational examination for anaesthesiologists.

Objectives: We explore the impact of examiner pairing on leniency and stringency, commonly referred to as the 'hawk-dove effect'. We investigate the potential variations in grading approaches, resulting from different examiner pairs and their implications for candidate performance.

Design: Retrospective cohort, observational design.

Setting: EDAIC Part II examination data from 2021 to 2023.

Participants: Three hundred and twenty-five examiners across 122 EDAIC Part II single-day examination sessions.

Interventions: We analysed the influence of examiner leniency and examiner pairing on candidate performance in the EDAIC Part II using many-facet Rasch modelling.

Main outcome measures: The study's main outcome measure was determining a leniency score among the examiner population. The study also aimed to assess how examiner pairing influenced candidate performance, as measured by their scores in the examination.

Results: During the study period, the number of examiners who participated in 2021, 2022 and 2023 were 253, 242 and 247, respectively. The median [IQR] single-day sessions attended were 7.0 [3 to 10]. The examination data revealed a mean leniency score of 0 (95% confidence interval (CI) -0.046 to 0.046), with the standard deviation being one-third that of the candidates' ability scores. There were 1424 different pairs of examiners, with most pairs (97%) having only a one-point difference in marking. The mean leniency score for the pair of examiners was -0.053 (95% CI -0.069 to -0.037).

Conclusion: The variations in grading approaches associated with different pairings emphasise the potential for the 'hawk-dove effect' to influence candidate performance and outcomes. Understanding these variations can guide curriculum development, examiner training and coupling, ensuring a balanced and equitable assessment process.

Trial registration: None.

查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
欧洲麻醉学和重症监护文凭第二部分考试中考官宽严搭配("鹰鸽效应")的影响:一项队列研究。
背景:欧洲麻醉学和重症监护文凭(EDAIC)第二部分考试是面向麻醉医师的超国家考试:我们探讨了考官配对对宽严程度的影响,即通常所说的 "鹰鸽效应"。我们研究了不同考官配对对评分方法的潜在影响及其对考生成绩的影响:设计:回顾性队列观察设计:背景:2021 年至 2023 年的 EDAIC 第二部分考试数据:325名考官参加了122场EDAIC第二部分考试:我们使用多面拉施模型分析了考官宽松度和考官配对对考生在EDAIC第二部分考试中成绩的影响:研究的主要结果测量是确定考官群体中的宽松度得分。研究还旨在评估考官配对对考生成绩的影响,考生成绩以考试成绩来衡量:在研究期间,参加 2021 年、2022 年和 2023 年考试的考官人数分别为 253 人、242 人和 247 人。参加考试次数的中位数为 7.0 次(3 至 10 次)。考试数据显示,宽松度的平均得分为 0 [95%置信区间(CI)-0.046 至 0.046],标准差为考生能力得分的三分之一。共有 1424 对不同的考官,其中大多数考官(97%)的评分仅相差一分。一对考官的平均宽松度为-0.053(95% CI -0.069至-0.037):不同配对考官在评分方法上的差异强调了 "鹰鸽效应 "影响考生成绩和结果的可能性。了解这些差异可以为课程开发、考官培训和搭配提供指导,确保评估过程的平衡和公平:无。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
CiteScore
6.90
自引率
11.10%
发文量
351
审稿时长
6-12 weeks
期刊介绍: The European Journal of Anaesthesiology (EJA) publishes original work of high scientific quality in the field of anaesthesiology, pain, emergency medicine and intensive care. Preference is given to experimental work or clinical observation in man, and to laboratory work of clinical relevance. The journal also publishes commissioned reviews by an authority, editorials, invited commentaries, special articles, pro and con debates, and short reports (correspondences, case reports, short reports of clinical studies).
期刊最新文献
A big little problem - postoperative nausea and vomiting incidences are too low! Is it time to add the letter E to the airway management guidelines? Is permissive hypercapnia really pneumoprotective? Reply to: importance of accounting for repeated measure designs when evaluating treatment effects at multiple postoperative days. Rethinking the utility of comparative studies between direct and video laryngoscopy in neonates and infants.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1