BMJ Evidence-Based Medicine最新文献

{"title":"Post-publication peer review and the identification of methodological and reporting issues in COVID-19 trials: a qualitative study.","authors":"Mauricia Davidson, Christoffer Bruun Korfitsen, Carolina Riveros, Anna Chaimani, Isabelle Boutron","doi":"10.1136/bmjebm-2024-113068","DOIUrl":"10.1136/bmjebm-2024-113068","url":null,"abstract":"<p><strong>Objectives: </strong>We aimed to determine to what extent systematic reviewers and post-preprint and post-publication peer review identified methodological and reporting issues in COVID-19 trials that could be easily resolved by the authors.</p><p><strong>Design: </strong>Qualitative study.</p><p><strong>Data sources: </strong>COVID-NMA living systematic review (covid-nma.com), PubPeer, medRxiv, Research Square, SSRN.</p><p><strong>Methods: </strong>We considered randomised controlled trials (RCTs) in COVID-NMA that evaluated pharmacological treatments for COVID-19 and retrieved systematic reviewers' assessments of the risk of bias and outcome reporting bias. We also searched for commentary data on PubPeer and preprint servers up to 6 November 2023. We employed qualitative content analysis to develop themes and domains of methodological and reporting issues identified by commenters.</p><p><strong>Results: </strong>We identified 500 eligible RCTs. Systematic reviewers identified methodological and reporting issues in 446 (89%) RCTs. In 391 (78%) RCTs, the issues could be easily resolved by the trial authors; issues included incomplete reporting (49%), selection of the reported results (52%) and no access to the pre-specified plan (25%). Alternatively, 74 (15%) RCTs had received at least one comment on PubPeer or preprint servers, totalling 348 comments. In 46 (9%) RCTs, the issues identified by post-preprint and post-publication peer review comments could be easily resolved by the trial authors; the issues were related to incomplete reporting (6%), errors (5%), statistical analysis (3%), inconsistent reporting of methods and analyses (2%), spin (2%), selection of the reported results (1%) and no access to the raw data/pre-specified plan (1%).</p><p><strong>Conclusions: </strong>Without changing their process, systematic reviewers identified issues in most RCTs that could be easily resolved by the trial authors; however, the lack of an established author feedback mechanism represents a wasted opportunity for facilitating improvement and enhancing the overall manuscript quality. On the other hand, despite the existing feedback loop to authors present in post-publication peer review, it demonstrated limited effectiveness in identifying methodological and reporting issues.</p>","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":" ","pages":""},"PeriodicalIF":9.0,"publicationDate":"2025-03-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143466997","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"What makes a 'good' decision with artificial intelligence? A grounded theory study in paediatric care.","authors":"Melissa D McCradden, Kelly Thai, Azadeh Assadi, Sana Tonekaboni, Ian Stedman, Shalmali Joshi, Minfan Zhang, Fanny Chevalier, Anna Goldenberg","doi":"10.1136/bmjebm-2024-112919","DOIUrl":"https://doi.org/10.1136/bmjebm-2024-112919","url":null,"abstract":"&lt;p&gt;&lt;strong&gt;Objective: &lt;/strong&gt;To develop a framework for good clinical decision-making using machine learning (ML) models for interventional, patient-level decisions.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Design: &lt;/strong&gt;Grounded theory qualitative interview study.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Setting: &lt;/strong&gt;Primarily single-site at a major urban academic paediatric hospital, with external sampling.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Participants: &lt;/strong&gt;Sixteen participants representing physicians (n=10), nursing (n=3), respiratory therapists (n=2) and an ML specialist (n=1) with experience working in acute care environments were identified through purposive sampling. Individuals were recruited to represent a spectrum of ML knowledge (three expert, four knowledgeable and nine non-expert) and years of experience (median=12.9 years postgraduation). Recruitment proceeded through snowball sampling, with individuals approached to represent a diversity of fields, levels of experience and attitudes towards artificial intelligence (AI)/ML. A member check step and consultation with patients was undertaken to vet the framework, which resulted in some minor revisions to the wording and framing.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Interventions: &lt;/strong&gt;A semi-structured virtual interview simulating an intensive care unit handover for a hypothetical patient case using a simulated ML model and seven visualisations using known methods addressing interpretability of models in healthcare. Participants were asked to make an initial care plan for the patient, then were presented with a model prediction followed by the seven visualisations to explore their judgement and potential influence and understanding of the visualisations. Two visualisations contained contradicting information to probe participants' resolution process for the contrasting information. The ethical justifiability and clinical reasoning process were explored.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Main outcome: &lt;/strong&gt;A comprehensive framework was developed that is grounded in established medicolegal and ethical standards and accounts for the incorporation of inference from ML models.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Results: &lt;/strong&gt;We found that for making good decisions, participants reflected across six main categories: evidence, facts and medical knowledge relevant to the patient's condition; how that knowledge may be applied to this particular patient; patient-level, family-specific and local factors; facts about the model, its development and testing; the patient-level knowledge sufficiently represented by the model; the model's incorporation of relevant contextual factors. This judgement was centred on and anchored most heavily on the overall balance of benefits and risks to the patient, framed by the goals of care. We found evidence of automation bias, with many participants assuming that if the model's explanation conflicted with their prior knowledge that their judgement was incorrect; others concluded the exact opposite, drawing from their medical knowledge base to reject the incorrect informati","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":" ","pages":""},"PeriodicalIF":9.0,"publicationDate":"2025-02-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143405686","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Expanded disease definitions in Alzheimer's disease and the new era of disease-modifying drugs.","authors":"Su Jin Yim, Sevil Yasar, Nancy Schoenborn, Eddy Lang","doi":"10.1136/bmjebm-2023-112588","DOIUrl":"https://doi.org/10.1136/bmjebm-2023-112588","url":null,"abstract":"","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":" ","pages":""},"PeriodicalIF":9.0,"publicationDate":"2025-02-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143405685","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Rapid reviews methods series (paper 7): guidance on rapid scoping, mapping and evidence and gap map ('Big Picture Reviews').","authors":"Fiona Campbell, Anthea Sutton, Danielle Pollock, Chantelle Garritty, Andrea C Tricco, Lena Schmidt, Hanan Khalil","doi":"10.1136/bmjebm-2023-112389","DOIUrl":"https://doi.org/10.1136/bmjebm-2023-112389","url":null,"abstract":"","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":" ","pages":""},"PeriodicalIF":9.0,"publicationDate":"2025-02-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143188195","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"AI in healthcare: an introduction for clinicians.","authors":"Ahmed Maiter, Samer Alabed, Genevera Allen, Fares Alahdab","doi":"10.1136/bmjebm-2024-112966","DOIUrl":"https://doi.org/10.1136/bmjebm-2024-112966","url":null,"abstract":"","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":" ","pages":""},"PeriodicalIF":9.0,"publicationDate":"2025-01-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143036402","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Proposed framework for unifying disease definitions in guideline development.","authors":"Hassan Kawtharany, Muayad Azzam, M Hassan Murad, Rebecca L Morgan, Yngve Falck-Ytter, Shahnaz Sultan, Philipp Dahm, Reem A Mustafa","doi":"10.1136/bmjebm-2024-113134","DOIUrl":"https://doi.org/10.1136/bmjebm-2024-113134","url":null,"abstract":"","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":" ","pages":""},"PeriodicalIF":9.0,"publicationDate":"2025-01-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143036403","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Efforts towards the institutionalisation of evidence-informed decision-making.","authors":"Laura Boeira, Emily Hayter, Sandy Oliver, Laurenz Mahlanza-Langer, Donald Simeon, Mukdarut Bangpan, Veronica Osorio Calderon, Ludovic Reveiz, Shelly-Ann Hunte, Firmaye Bogale Wolde, Tanja Kuchenmuller","doi":"10.1136/bmjebm-2024-112962","DOIUrl":"10.1136/bmjebm-2024-112962","url":null,"abstract":"","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":" ","pages":""},"PeriodicalIF":9.0,"publicationDate":"2025-01-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7617397/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143027719","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Risk ratios, odds ratios and the risk difference.","authors":"Rachel Richardson, Afroditi Kanellopoulou, Kerry Dwan","doi":"10.1136/bmjebm-2024-113088","DOIUrl":"10.1136/bmjebm-2024-113088","url":null,"abstract":"","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":" ","pages":"66-67"},"PeriodicalIF":9.0,"publicationDate":"2025-01-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141859009","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"What is the vibration of effects?","authors":"Constant Vinatier, Sabine Hoffmann, Chirag Patel, Nicholas J DeVito, Ioana Alina Cristea, Braden Tierney, John P A Ioannidis, Florian Naudet","doi":"10.1136/bmjebm-2023-112747","DOIUrl":"10.1136/bmjebm-2023-112747","url":null,"abstract":"","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":" ","pages":"61-65"},"PeriodicalIF":9.0,"publicationDate":"2025-01-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11874424/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141598379","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Pilot study on large language models for risk-of-bias assessments in systematic reviews: A(I) new type of bias?","authors":"Joseph Barsby, Samuel Hume, Hamish Al Lemmey, Joseph Cutteridge, Regent Lee, Katarzyna D Bera","doi":"10.1136/bmjebm-2024-112990","DOIUrl":"10.1136/bmjebm-2024-112990","url":null,"abstract":"","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":" ","pages":"71-74"},"PeriodicalIF":9.0,"publicationDate":"2025-01-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141086781","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}