Pub Date : 2025-02-12DOI: 10.1136/bmjebm-2024-112919
Melissa D McCradden, Kelly Thai, Azadeh Assadi, Sana Tonekaboni, Ian Stedman, Shalmali Joshi, Minfan Zhang, Fanny Chevalier, Anna Goldenberg
<p><strong>Objective: </strong>To develop a framework for good clinical decision-making using machine learning (ML) models for interventional, patient-level decisions.</p><p><strong>Design: </strong>Grounded theory qualitative interview study.</p><p><strong>Setting: </strong>Primarily single-site at a major urban academic paediatric hospital, with external sampling.</p><p><strong>Participants: </strong>Sixteen participants representing physicians (n=10), nursing (n=3), respiratory therapists (n=2) and an ML specialist (n=1) with experience working in acute care environments were identified through purposive sampling. Individuals were recruited to represent a spectrum of ML knowledge (three expert, four knowledgeable and nine non-expert) and years of experience (median=12.9 years postgraduation). Recruitment proceeded through snowball sampling, with individuals approached to represent a diversity of fields, levels of experience and attitudes towards artificial intelligence (AI)/ML. A member check step and consultation with patients was undertaken to vet the framework, which resulted in some minor revisions to the wording and framing.</p><p><strong>Interventions: </strong>A semi-structured virtual interview simulating an intensive care unit handover for a hypothetical patient case using a simulated ML model and seven visualisations using known methods addressing interpretability of models in healthcare. Participants were asked to make an initial care plan for the patient, then were presented with a model prediction followed by the seven visualisations to explore their judgement and potential influence and understanding of the visualisations. Two visualisations contained contradicting information to probe participants' resolution process for the contrasting information. The ethical justifiability and clinical reasoning process were explored.</p><p><strong>Main outcome: </strong>A comprehensive framework was developed that is grounded in established medicolegal and ethical standards and accounts for the incorporation of inference from ML models.</p><p><strong>Results: </strong>We found that for making good decisions, participants reflected across six main categories: evidence, facts and medical knowledge relevant to the patient's condition; how that knowledge may be applied to this particular patient; patient-level, family-specific and local factors; facts about the model, its development and testing; the patient-level knowledge sufficiently represented by the model; the model's incorporation of relevant contextual factors. This judgement was centred on and anchored most heavily on the overall balance of benefits and risks to the patient, framed by the goals of care. We found evidence of automation bias, with many participants assuming that if the model's explanation conflicted with their prior knowledge that their judgement was incorrect; others concluded the exact opposite, drawing from their medical knowledge base to reject the incorrect informati
{"title":"What makes a 'good' decision with artificial intelligence? A grounded theory study in paediatric care.","authors":"Melissa D McCradden, Kelly Thai, Azadeh Assadi, Sana Tonekaboni, Ian Stedman, Shalmali Joshi, Minfan Zhang, Fanny Chevalier, Anna Goldenberg","doi":"10.1136/bmjebm-2024-112919","DOIUrl":"https://doi.org/10.1136/bmjebm-2024-112919","url":null,"abstract":"<p><strong>Objective: </strong>To develop a framework for good clinical decision-making using machine learning (ML) models for interventional, patient-level decisions.</p><p><strong>Design: </strong>Grounded theory qualitative interview study.</p><p><strong>Setting: </strong>Primarily single-site at a major urban academic paediatric hospital, with external sampling.</p><p><strong>Participants: </strong>Sixteen participants representing physicians (n=10), nursing (n=3), respiratory therapists (n=2) and an ML specialist (n=1) with experience working in acute care environments were identified through purposive sampling. Individuals were recruited to represent a spectrum of ML knowledge (three expert, four knowledgeable and nine non-expert) and years of experience (median=12.9 years postgraduation). Recruitment proceeded through snowball sampling, with individuals approached to represent a diversity of fields, levels of experience and attitudes towards artificial intelligence (AI)/ML. A member check step and consultation with patients was undertaken to vet the framework, which resulted in some minor revisions to the wording and framing.</p><p><strong>Interventions: </strong>A semi-structured virtual interview simulating an intensive care unit handover for a hypothetical patient case using a simulated ML model and seven visualisations using known methods addressing interpretability of models in healthcare. Participants were asked to make an initial care plan for the patient, then were presented with a model prediction followed by the seven visualisations to explore their judgement and potential influence and understanding of the visualisations. Two visualisations contained contradicting information to probe participants' resolution process for the contrasting information. The ethical justifiability and clinical reasoning process were explored.</p><p><strong>Main outcome: </strong>A comprehensive framework was developed that is grounded in established medicolegal and ethical standards and accounts for the incorporation of inference from ML models.</p><p><strong>Results: </strong>We found that for making good decisions, participants reflected across six main categories: evidence, facts and medical knowledge relevant to the patient's condition; how that knowledge may be applied to this particular patient; patient-level, family-specific and local factors; facts about the model, its development and testing; the patient-level knowledge sufficiently represented by the model; the model's incorporation of relevant contextual factors. This judgement was centred on and anchored most heavily on the overall balance of benefits and risks to the patient, framed by the goals of care. We found evidence of automation bias, with many participants assuming that if the model's explanation conflicted with their prior knowledge that their judgement was incorrect; others concluded the exact opposite, drawing from their medical knowledge base to reject the incorrect informati","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":" ","pages":""},"PeriodicalIF":9.0,"publicationDate":"2025-02-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143405686","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-02-12DOI: 10.1136/bmjebm-2023-112588
Su Jin Yim, Sevil Yasar, Nancy Schoenborn, Eddy Lang
{"title":"Expanded disease definitions in Alzheimer's disease and the new era of disease-modifying drugs.","authors":"Su Jin Yim, Sevil Yasar, Nancy Schoenborn, Eddy Lang","doi":"10.1136/bmjebm-2023-112588","DOIUrl":"https://doi.org/10.1136/bmjebm-2023-112588","url":null,"abstract":"","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":" ","pages":""},"PeriodicalIF":9.0,"publicationDate":"2025-02-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143405685","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-02-04DOI: 10.1136/bmjebm-2024-113235
Ignazio Geraci, Silvia Bargeri, Giacomo Basso, Greta Castellini, Alessandro Chiarotto, Silvia Gianola, Raymond Ostelo, Marco Testa, Tiziano Innocenti
Objective: To assess the therapeutic quality of exercise interventions delivered in chronic low back pain (cLBP) trials using the international Consensus on Therapeutic Exercise aNd Training (i-CONTENT) tool and its inter-rater agreement.
Methods: We performed a meta-research study, starting from the trials' arms included in the published Cochrane review (2021) 'Exercise therapy for chronic low back pain'. Two pairs of independent reviewers applied the i-CONTENT tool, a standardised tool designed to ensure the quality of exercise therapy intervention, in a random sample of 100 different exercise arms. We assessed the inter-rater agreement of each category calculating the specific agreement. A percentage of 70% was considered satisfactory.
Results: We included 100 arms from 68 randomised controlled trials published between 1991 and 2019. The most assessed exercise types were core strengthening (n=27 arms) and motor control (n=13 arms). Among alternative approaches, yoga (n=11) and Pilates (n=7) were the most representative. Overall, most exercise interventions were rated as having a low risk of ineffectiveness for patient selection (100%), exercise type (92%), outcome type and timing (89%) and qualified supervisor (84%). Conversely, some items showed more uncertainty: the safety of exercise programmes was rated as 'probably low risk' in 58% of cases, exercise dosage in 34% and adherence to exercise in 44%. The items related to exercise dosage (31%) and adherence (29%) had heterogenous judgements, scoring as high risk of ineffectiveness or probably not done. Among all exercise types, Pilates scored best in all domains. A satisfactory specific agreement for 'low risk category' was achieved in all items, except dosage of exercise (60%) and adherence to exercise (54%).
Conclusion: Exercises delivered for patients with cLBP generally demonstrate favourable therapeutic quality, although some exercise modalities may present poor therapeutic quality related to dosage and adherence. While the i-CONTENT judgements generally showed satisfactory specific agreement between raters, disagreements arose in evaluating some crucial items.
{"title":"Therapeutic quality of exercise interventions for chronic low back pain: a meta-research study using i-CONTENT tool.","authors":"Ignazio Geraci, Silvia Bargeri, Giacomo Basso, Greta Castellini, Alessandro Chiarotto, Silvia Gianola, Raymond Ostelo, Marco Testa, Tiziano Innocenti","doi":"10.1136/bmjebm-2024-113235","DOIUrl":"10.1136/bmjebm-2024-113235","url":null,"abstract":"<p><strong>Objective: </strong>To assess the therapeutic quality of exercise interventions delivered in chronic low back pain (cLBP) trials using the international Consensus on Therapeutic Exercise aNd Training (i-CONTENT) tool and its inter-rater agreement.</p><p><strong>Methods: </strong>We performed a meta-research study, starting from the trials' arms included in the published Cochrane review (2021) 'Exercise therapy for chronic low back pain'. Two pairs of independent reviewers applied the i-CONTENT tool, a standardised tool designed to ensure the quality of exercise therapy intervention, in a random sample of 100 different exercise arms. We assessed the inter-rater agreement of each category calculating the specific agreement. A percentage of 70% was considered satisfactory.</p><p><strong>Results: </strong>We included 100 arms from 68 randomised controlled trials published between 1991 and 2019. The most assessed exercise types were core strengthening (n=27 arms) and motor control (n=13 arms). Among alternative approaches, yoga (n=11) and Pilates (n=7) were the most representative. Overall, most exercise interventions were rated as having a low risk of ineffectiveness for patient selection (100%), exercise type (92%), outcome type and timing (89%) and qualified supervisor (84%). Conversely, some items showed more uncertainty: the safety of exercise programmes was rated as 'probably low risk' in 58% of cases, exercise dosage in 34% and adherence to exercise in 44%. The items related to exercise dosage (31%) and adherence (29%) had heterogenous judgements, scoring as high risk of ineffectiveness or probably not done. Among all exercise types, Pilates scored best in all domains. A satisfactory specific agreement for 'low risk category' was achieved in all items, except dosage of exercise (60%) and adherence to exercise (54%).</p><p><strong>Conclusion: </strong>Exercises delivered for patients with cLBP generally demonstrate favourable therapeutic quality, although some exercise modalities may present poor therapeutic quality related to dosage and adherence. While the i-CONTENT judgements generally showed satisfactory specific agreement between raters, disagreements arose in evaluating some crucial items.</p>","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":" ","pages":""},"PeriodicalIF":9.0,"publicationDate":"2025-02-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143027793","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-02-04DOI: 10.1136/bmjebm-2023-112389
Fiona Campbell, Anthea Sutton, Danielle Pollock, Chantelle Garritty, Andrea C Tricco, Lena Schmidt, Hanan Khalil
{"title":"Rapid reviews methods series (paper 7): guidance on rapid scoping, mapping and evidence and gap map ('Big Picture Reviews').","authors":"Fiona Campbell, Anthea Sutton, Danielle Pollock, Chantelle Garritty, Andrea C Tricco, Lena Schmidt, Hanan Khalil","doi":"10.1136/bmjebm-2023-112389","DOIUrl":"https://doi.org/10.1136/bmjebm-2023-112389","url":null,"abstract":"","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":" ","pages":""},"PeriodicalIF":9.0,"publicationDate":"2025-02-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143188195","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-22DOI: 10.1136/bmjebm-2023-112747
Constant Vinatier, Sabine Hoffmann, Chirag Patel, Nicholas J DeVito, Ioana Alina Cristea, Braden Tierney, John P A Ioannidis, Florian Naudet
{"title":"What is the vibration of effects?","authors":"Constant Vinatier, Sabine Hoffmann, Chirag Patel, Nicholas J DeVito, Ioana Alina Cristea, Braden Tierney, John P A Ioannidis, Florian Naudet","doi":"10.1136/bmjebm-2023-112747","DOIUrl":"10.1136/bmjebm-2023-112747","url":null,"abstract":"","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":" ","pages":"61-65"},"PeriodicalIF":9.0,"publicationDate":"2025-01-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141598379","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-22DOI: 10.1136/bmjebm-2024-112990
Joseph Barsby, Samuel Hume, Hamish Al Lemmey, Joseph Cutteridge, Regent Lee, Katarzyna D Bera
{"title":"Pilot study on large language models for risk-of-bias assessments in systematic reviews: A(I) new type of bias?","authors":"Joseph Barsby, Samuel Hume, Hamish Al Lemmey, Joseph Cutteridge, Regent Lee, Katarzyna D Bera","doi":"10.1136/bmjebm-2024-112990","DOIUrl":"10.1136/bmjebm-2024-112990","url":null,"abstract":"","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":" ","pages":"71-74"},"PeriodicalIF":9.0,"publicationDate":"2025-01-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141086781","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：