Objective: It is frequent to find overlapping network meta-analyses (NMAs) on the same topic with differences in terms of both treatments included and effect estimates. We aimed to evaluate the impact on effect estimates of selecting different treatment combinations (ie, network geometries) for inclusion in NMAs.
Design: Multiverse analysis, covering all possible NMAs on different combinations of treatments.
Setting: Data from a previously published NMA exploring the comparative effectiveness of 22 treatments (21 antidepressants and a placebo) for the treatment of acute major depressive disorder.
Participants: Cipriani et al explored a dataset of 116 477 patients included in 522 randomised controlled trials.
Main outcome measures: For each possible treatment selection, we performed an NMA to estimate comparative effectiveness on treatment response and treatment discontinuation for the treatments included (231 between-treatment comparisons). The distribution of effect estimates of between-treatment comparisons across NMAs was computed, and the direction, magnitude and statistical significance of the 1st and 99th percentiles were compared.
Results: 4 116 254 different NMAs concerned treatment response. Among possible network geometries, 172/231 (74%) pairwise comparisons exhibited opposite effects between the 1st and 99th percentiles, 57/231 (25%) comparisons exhibited statistically significant results in opposite directions, 118 of 231 (51%) comparisons derived results that were both significant and non-significant at 5% risk and 56/231 (24%) treatment pairs obtained consistent results with only significant differences (or only non-significant differences) at 5% risk. Comparisons based on indirect evidence only were associated with greater variability in effect estimates. Comparisons with small absolute values observed in the complete NMA more frequently obtained statistically significant results in opposite directions. Similar results were observed for treatment discontinuation.
Conclusion: In this multiverse analysis, we observed that the selection of treatments to be included in an NMA could have considerable consequences on treatment effect estimations.
{"title":"Vibration of effects resulting from treatment selection in mixed-treatment comparisons: a multiverse analysis on network meta-analyses of antidepressants in major depressive disorder.","authors":"Constant Vinatier, Clement Palpacuer, Alexandre Scanff, Florian Naudet","doi":"10.1136/bmjebm-2024-112848","DOIUrl":"10.1136/bmjebm-2024-112848","url":null,"abstract":"<p><strong>Objective: </strong>It is frequent to find overlapping network meta-analyses (NMAs) on the same topic with differences in terms of both treatments included and effect estimates. We aimed to evaluate the impact on effect estimates of selecting different treatment combinations (ie, network geometries) for inclusion in NMAs.</p><p><strong>Design: </strong>Multiverse analysis, covering all possible NMAs on different combinations of treatments.</p><p><strong>Setting: </strong>Data from a previously published NMA exploring the comparative effectiveness of 22 treatments (21 antidepressants and a placebo) for the treatment of acute major depressive disorder.</p><p><strong>Participants: </strong>Cipriani <i>et al</i> explored a dataset of 116 477 patients included in 522 randomised controlled trials.</p><p><strong>Main outcome measures: </strong>For each possible treatment selection, we performed an NMA to estimate comparative effectiveness on treatment response and treatment discontinuation for the treatments included (231 between-treatment comparisons). The distribution of effect estimates of between-treatment comparisons across NMAs was computed, and the direction, magnitude and statistical significance of the 1st and 99th percentiles were compared.</p><p><strong>Results: </strong>4 116 254 different NMAs concerned treatment response. Among possible network geometries, 172/231 (74%) pairwise comparisons exhibited opposite effects between the 1st and 99th percentiles, 57/231 (25%) comparisons exhibited statistically significant results in opposite directions, 118 of 231 (51%) comparisons derived results that were both significant and non-significant at 5% risk and 56/231 (24%) treatment pairs obtained consistent results with only significant differences (or only non-significant differences) at 5% risk. Comparisons based on indirect evidence only were associated with greater variability in effect estimates. Comparisons with small absolute values observed in the complete NMA more frequently obtained statistically significant results in opposite directions. Similar results were observed for treatment discontinuation.</p><p><strong>Conclusion: </strong>In this multiverse analysis, we observed that the selection of treatments to be included in an NMA could have considerable consequences on treatment effect estimations.</p><p><strong>Trial registration: </strong>https://osf.io/mb5dy.</p>","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":" ","pages":"324-332"},"PeriodicalIF":9.0,"publicationDate":"2024-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11503165/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141070460","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-19DOI: 10.1136/bmjebm-2024-113038
Mohammad Hassan Murad, Zhen Wang, Yngve Falck-Ytter
{"title":"Facilitating GRADE judgements about the inconsistency of effects using a novel visualisation approach.","authors":"Mohammad Hassan Murad, Zhen Wang, Yngve Falck-Ytter","doi":"10.1136/bmjebm-2024-113038","DOIUrl":"https://doi.org/10.1136/bmjebm-2024-113038","url":null,"abstract":"","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":" ","pages":""},"PeriodicalIF":9.0,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142280158","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-18DOI: 10.1136/bmjebm-2024-112984
Mariska Tuut, Jochen Cals, Jesse Jansen, Jako S Burgers
Recommendations about healthcare related testing in guidelines are common. Tests can be used for several purposes: screening, surveillance, risk classification, diagnosis, staging, treatment triage, determination of prognosis and monitoring/follow-up.1 The development of testing recommendations in guidelines is challenging, especially because the benefit of a test not only depends on test characteristics, such as sensitivity and specificity, but also on population characteristics and test consequences, such as management.2–4 Furthermore, the role of a new test in comparison to the existing testing scenario should be defined, since this influences the interpretation of the new test’s value. The following roles of new tests have been identified in the literature: triage, replacement, add-on, and parallel/combined.5 As with treatment, testing can have negative consequences, including physical impairment, psychological distress, disease labelling, and costs.6 There is limited evidence on harms of testing, and healthcare professionals often overestimate its benefits while underestimating its harms.7 This is also true for patients' expectations of testing.8 Additionally, testing occasionally yields unexpected and coincidental findings, which may result in additional testing and treatment. There is a lack of transparency in processing the evidence and considerations that support testing recommendations in guidelines.9 To facilitate the development of test recommendations, we determined the minimum required knowledge for guideline panel members involved, supplementing the competency-based framework available for guideline development.10 11 The concept of the test-management pathway (figure 1) appeared key to understand. Figure 1 Test-management pathway concept. During our developmental study, the need for practical examples of test-management pathways became apparent.10 In our subsequent teach-the-teacher workshop at the 2023 Guideline International Network conference,12 participants requested additional elaboration of pathways for different test outcomes (such as false positives and false negatives) being helpful for explaining the test-management pathway concept to guideline …
{"title":"Developing guideline recommendations about tests: educational examples of test-management pathways","authors":"Mariska Tuut, Jochen Cals, Jesse Jansen, Jako S Burgers","doi":"10.1136/bmjebm-2024-112984","DOIUrl":"https://doi.org/10.1136/bmjebm-2024-112984","url":null,"abstract":"Recommendations about healthcare related testing in guidelines are common. Tests can be used for several purposes: screening, surveillance, risk classification, diagnosis, staging, treatment triage, determination of prognosis and monitoring/follow-up.1 The development of testing recommendations in guidelines is challenging, especially because the benefit of a test not only depends on test characteristics, such as sensitivity and specificity, but also on population characteristics and test consequences, such as management.2–4 Furthermore, the role of a new test in comparison to the existing testing scenario should be defined, since this influences the interpretation of the new test’s value. The following roles of new tests have been identified in the literature: triage, replacement, add-on, and parallel/combined.5 As with treatment, testing can have negative consequences, including physical impairment, psychological distress, disease labelling, and costs.6 There is limited evidence on harms of testing, and healthcare professionals often overestimate its benefits while underestimating its harms.7 This is also true for patients' expectations of testing.8 Additionally, testing occasionally yields unexpected and coincidental findings, which may result in additional testing and treatment. There is a lack of transparency in processing the evidence and considerations that support testing recommendations in guidelines.9 To facilitate the development of test recommendations, we determined the minimum required knowledge for guideline panel members involved, supplementing the competency-based framework available for guideline development.10 11 The concept of the test-management pathway (figure 1) appeared key to understand. Figure 1 Test-management pathway concept. During our developmental study, the need for practical examples of test-management pathways became apparent.10 In our subsequent teach-the-teacher workshop at the 2023 Guideline International Network conference,12 participants requested additional elaboration of pathways for different test outcomes (such as false positives and false negatives) being helpful for explaining the test-management pathway concept to guideline …","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":"201 1","pages":""},"PeriodicalIF":5.8,"publicationDate":"2024-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142254700","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-12DOI: 10.1136/bmjebm-2024-113033
Ryan Muller, Giovanni Ferreira, Geronimo Bejarano, Andrew R Gamble, James Kirk, James Sindone, Joshua R Zadro
Objective To compare the prevalence of ‘spin’, and specific reporting strategies for spin, between infographics, abstracts and full texts of randomised controlled trials (RCTs) reporting non-significant findings in the field of health and medicine and to assess factors associated with the presence of spin. Design Cross-sectional observational study. Data source Publications in top quintile health and medical journals from August 2018 to October 2020 (Journal Citation Reports database). Eligibility criteria Infographics, abstracts and full texts of RCTs with non-significant results for a primary outcome. Main outcome(s) and measure(s) Presence of spin (any spin and spin in the results and conclusions of infographics, abstracts and full texts). Exposure(s) Conflicts of interest, industry sponsorship, trial registration, journal impact factor, spin in the abstract, spin in the full text. Results 119 studies from 40 journals were included. One-third (33%) of infographics contained spin. Infographics were not more likely to contain any spin than abstracts (33% vs 26%, OR 1.4; 95% CI 0.8 to 2.4) or full texts (33% vs 26%, OR 1.4; 95% CI 0.8 to 2.4). Higher journal impact factor was associated with slightly lower odds of spin in infographics and full texts, but not abstracts. Infographics, but not abstracts or full texts, were less likely to contain spin if the trial was prospectively registered. No other significant associations were found. Conclusions Nearly one-third of infographics spin the findings of RCTs with non-significant results for a primary outcome, but the prevalence of spin is not higher than in abstracts and full texts. Given the increasing popularity of infographics to disseminate research findings, there is an urgent need to improve the reporting of research in infographics. Data are available on reasonable request. Original data may be made available by the authors on reasonable request.
目的 比较在健康和医学领域报告非显著性研究结果的随机对照试验（RCT）的信息图表、摘要和全文中 "自旋 "的普遍程度以及针对自旋的特定报告策略，并评估与自旋存在相关的因素。设计 横断面观察研究。数据来源 2018年8月至2020年10月在前五分之一健康与医学期刊上发表的文章（期刊引文报告数据库）。资格标准 主要结果不显著的 RCT 的信息图表、摘要和全文。主要结果和衡量标准 是否存在自旋（信息图表、摘要和全文的结果和结论中的任何自旋和自旋）。利益冲突、行业赞助、试验注册、期刊影响因子、摘要中的自旋、全文中的自旋。结果 共收录了来自 40 种期刊的 119 项研究。三分之一（33%）的信息图表中包含自旋内容。与摘要（33% vs 26%，OR 1.4；95% CI 0.8-2.4）或全文（33% vs 26%，OR 1.4；95% CI 0.8-2.4）相比，信息图表中含有自旋成分的可能性并不大。期刊影响因子越高，信息图表和全文中出现自旋的几率就越低，而摘要中出现自旋的几率则不高。如果试验是前瞻性注册的，信息图表（而非摘要或全文）中包含自旋的可能性较低。没有发现其他重要关联。结论 近三分之一的信息图表对主要结果不显著的 RCT 结果进行了转述，但转述的普遍程度并不高于摘要和全文。鉴于信息图表在传播研究成果方面越来越受欢迎，因此迫切需要改进信息图表中的研究报告。如有合理要求，可提供数据。作者可根据合理要求提供原始数据。
{"title":"Do infographics ‘spin’ the findings of health and medical research?","authors":"Ryan Muller, Giovanni Ferreira, Geronimo Bejarano, Andrew R Gamble, James Kirk, James Sindone, Joshua R Zadro","doi":"10.1136/bmjebm-2024-113033","DOIUrl":"https://doi.org/10.1136/bmjebm-2024-113033","url":null,"abstract":"Objective To compare the prevalence of ‘spin’, and specific reporting strategies for spin, between infographics, abstracts and full texts of randomised controlled trials (RCTs) reporting non-significant findings in the field of health and medicine and to assess factors associated with the presence of spin. Design Cross-sectional observational study. Data source Publications in top quintile health and medical journals from August 2018 to October 2020 (Journal Citation Reports database). Eligibility criteria Infographics, abstracts and full texts of RCTs with non-significant results for a primary outcome. Main outcome(s) and measure(s) Presence of spin (any spin and spin in the results and conclusions of infographics, abstracts and full texts). Exposure(s) Conflicts of interest, industry sponsorship, trial registration, journal impact factor, spin in the abstract, spin in the full text. Results 119 studies from 40 journals were included. One-third (33%) of infographics contained spin. Infographics were not more likely to contain any spin than abstracts (33% vs 26%, OR 1.4; 95% CI 0.8 to 2.4) or full texts (33% vs 26%, OR 1.4; 95% CI 0.8 to 2.4). Higher journal impact factor was associated with slightly lower odds of spin in infographics and full texts, but not abstracts. Infographics, but not abstracts or full texts, were less likely to contain spin if the trial was prospectively registered. No other significant associations were found. Conclusions Nearly one-third of infographics spin the findings of RCTs with non-significant results for a primary outcome, but the prevalence of spin is not higher than in abstracts and full texts. Given the increasing popularity of infographics to disseminate research findings, there is an urgent need to improve the reporting of research in infographics. Data are available on reasonable request. Original data may be made available by the authors on reasonable request.","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":"11 1","pages":""},"PeriodicalIF":5.8,"publicationDate":"2024-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142175805","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-10DOI: 10.1136/bmjebm-2024-112871
Glyn Elwyn, Marie Anne Durand, Thomas Agoritsas, Martin Mayer
The work of transforming scientific publications into tools that can support people in comparing treatments, tests and other interventions has been driven by efforts to support shared decision-making. The International Patient Decision Aids Standards (IPDAS) Collaboration has published many articles that guide this process. IPDAS, and others in this field, have considered the challenges of representing complex concepts such as effect sizes, ORs and relative risk rates in terms and formats that are easier to understand by people with varying levels of health literacy and numeracy. The underlying task is how to simplify research results without misleading people, which is essential when communicating healthcare information. Selecting outcome probabilities and comparing data collected from different populations with various study designs would be misleading. Further, only providing relative instead of absolute risk estimates would lead to misinterpretation in most situations, framing that affects both professionals and patients.1 For example, imagine a person with a 4% 10-year risk of breast cancer recurrence: telling that person their risk of breast cancer coming back could be cut in half, or reduced by 50%, using treatment Y (relative risk) is misleading. The more informative approach would be telling them their risk could be reduced from 4% to 2% (absolute risk), a risk that is perhaps even clearer if portrayed in natural frequency (n in 100). This article tackles the challenge of untangling complicated, heterogeneous evidence to deliver comparative information to non-scientists while at the same time avoiding misrepresenting the underlying data. Our focus here is to describe some of the editorial challenges of presenting complex scientific information, particularly when published as continuous outcomes in ways that are accurate yet accessible to a broad public, irrespective of their health literacy and numeracy levels. Those editorial challenges have materialised when developing tools to promote shared decision-making. The …
将科学出版物转化为可帮助人们比较治疗、检验和其他干预措施的工具的工作，是由支持共同决策的努力所推动的。国际患者决策辅助标准（IPDAS）合作组织发表了许多文章，为这一过程提供指导。IPDAS 以及该领域的其他组织已经考虑了如何用术语和格式来表示效应大小、OR 和相对风险率等复杂概念的挑战，以便让不同健康知识水平和计算能力的人更容易理解。其根本任务是如何在不误导人们的情况下简化研究结果，这对于传播医疗保健信息至关重要。选择结果概率和比较从不同人群和不同研究设计中收集的数据会产生误导。此外，在大多数情况下，只提供相对而非绝对的风险估计值会导致误解，这对专业人士和患者都有影响。1 例如，想象一个 10 年乳腺癌复发风险为 4% 的人，告诉他使用 Y 治疗方法（相对风险）可将其乳腺癌复发风险降低一半或 50%，这就是误导。更有参考价值的方法是告诉他们，他们的风险可以从 4% 降到 2%（绝对风险），如果用自然频率（n in 100）来描述这种风险，也许会更加清晰。本文探讨的难题是如何理清复杂的异质证据，向非科学家提供比较信息，同时避免对基础数据的错误表述。我们在此重点描述了编辑在呈现复杂的科学信息时所面临的一些挑战，尤其是在以连续性结果的形式发布时，既要准确无误，又要让广大公众能够理解，无论他们的健康知识水平和计算能力如何。在开发促进共同决策的工具时，这些编辑方面的挑战就显现出来了。这...
{"title":"How to best convey continuous outcomes in patient decision aids","authors":"Glyn Elwyn, Marie Anne Durand, Thomas Agoritsas, Martin Mayer","doi":"10.1136/bmjebm-2024-112871","DOIUrl":"https://doi.org/10.1136/bmjebm-2024-112871","url":null,"abstract":"The work of transforming scientific publications into tools that can support people in comparing treatments, tests and other interventions has been driven by efforts to support shared decision-making. The International Patient Decision Aids Standards (IPDAS) Collaboration has published many articles that guide this process. IPDAS, and others in this field, have considered the challenges of representing complex concepts such as effect sizes, ORs and relative risk rates in terms and formats that are easier to understand by people with varying levels of health literacy and numeracy. The underlying task is how to simplify research results without misleading people, which is essential when communicating healthcare information. Selecting outcome probabilities and comparing data collected from different populations with various study designs would be misleading. Further, only providing relative instead of absolute risk estimates would lead to misinterpretation in most situations, framing that affects both professionals and patients.1 For example, imagine a person with a 4% 10-year risk of breast cancer recurrence: telling that person their risk of breast cancer coming back could be cut in half, or reduced by 50%, using treatment Y (relative risk) is misleading. The more informative approach would be telling them their risk could be reduced from 4% to 2% (absolute risk), a risk that is perhaps even clearer if portrayed in natural frequency (n in 100). This article tackles the challenge of untangling complicated, heterogeneous evidence to deliver comparative information to non-scientists while at the same time avoiding misrepresenting the underlying data. Our focus here is to describe some of the editorial challenges of presenting complex scientific information, particularly when published as continuous outcomes in ways that are accurate yet accessible to a broad public, irrespective of their health literacy and numeracy levels. Those editorial challenges have materialised when developing tools to promote shared decision-making. The …","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":"85 1","pages":""},"PeriodicalIF":5.8,"publicationDate":"2024-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142175810","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-03DOI: 10.1136/bmjebm-2024-113025
Alison Huffstetler, Kenneth W Lin, Russell P Harris
{"title":"Assessing proposals to update established screening strategies.","authors":"Alison Huffstetler, Kenneth W Lin, Russell P Harris","doi":"10.1136/bmjebm-2024-113025","DOIUrl":"https://doi.org/10.1136/bmjebm-2024-113025","url":null,"abstract":"","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":" ","pages":""},"PeriodicalIF":9.0,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142124772","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-29DOI: 10.1136/bmjebm-2024-113031
Madelin R Siedler, Reem A Mustafa, Lifeng Lin, Rebecca L Morgan, Yngve Falck-Ytter, Philipp Dahm, Shahnaz Sultan, Mohammad Hassan Murad
{"title":"Meta-analysis of continuous outcomes: a user's guide for analysis and interpretation.","authors":"Madelin R Siedler, Reem A Mustafa, Lifeng Lin, Rebecca L Morgan, Yngve Falck-Ytter, Philipp Dahm, Shahnaz Sultan, Mohammad Hassan Murad","doi":"10.1136/bmjebm-2024-113031","DOIUrl":"https://doi.org/10.1136/bmjebm-2024-113031","url":null,"abstract":"","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":" ","pages":""},"PeriodicalIF":9.0,"publicationDate":"2024-08-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142104128","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号