Australia's national living guidelines for COVID-19 were launched in April 2020 and include over 200 recommendations to guide the clinical care of patients with COVID-19. Until the guidelines were retired in August 2023, new evidence was continually monitored through daily surveillance. This paper describes the initial design for evidence surveillance (at a time when efforts to collate information on the novel coronavirus were in their infancy and often duplicative) and how it was refined throughout the pandemic. Among the wide range of sources monitored, the collections of COVID-19 research from leading medical journals, medRxiv preprints and PubMed auto alerts proved the most enduring in identifying time-critical and impactful evidence. The paper also explores how evidence was tracked and surveillance integrated into the overall evidence workflow by using messaging apps and communication platforms. Finally, we consider the implications for living guidelines and reflect on factors that contributed to the feasibility of daily surveillance: the clearly defined scope of the guidelines; focusing efforts on maximum impact; minimizing duplication by partnering with others; setting up simple but effective processes for managing evidence; and the value of continuous involvement of personnel from the outset. Australia's living COVID-19 guidelines were underpinned by a novel approach to evidence surveillance that was feasible and essential in maintaining COVID-19 living guidelines for over 3 years.
{"title":"Designing, refining and reflecting on 3 years of daily evidence surveillance for Australia's living national COVID-19 guidelines","authors":"Steve McDonald, Tari Turner","doi":"10.1002/cesm.12045","DOIUrl":"https://doi.org/10.1002/cesm.12045","url":null,"abstract":"<p>Australia's national living guidelines for COVID-19 were launched in April 2020 and include over 200 recommendations to guide the clinical care of patients with COVID-19. Until the guidelines were retired in August 2023, new evidence was continually monitored through daily surveillance. This paper describes the initial design for evidence surveillance (at a time when efforts to collate information on the novel coronavirus were in their infancy and often duplicative) and how it was refined throughout the pandemic. Among the wide range of sources monitored, the collections of COVID-19 research from leading medical journals, medRxiv preprints and PubMed auto alerts proved the most enduring in identifying time-critical and impactful evidence. The paper also explores how evidence was tracked and surveillance integrated into the overall evidence workflow by using messaging apps and communication platforms. Finally, we consider the implications for living guidelines and reflect on factors that contributed to the feasibility of daily surveillance: the clearly defined scope of the guidelines; focusing efforts on maximum impact; minimizing duplication by partnering with others; setting up simple but effective processes for managing evidence; and the value of continuous involvement of personnel from the outset. Australia's living COVID-19 guidelines were underpinned by a novel approach to evidence surveillance that was feasible and essential in maintaining COVID-19 living guidelines for over 3 years.</p>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"2 3","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-02-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.12045","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139976593","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kusala Pussegoda, Tricia Corrin, Austyn Baumeister, Dima Ayache, Lisa Waddell
Background
In May of 2022, several cases of mpox were identified in multiple nonendemic countries and on July 23, 2022 the World Health Organization declared mpox a Public Health Emergency of International Concern. During the first six months of the outbreak there was an urgent need to have up-to-date synthesized evidence on mpox to inform public health decision-making. At this point, evidence is changing too quickly for traditional evidence synthesis methods, as systematic reviews were out-of-date before publication. This paper describes the framework developed to manage and maintain a living evidence profile (LEP) to systematically identify, classify and synthesize evidence on a broad range of mpox topics at a rapid pace as the outbreak unfolded.
Methods
The LEP framework was based on principles of evidence synthesis, risk assessment, priority epidemiological parameters for infectious disease modeling and consultation with experts. The framework consisted of a systematic search conducted twice weekly; study selection; categorization into pre-determined foci and data extraction; integration and synthesis of evidence; internal peer-review and dissemination to stakeholders.
Results
Between April 14 and December 15, 2022, 2287 citations were identified, 687 were primary research studies or surveillance reports on the 2022 mpox outbreak and 496 were included in the final LEP. Each study was mapped to one of 32 foci and evidence was narratively synthesized. From June to December 2022, 23 LEPs were produced (approximately weekly) along with a searchable database of extracted data of the mpox literature. They were disseminated globally to public health researchers and decision-makers to inform public health response efforts.
Conclusions
The LEP framework is applicable to other public health emergencies when a rapid synthesis cycle is required because the evidence is evolving quickly. This efficient methodology for creating up-to-date summaries of the current evidence during the first few months of an outbreak or emergency supports public health decision-making and response activities.
{"title":"Methods for conducting a living evidence profile on mpox: An evidence map of the literature","authors":"Kusala Pussegoda, Tricia Corrin, Austyn Baumeister, Dima Ayache, Lisa Waddell","doi":"10.1002/cesm.12044","DOIUrl":"https://doi.org/10.1002/cesm.12044","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Background</h3>\u0000 \u0000 <p>In May of 2022, several cases of mpox were identified in multiple nonendemic countries and on July 23, 2022 the World Health Organization declared mpox a Public Health Emergency of International Concern. During the first six months of the outbreak there was an urgent need to have up-to-date synthesized evidence on mpox to inform public health decision-making. At this point, evidence is changing too quickly for traditional evidence synthesis methods, as systematic reviews were out-of-date before publication. This paper describes the framework developed to manage and maintain a living evidence profile (LEP) to systematically identify, classify and synthesize evidence on a broad range of mpox topics at a rapid pace as the outbreak unfolded.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>The LEP framework was based on principles of evidence synthesis, risk assessment, priority epidemiological parameters for infectious disease modeling and consultation with experts. The framework consisted of a systematic search conducted twice weekly; study selection; categorization into pre-determined foci and data extraction; integration and synthesis of evidence; internal peer-review and dissemination to stakeholders.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>Between April 14 and December 15, 2022, 2287 citations were identified, 687 were primary research studies or surveillance reports on the 2022 mpox outbreak and 496 were included in the final LEP. Each study was mapped to one of 32 foci and evidence was narratively synthesized. From June to December 2022, 23 LEPs were produced (approximately weekly) along with a searchable database of extracted data of the mpox literature. They were disseminated globally to public health researchers and decision-makers to inform public health response efforts.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusions</h3>\u0000 \u0000 <p>The LEP framework is applicable to other public health emergencies when a rapid synthesis cycle is required because the evidence is evolving quickly. This efficient methodology for creating up-to-date summaries of the current evidence during the first few months of an outbreak or emergency supports public health decision-making and response activities.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"2 2","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-02-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.12044","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139937298","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Plain language summaries (PLSs) make complex healthcare evidence accessible to patients and the public. Large language models (LLMs) may assist in generating accurate, readable PLSs. This study explored using the LLM Claude 2 to create PLSs of evidence reviews from the Agency for Healthcare Research and Quality (AHRQ) Effective Health Care Program.
Methods
We selected 10 evidence reviews published from 2021 to 2023, representing a range of methods and topics. We iteratively developed a prompt to guide Claude 2 in creating PLSs which included specifications for plain language, reading level, length, organizational structure, active voice, and inclusive language. PLSs were assessed for adherence to prompt specifications, comprehensiveness, accuracy, readability, and cultural sensitivity.
Results
All PLSs met the word count. We judged one PLS as fully comprehensive; seven mostly comprehensive. We judged two PLSs as fully capturing the PICO elements; five with minor PICO errors. We judged three PLSs as accurately reporting the results; and four with minor result errors. We judged three PLSs as having major result errors for incorrectly reporting total participants. Five PLSs met the target 6th to 8th grade reading level. Passive voice use averaged 16%. All PLSs used inclusive language.
Conclusions
LLMs show promise for assisting in PLS creation but likely require human input to ensure accuracy, comprehensiveness, and the appropriate nuances of interpretation. Iterative prompt refinement may improve results and address the needs of specific reviews and audiences. As text-only summaries, the AI-generated PLSs could not meet all consumer communication criteria, such as textual design and visual representations. Further testing should include consumer reviewers and explore how to best leverage LLM support in drafting PLS text for complex evidence reviews.
{"title":"The use of a large language model to create plain language summaries of evidence reviews in healthcare: A feasibility study","authors":"Colleen Ovelman, Shannon Kugley, Gerald Gartlehner, Meera Viswanathan","doi":"10.1002/cesm.12041","DOIUrl":"https://doi.org/10.1002/cesm.12041","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Introduction</h3>\u0000 \u0000 <p>Plain language summaries (PLSs) make complex healthcare evidence accessible to patients and the public. Large language models (LLMs) may assist in generating accurate, readable PLSs. This study explored using the LLM Claude 2 to create PLSs of evidence reviews from the Agency for Healthcare Research and Quality (AHRQ) Effective Health Care Program.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>We selected 10 evidence reviews published from 2021 to 2023, representing a range of methods and topics. We iteratively developed a prompt to guide Claude 2 in creating PLSs which included specifications for plain language, reading level, length, organizational structure, active voice, and inclusive language. PLSs were assessed for adherence to prompt specifications, comprehensiveness, accuracy, readability, and cultural sensitivity.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>All PLSs met the word count. We judged one PLS as fully comprehensive; seven mostly comprehensive. We judged two PLSs as fully capturing the PICO elements; five with minor PICO errors. We judged three PLSs as accurately reporting the results; and four with minor result errors. We judged three PLSs as having major result errors for incorrectly reporting total participants. Five PLSs met the target 6th to 8th grade reading level. Passive voice use averaged 16%. All PLSs used inclusive language.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusions</h3>\u0000 \u0000 <p>LLMs show promise for assisting in PLS creation but likely require human input to ensure accuracy, comprehensiveness, and the appropriate nuances of interpretation. Iterative prompt refinement may improve results and address the needs of specific reviews and audiences. As text-only summaries, the AI-generated PLSs could not meet all consumer communication criteria, such as textual design and visual representations. Further testing should include consumer reviewers and explore how to best leverage LLM support in drafting PLS text for complex evidence reviews.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"2 2","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-02-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.12041","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139682823","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Wilson Jiang, Bill Wang, Sandro Sperandei, Aidan Christopher Tan
Background
There are few randomized controlled trials (RCTs) in pediatric surgery, and their risk of bias is unknown. There is also little known about the methodological or reporting quality of systematic reviews (with or without meta-analyses) in pediatric surgery. Therefore, we conducted a cross-sectional meta-research study to determine the risk of bias and reporting quality of RCTs and systematic reviews and meta-analyses in pediatric surgery, and the associations between these outcomes and study characteristics.
Methods
We searched MEDLINE, Embase, Cochrane Library, JBI EBP Database, Centre for Reviews and Dissemination and Web of Science for all RCTs and systematic reviews in pediatric surgery published in 2021. We also searched the 2021 indexes of high-impact pediatric surgery journals. We assessed the risk of bias and reporting quality of RCTs using the RoB 2 and CONSORT tools respectively. We assessed the same parameters for systematic reviews and meta-analyses using the ROBIS and Preferred Reporting Items for Systematic Reviews and Meta-analyses tools.
Findings
We found 82 RCTs and 289 systematic reviews/meta-analyses published in 2021. More than half (n = 46, 56%) of RCTs and almost all (n = 278, 96%) systematic reviews and meta-analyses were at high risk of bias. Only one (1%) RCT and four (1%) systematic reviews and meta-analyses were adequately reported. Less than half (n = 40, 49%) of RCTs and just over a quarter (n = 77, 27%) of systematic reviews and meta-analyses had a registered protocol. Surprisingly, we found that more than half of systematic reviews and meta-analyse (n = 162, 56.1%), had no risk of bias assessment.
Conclusions
Recently published RCTs and systematic reviews in pediatric surgery are at high risk of bias and have poor reporting quality. Journals, universities, and research institutions should train authors to conduct and report higher quality studies and develop strategies to reduce risk of bias. However, research with high bias and low reporting does not necessarily lack value.
{"title":"Quality of randomized controlled trials and systematic reviews in pediatric surgery: A cross-sectional meta-research study","authors":"Wilson Jiang, Bill Wang, Sandro Sperandei, Aidan Christopher Tan","doi":"10.1002/cesm.12042","DOIUrl":"https://doi.org/10.1002/cesm.12042","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Background</h3>\u0000 \u0000 <p>There are few randomized controlled trials (RCTs) in pediatric surgery, and their risk of bias is unknown. There is also little known about the methodological or reporting quality of systematic reviews (with or without meta-analyses) in pediatric surgery. Therefore, we conducted a cross-sectional meta-research study to determine the risk of bias and reporting quality of RCTs and systematic reviews and meta-analyses in pediatric surgery, and the associations between these outcomes and study characteristics.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>We searched MEDLINE, Embase, Cochrane Library, JBI EBP Database, Centre for Reviews and Dissemination and Web of Science for all RCTs and systematic reviews in pediatric surgery published in 2021. We also searched the 2021 indexes of high-impact pediatric surgery journals. We assessed the risk of bias and reporting quality of RCTs using the RoB 2 and CONSORT tools respectively. We assessed the same parameters for systematic reviews and meta-analyses using the ROBIS and Preferred Reporting Items for Systematic Reviews and Meta-analyses tools.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Findings</h3>\u0000 \u0000 <p>We found 82 RCTs and 289 systematic reviews/meta-analyses published in 2021. More than half (<i>n</i> = 46, 56%) of RCTs and almost all (n = 278, 96%) systematic reviews and meta-analyses were at high risk of bias. Only one (1%) RCT and four (1%) systematic reviews and meta-analyses were adequately reported. Less than half (<i>n</i> = 40, 49%) of RCTs and just over a quarter (<i>n</i> = 77, 27%) of systematic reviews and meta-analyses had a registered protocol. Surprisingly, we found that more than half of systematic reviews and meta-analyse (<i>n</i> = 162, 56.1%), had no risk of bias assessment.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusions</h3>\u0000 \u0000 <p>Recently published RCTs and systematic reviews in pediatric surgery are at high risk of bias and have poor reporting quality. Journals, universities, and research institutions should train authors to conduct and report higher quality studies and develop strategies to reduce risk of bias. However, research with high bias and low reporting does not necessarily lack value.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"2 2","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-02-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.12042","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139683007","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kathiresan Jeyashree, Rizwan S. Abdulkader, Madhumitha Haridoss, Ranjithkumar Govindaraju, Amanda Brand, Marianne Visser, Sarah Gordon, Hemant Tiwari, T. S. Sumitha, Krupa Chandran, Denny Mabetha, Solange Durão
Background
Menu labeling and portion size control interventions may be effective strategies to mitigate the health risks posed by the out-of-home food environment. We conducted this scoping review to map the body of evidence (BoE) addressing the effects of menu labeling and portion size control interventions in the out-of-home food environment and to summarize the research gaps in this evidence base.
Methods
We searched PubMed, Embase, Epistemonikos, and PROSPERO in phase 1 for systematic reviews (SRs) and PubMed and Embase in phase 2 for primary studies in areas with insufficient SR evidence. We used a comprehensive search strategy without any restrictions on publication date, language, study population characteristics or outcomes. We screened all titles independently and in duplicate. We mapped the number of systematic reviews providing evidence per intervention-setting combination in a matrix. The gaps in the matrix informed the searches for primary studies in phase 2. For the included SR protocols and primary studies, we charted the population, intervention, comparator, outcome, period, and study design to facilitate their evaluation and inclusion in future evidence syntheses.
Results
We included 69 completed SRs; 37 on menu labeling, 9 on portion size control, and 23 on both. The types of menu labeling interventions studied were quantitative nutrient information (74%), interpretational guidance (48%), or contextual guidance (13%). Most reviews were from the United States, United Kingdom, and Canada. Most SRs included studies in establishments like cafeterias (51%) or restaurants (39%) and measured change in the quantity of food offered/ordered/consumed (96%). Phase 2 search yielded 24 primary studies; 16 experimental, 6 quasi-experimental, and 2 observational studies.
Conclusion
The BoE on the effectiveness of menu labeling and portion size control is predominantly from the developed world, on nutrient information labeling and reporting impact on consumer food choice. There is a need for studies in the online environment and reporting distal health outcomes.
{"title":"Menu labeling and portion size control to improve the out-of-home food environment: A scoping review","authors":"Kathiresan Jeyashree, Rizwan S. Abdulkader, Madhumitha Haridoss, Ranjithkumar Govindaraju, Amanda Brand, Marianne Visser, Sarah Gordon, Hemant Tiwari, T. S. Sumitha, Krupa Chandran, Denny Mabetha, Solange Durão","doi":"10.1002/cesm.12039","DOIUrl":"https://doi.org/10.1002/cesm.12039","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Background</h3>\u0000 \u0000 <p>Menu labeling and portion size control interventions may be effective strategies to mitigate the health risks posed by the out-of-home food environment. We conducted this scoping review to map the body of evidence (BoE) addressing the effects of menu labeling and portion size control interventions in the out-of-home food environment and to summarize the research gaps in this evidence base.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>We searched PubMed, Embase, Epistemonikos, and PROSPERO in phase 1 for systematic reviews (SRs) and PubMed and Embase in phase 2 for primary studies in areas with insufficient SR evidence. We used a comprehensive search strategy without any restrictions on publication date, language, study population characteristics or outcomes. We screened all titles independently and in duplicate. We mapped the number of systematic reviews providing evidence per intervention-setting combination in a matrix. The gaps in the matrix informed the searches for primary studies in phase 2. For the included SR protocols and primary studies, we charted the population, intervention, comparator, outcome, period, and study design to facilitate their evaluation and inclusion in future evidence syntheses.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>We included 69 completed SRs; 37 on menu labeling, 9 on portion size control, and 23 on both. The types of menu labeling interventions studied were quantitative nutrient information (74%), interpretational guidance (48%), or contextual guidance (13%). Most reviews were from the United States, United Kingdom, and Canada. Most SRs included studies in establishments like cafeterias (51%) or restaurants (39%) and measured change in the quantity of food offered/ordered/consumed (96%). Phase 2 search yielded 24 primary studies; 16 experimental, 6 quasi-experimental, and 2 observational studies.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusion</h3>\u0000 \u0000 <p>The BoE on the effectiveness of menu labeling and portion size control is predominantly from the developed world, on nutrient information labeling and reporting impact on consumer food choice. There is a need for studies in the online environment and reporting distal health outcomes.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"2 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-01-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.12039","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139550238","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Luis Garegnani, Diego Ivaldi, Mariana Burgos, Gisela Oltra, Camila M. Escobar Liquitay
Introduction
The Cochrane Handbook acknowledges core outcome sets' (COS) relevance in defining review questions and planning systematic reviews. We aimed to assess the uptake of the mechanical ventilation (MV) COS in Cochrane systematic reviews of interventions.
Methods
A cross-sectional study. We included Cochrane systematic reviews and protocols of Cochrane systematic reviews of any intervention related to mechanically ventilated patients through a search in the Cochrane database of systematic reviews. We did not apply restrictions based on age or setting. One reviewer assessed the studies for eligibility and extracted data. Both processes were validated by a second author.
Results
We identified 233 reviews and protocols through our search strategy. We finally included 36 records. Thirty-four (94.44%) were Cochrane reviews and two (5.56%) were protocols. The included Cochrane reviews and protocols assessed a median of 13 (interquartile range [IQR]: 9–17) outcomes, with 35 (97.22%) reviews reporting at least one outcome from the MV COS. The median number of outcomes from the MV COS reported in the Cochrane reviews and protocols was 2.5 (IQR: 2–3). Only one (2.78%) study reported all the outcomes from the MV COS. None of the included Cochrane reviews and protocols cited the MV COS publication.
Conclusion
Completed Cochrane systematic reviews and protocols of Cochrane systematic reviews of interventions related to mechanically ventilated patients have an overall limited uptake of the MV COS so far. Mortality and duration of stay were the most reported outcomes, while extubation and reintubation were the least informed. These findings may serve as a starting point for the Cochrane Critical Care community to develop dissemination strategies for improving the MV COS uptake.
{"title":"Mechanical ventilation core outcome set uptake in Cochrane systematic reviews: A cross-sectional study","authors":"Luis Garegnani, Diego Ivaldi, Mariana Burgos, Gisela Oltra, Camila M. Escobar Liquitay","doi":"10.1002/cesm.12038","DOIUrl":"https://doi.org/10.1002/cesm.12038","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Introduction</h3>\u0000 \u0000 <p>The Cochrane Handbook acknowledges core outcome sets' (COS) relevance in defining review questions and planning systematic reviews. We aimed to assess the uptake of the mechanical ventilation (MV) COS in Cochrane systematic reviews of interventions.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>A cross-sectional study. We included Cochrane systematic reviews and protocols of Cochrane systematic reviews of any intervention related to mechanically ventilated patients through a search in the Cochrane database of systematic reviews. We did not apply restrictions based on age or setting. One reviewer assessed the studies for eligibility and extracted data. Both processes were validated by a second author.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>We identified 233 reviews and protocols through our search strategy. We finally included 36 records. Thirty-four (94.44%) were Cochrane reviews and two (5.56%) were protocols. The included Cochrane reviews and protocols assessed a median of 13 (interquartile range [IQR]: 9–17) outcomes, with 35 (97.22%) reviews reporting at least one outcome from the MV COS. The median number of outcomes from the MV COS reported in the Cochrane reviews and protocols was 2.5 (IQR: 2–3). Only one (2.78%) study reported all the outcomes from the MV COS. None of the included Cochrane reviews and protocols cited the MV COS publication.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusion</h3>\u0000 \u0000 <p>Completed Cochrane systematic reviews and protocols of Cochrane systematic reviews of interventions related to mechanically ventilated patients have an overall limited uptake of the MV COS so far. Mortality and duration of stay were the most reported outcomes, while extubation and reintubation were the least informed. These findings may serve as a starting point for the Cochrane Critical Care community to develop dissemination strategies for improving the MV COS uptake.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"2 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.12038","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139419769","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This tutorial focuses on multiarm studies. We will explain what multiarm studies are and how to include data from them in a meta-analysis.
本教程的重点是多臂研究。我们将解释什么是多臂研究，以及如何将多臂研究的数据纳入荟萃分析。
{"title":"Multiarm studies and how to handle them in a meta-analysis: A tutorial","authors":"Emma Axon, Kerry Dwan, Rachel Richardson","doi":"10.1002/cesm.12033","DOIUrl":"10.1002/cesm.12033","url":null,"abstract":"<p>This tutorial focuses on multiarm studies. We will explain what multiarm studies are and how to include data from them in a meta-analysis.</p>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"1 10","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.12033","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138822500","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
There is increasing concern that a significant proportion of randomized controlled trials (RCTs) included in Cochrane reviews may not be trustworthy. Applying the Cochrane Pregnancy and Childbirth Trustworthiness Screening Tool (CPC-TST) has already had a clinically important effect on several reviews published by the Cochrane Pregnancy and Childbirth Group.
Objectives
We wanted to assess the impact of removing untrustworthy RCTs from already-published Cochrane reviews on a defined clinical area (antenatal and postnatal nutritional interventions).
Methods
We applied the tool to 18 Cochrane reviews (374 RCTs). The tool had four domains: (i) is the research governance trustworthy; (ii) are the baseline characteristics trustworthy; (iii) is the study feasible; (iv) are the results plausible? When additional information was needed, authors were contacted using a standard template. At least two attempts were made to contact the authors. At the end of the evaluation process each study was classified as: (i) included (YES to all questions); (ii) excluded (retracted study); or (iii) awaiting classification (any NO to the questions).
Results
Ninety-three out of 374 included studies (25%) were reclassified as “excluded” or “awaiting classification.” The number of included RCTs was reduced in 14 out of 18 reviews. Six reviews (33%) were judged to require updating because of important differences in the Summary of Findings tables (direction and size of effects and/or GRADE ratings), conclusions, implication for practice, and/or implication for research.
Conclusions
Formal assessment of trustworthiness, and inclusion only of studies that satisfy prespecified criteria for trustworthiness, affect conclusions in a relatively large number of Cochrane reviews, with potentially important clinical implications for practice and research.
{"title":"Trustworthiness assessment as an inclusion criterion for systematic reviews—What is the impact on results?","authors":"Jo Weeks, Anna Cuthbert, Zarko Alfirevic","doi":"10.1002/cesm.12037","DOIUrl":"https://doi.org/10.1002/cesm.12037","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Background</h3>\u0000 \u0000 <p>There is increasing concern that a significant proportion of randomized controlled trials (RCTs) included in Cochrane reviews may not be trustworthy. Applying the Cochrane Pregnancy and Childbirth Trustworthiness Screening Tool (CPC-TST) has already had a clinically important effect on several reviews published by the Cochrane Pregnancy and Childbirth Group.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Objectives</h3>\u0000 \u0000 <p>We wanted to assess the impact of removing untrustworthy RCTs from already-published Cochrane reviews on a defined clinical area (antenatal and postnatal nutritional interventions).</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>We applied the tool to 18 Cochrane reviews (374 RCTs). The tool had four domains: (i) is the research governance trustworthy; (ii) are the baseline characteristics trustworthy; (iii) is the study feasible; (iv) are the results plausible? When additional information was needed, authors were contacted using a standard template. At least two attempts were made to contact the authors. At the end of the evaluation process each study was classified as: (i) included (YES to all questions); (ii) excluded (retracted study); or (iii) awaiting classification (any NO to the questions).</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>Ninety-three out of 374 included studies (25%) were reclassified as “excluded” or “awaiting classification.” The number of included RCTs was reduced in 14 out of 18 reviews. Six reviews (33%) were judged to require updating because of important differences in the Summary of Findings tables (direction and size of effects and/or GRADE ratings), conclusions, implication for practice, and/or implication for research.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusions</h3>\u0000 \u0000 <p>Formal assessment of trustworthiness, and inclusion only of studies that satisfy prespecified criteria for trustworthiness, affect conclusions in a relatively large number of Cochrane reviews, with potentially important clinical implications for practice and research.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"1 10","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.12037","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138634199","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Miranda S. Cumpston, Joanne E. McKenzie, Rebecca Ryan, Ella Flemyng, James Thomas, Sue E. Brennan
Introduction
Methods guidance and appraisal tools for systematic reviews require specification of the question and eligibility criteria for the review (“PICO for the review”). Less emphasis has been given to specifying the question and criteria for each synthesis (“PICO for each synthesis”), yet decisions about which studies to include in each synthesis can critically influence the utility and findings of a review. This paper describes the rationale and methods for developing the InSynQ (Intervention Synthesis Questions) tool for planning and reporting synthesis questions in reviews of interventions. The aim is to provide transparency about the basis of the tool and contribute to evidence on methods for developing guidance for research conduct and reporting.
Methods
Informed by EQUATOR Network methods, we (1) established a project group; (2) examined reporting of the “PICO for each synthesis” in published reviews; (3) reviewed existing reporting guidance and the Cochrane Handbook for Systematic Reviews of Interventions; (4) drafted items with elaboration, explanation, and examples; (5) conducted consultation meetings, an evaluation survey and pilot test; (6) incorporated feedback.
Results
Existing reporting guidelines do not distinguish the review and synthesis PICO, with limited coverage of the elements needed to specify the PICO for each synthesis. Using the PRISMA 2020 format, the draft tool contained 10 items with essential and additional reporting elements, explanations, and examples. Revisions arising from consultation meetings (>30 people), included adding an eleventh item on consumer and stakeholder involvement, a figure explaining PICO for each synthesis, and integrating examples into elements/explanations. All respondents to the survey (12 people) said the tool would help them plan or appraise synthesis questions. InSynQ is available at https://InSynQ.info.
Conclusion
Transparent reporting of the development process contributes to the evidence base for methods to develop guidance. It may improve uptake of InSynQ, in turn enhancing the clarity of syntheses.
{"title":"Development of the InSynQ checklist: A tool for planning and reporting the synthesis questions in systematic reviews of interventions","authors":"Miranda S. Cumpston, Joanne E. McKenzie, Rebecca Ryan, Ella Flemyng, James Thomas, Sue E. Brennan","doi":"10.1002/cesm.12036","DOIUrl":"https://doi.org/10.1002/cesm.12036","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Introduction</h3>\u0000 \u0000 <p>Methods guidance and appraisal tools for systematic reviews require specification of the question and eligibility criteria for the review (“PICO for the review”). Less emphasis has been given to specifying the question and criteria for each synthesis (“PICO for each synthesis”), yet decisions about which studies to include in each synthesis can critically influence the utility and findings of a review. This paper describes the rationale and methods for developing the InSynQ (Intervention Synthesis Questions) tool for planning and reporting synthesis questions in reviews of interventions. The aim is to provide transparency about the basis of the tool and contribute to evidence on methods for developing guidance for research conduct and reporting.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>Informed by EQUATOR Network methods, we (1) established a project group; (2) examined reporting of the “PICO for each synthesis” in published reviews; (3) reviewed existing reporting guidance and the <i>Cochrane Handbook for Systematic Reviews of Interventions</i>; (4) drafted items with elaboration, explanation, and examples; (5) conducted consultation meetings, an evaluation survey and pilot test; (6) incorporated feedback.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>Existing reporting guidelines do not distinguish the review and synthesis PICO, with limited coverage of the elements needed to specify the PICO for each synthesis. Using the PRISMA 2020 format, the draft tool contained 10 items with essential and additional reporting elements, explanations, and examples. Revisions arising from consultation meetings (>30 people), included adding an eleventh item on consumer and stakeholder involvement, a figure explaining PICO for each synthesis, and integrating examples into elements/explanations. All respondents to the survey (12 people) said the tool would help them plan or appraise synthesis questions. InSynQ is available at https://InSynQ.info.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusion</h3>\u0000 \u0000 <p>Transparent reporting of the development process contributes to the evidence base for methods to develop guidance. It may improve uptake of InSynQ, in turn enhancing the clarity of syntheses.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"1 10","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.12036","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138571044","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Theresa H. M. Moore, Julian P. T. Higgins, Kerry Dwan
Introduction
RoB 2 is a tool used by systematic reviewers to assess risk of bias in randomized trials. Over a period of 19 months working as editors for Cochrane, we saw many instances where users of RoB 2 frequently applied the tool in ways the developers had not intended, despite availability of detailed guidance, webinars and FAQs.
Methods
In this paper we highlight the ten main issues that we observed, with the aims of optimising the application of the RoB 2 tool, avoiding some of the frequent misapplications of the tool.
Results
Issues noted included failure to state an effect of interest, applying the tool to an entire study rather than to a specific numerical result, omitting key signaling questions and relying on outdated views of causes of bias.
Conclusion
Such omissions and misapplications can lead to overly harsh or lenient assessments of bias with potential to change the confidence we have in an evidence base of randomized trials. We recommend that teams planning to use RoB 2 include at least one member familiar with the RoB 2 detailed guidance and that they use the free resources, such as webinars and FAQs, from the developers of RoB 2 and Cochrane. Our ten tips should be useful to non-Cochrane systematic reviewers as well as to peer reviewers and editors in Cochrane and other journals.
RoB 2是系统审稿人用来评估随机试验偏倚风险的工具。在作为Cochrane编辑的19个月里，我们看到很多例子，RoB 2的用户经常以开发者意想不到的方式使用该工具，尽管有详细的指导、网络研讨会和常见问题解答。在本文中，我们强调了我们观察到的十个主要问题，目的是优化罗布2工具的应用，避免工具的一些频繁误用。注意到的问题包括未能说明感兴趣的影响，将工具应用于整个研究而不是特定的数字结果，忽略关键的信号问题以及依赖于过时的偏见原因观点。这样的遗漏和误用可能导致过于苛刻或宽松的偏倚评估，有可能改变我们对随机试验证据基础的信心。我们建议计划使用RoB 2的团队包括至少一名熟悉RoB 2详细指导的成员，并且他们使用RoB 2和Cochrane开发人员提供的免费资源，例如网络研讨会和faq。我们的十条建议应该对非Cochrane系统审稿人以及Cochrane和其他期刊的同行审稿人和编辑有用。
{"title":"Ten tips for successful assessment of risk of bias in randomized trials using the RoB 2 tool: Early lessons from Cochrane","authors":"Theresa H. M. Moore, Julian P. T. Higgins, Kerry Dwan","doi":"10.1002/cesm.12031","DOIUrl":"https://doi.org/10.1002/cesm.12031","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Introduction</h3>\u0000 \u0000 <p>RoB 2 is a tool used by systematic reviewers to assess risk of bias in randomized trials. Over a period of 19 months working as editors for Cochrane, we saw many instances where users of RoB 2 frequently applied the tool in ways the developers had not intended, despite availability of detailed guidance, webinars and FAQs.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>In this paper we highlight the ten main issues that we observed, with the aims of optimising the application of the RoB 2 tool, avoiding some of the frequent misapplications of the tool.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>Issues noted included failure to state an effect of interest, applying the tool to an entire study rather than to a specific numerical result, omitting key signaling questions and relying on outdated views of causes of bias.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusion</h3>\u0000 \u0000 <p>Such omissions and misapplications can lead to overly harsh or lenient assessments of bias with potential to change the confidence we have in an evidence base of randomized trials. We recommend that teams planning to use RoB 2 include at least one member familiar with the RoB 2 detailed guidance and that they use the free resources, such as webinars and FAQs, from the developers of RoB 2 and Cochrane. Our ten tips should be useful to non-Cochrane systematic reviewers as well as to peer reviewers and editors in Cochrane and other journals.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"1 10","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.12031","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138480934","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号