Mary Fredlund, Morwenna Rogers, Noreen Orr, Dylan Kneale, Kate Allen, Jo Thompson Coon
Introduction
Clarity on the characteristics of methods used to produce evidence and gap maps (EGMs) will highlight areas where method development is needed to ensure these increasingly produced tools are made following best practice to assure their quality and utility. This paper aims to describe the range, nature and variability of key methodological characteristics of studies publishing EGMs.
Methods
We followed a protocol, written a-prior and informed by PRISMA and MECCIR guidelines for undertaking systematic reviews. We searched nine data bases, from 2010, for studies across any discipline that included details of their methods used to produce an EGM. Search results were screened by two reviewers independently and the subsequent data was extracted and managed according to predefined criteria. We mapped these together with the year of publication and the area of research as the two primary dimensions. We followed established methods for mapping the evidence, including the process of developing the map framework and the filters for our interactive map. We sought input and involvement from stakeholders during this process.
Results
We found 145 studies from nine distinct research areas, with health research accounting for 67%. There were 11 map designs found, of these bubble plots were the most common design, before 2019, since then it has been a matrix map design. Stakeholders were involved in 47.7% of studies, 48.35% of studies stated finding gaps was an aim of their work, 42% reported publishing or registering a protocol and only 9.39% of studies mentioned a plan to update their evidence maps/EGMs.
Discussion/Conclusion
Key areas of methodological development relate to: the involvement of stakeholders, the conceptualization of gaps and the practices for updating maps. The issues of ambiguity in terminology, the flexibility of visualizations of the data and the lack of reporting detail were other aspects that needs further consideration in studies producing an EGM.
{"title":"What are the methodological characteristics of evidence and gap maps? A systematic review and evidence and gap map","authors":"Mary Fredlund, Morwenna Rogers, Noreen Orr, Dylan Kneale, Kate Allen, Jo Thompson Coon","doi":"10.1002/cesm.12096","DOIUrl":"https://doi.org/10.1002/cesm.12096","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Introduction</h3>\u0000 \u0000 <p>Clarity on the characteristics of methods used to produce evidence and gap maps (EGMs) will highlight areas where method development is needed to ensure these increasingly produced tools are made following best practice to assure their quality and utility. This paper aims to describe the range, nature and variability of key methodological characteristics of studies publishing EGMs.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>We followed a protocol, written a-prior and informed by PRISMA and MECCIR guidelines for undertaking systematic reviews. We searched nine data bases, from 2010, for studies across any discipline that included details of their methods used to produce an EGM. Search results were screened by two reviewers independently and the subsequent data was extracted and managed according to predefined criteria. We mapped these together with the year of publication and the area of research as the two primary dimensions. We followed established methods for mapping the evidence, including the process of developing the map framework and the filters for our interactive map. We sought input and involvement from stakeholders during this process.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>We found 145 studies from nine distinct research areas, with health research accounting for 67%. There were 11 map designs found, of these bubble plots were the most common design, before 2019, since then it has been a matrix map design. Stakeholders were involved in 47.7% of studies, 48.35% of studies stated finding gaps was an aim of their work, 42% reported publishing or registering a protocol and only 9.39% of studies mentioned a plan to update their evidence maps/EGMs.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Discussion/Conclusion</h3>\u0000 \u0000 <p>Key areas of methodological development relate to: the involvement of stakeholders, the conceptualization of gaps and the practices for updating maps. The issues of ambiguity in terminology, the flexibility of visualizations of the data and the lack of reporting detail were other aspects that needs further consideration in studies producing an EGM.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"2 8","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.12096","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141967325","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Heather M. R. Ames, Emma F. France, Sara Cooper, Mayara S. Bianchim, Simon Lewin, Bey-Marrié Schmidt, Isabelle Uny, Jane Noyes
Background
Well-conducted qualitative evidence syntheses (QESs) can provide invaluable insights into complex phenomena. However, the development of an in-depth understanding depends on the analysis of rich, thick data from the included primary qualitative studies. Sampling may be needed if there are too many eligible studies. Data richness and thickness are among several criteria that can be taken into consideration when sampling studies for inclusion. However, existing tools do not address explicitly the assessment of both data richness and thickness in the context of QES.
Methods
To address this gap, we have developed, piloted, and conducted initial user testing of a richness and thickness assessment tool. The tool has been in development since 2014. Three pilot versions from three review teams have been used in six Cochrane reviews. Key members from the original three review teams subsequently came together to create a consensus-based definitive version 1 of the tool. Four review authors piloted the version 1 tool, which has been subject to initial user testing. The version 1 assessment tool consists of two components: assessing the thickness of contextual data and assessing the richness of conceptual data. The accompanying guidance emphasizes the importance of assessing data that addresses the review question.
Results
The paper provides guidance on how to apply the tool, emphasizing the importance of reaching a consensus among review authors and fostering a shared understanding of what constitutes rich and thick data in the context of the review. The potential challenges related to the time and resource constraints of this additional review process are acknowledged.
Conclusion
Version 1 of the tool represents a significant development in QES methodology, filling a critical gap and enhancing the transparency and rigor of the sampling process. The authors invite feedback from the research community to further test, refine and improve this tool based on wider user experiences.
{"title":"Assessing qualitative data richness and thickness: Development of an evidence-based tool for use in qualitative evidence synthesis","authors":"Heather M. R. Ames, Emma F. France, Sara Cooper, Mayara S. Bianchim, Simon Lewin, Bey-Marrié Schmidt, Isabelle Uny, Jane Noyes","doi":"10.1002/cesm.12059","DOIUrl":"https://doi.org/10.1002/cesm.12059","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Background</h3>\u0000 \u0000 <p>Well-conducted qualitative evidence syntheses (QESs) can provide invaluable insights into complex phenomena. However, the development of an in-depth understanding depends on the analysis of rich, thick data from the included primary qualitative studies. Sampling may be needed if there are too many eligible studies. Data richness and thickness are among several criteria that can be taken into consideration when sampling studies for inclusion. However, existing tools do not address explicitly the assessment of both data richness and thickness in the context of QES.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>To address this gap, we have developed, piloted, and conducted initial user testing of a richness and thickness assessment tool. The tool has been in development since 2014. Three pilot versions from three review teams have been used in six Cochrane reviews. Key members from the original three review teams subsequently came together to create a consensus-based definitive version 1 of the tool. Four review authors piloted the version 1 tool, which has been subject to initial user testing. The version 1 assessment tool consists of two components: assessing the thickness of contextual data and assessing the richness of conceptual data. The accompanying guidance emphasizes the importance of assessing data that addresses the review question.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>The paper provides guidance on how to apply the tool, emphasizing the importance of reaching a consensus among review authors and fostering a shared understanding of what constitutes rich and thick data in the context of the review. The potential challenges related to the time and resource constraints of this additional review process are acknowledged.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusion</h3>\u0000 \u0000 <p>Version 1 of the tool represents a significant development in QES methodology, filling a critical gap and enhancing the transparency and rigor of the sampling process. The authors invite feedback from the research community to further test, refine and improve this tool based on wider user experiences.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"2 7","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.12059","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141488991","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
John Meisenheimer, Michelle Sobotka, Ronald Yang, Robert P. Dellavalle
Introduction
There is a growing body of evidence that ultraviolet (UV) tanning, whether practiced in indoor tanning salons or outdoors in the sun, is not only linked to detrimental health outcomes but is also addictive through both psychological and physiological mechanisms. In clinical practice, it can be challenging to determine which patients will continue tanning despite being at high risk for developing skin cancer. Our study seeks to identify all available screening questionnaires for tanning addiction that could be used in clinical practice and report on published measures of validity for each screening questionnaire.
Methods
An exhaustive literature search of EMBASE, PubMed, PsycINFO, and Scopus was performed using search criteria including the concepts “UV” and “Addiction.” The most recent search was performed in March 2024 and included all articles from database inception to the time of the search. Studies were included if they reported on screening questionnaires for UV addiction. Articles were excluded from the study if they did not report primary data or did not report on measures of questionnaire validity. Methodology was created using best practices for scoping reviews.
Results
After identifying 171 articles, 106 articles underwent full-text review, and 26 were included in data extraction. We identified nine questionnaires for tanning addiction, with the modified Cut-down, Annoyed, Guilty, Eye-opener (mCAGE), and modified Diagnostic and Statistical Manual of Mental Disorders (mDSM) being most frequently reported on, and the Behavioral Addiction Indoor Tanning Screener (BAITS) being the most promising for future use.
Conclusions
This information should be used to choose questionnaires to be studied against a “gold-standard” of a panel of psychologists. After defining accuracy of diagnostic tests, studies can be designed to examine interventions for treating tanning addiction, so at-risk patients can receive specialized therapy, reducing the overall burden of skin cancers.
{"title":"A scoping review: Screening questionnaires for identifying tanning addiction","authors":"John Meisenheimer, Michelle Sobotka, Ronald Yang, Robert P. Dellavalle","doi":"10.1002/cesm.12092","DOIUrl":"https://doi.org/10.1002/cesm.12092","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Introduction</h3>\u0000 \u0000 <p>There is a growing body of evidence that ultraviolet (UV) tanning, whether practiced in indoor tanning salons or outdoors in the sun, is not only linked to detrimental health outcomes but is also addictive through both psychological and physiological mechanisms. In clinical practice, it can be challenging to determine which patients will continue tanning despite being at high risk for developing skin cancer. Our study seeks to identify all available screening questionnaires for tanning addiction that could be used in clinical practice and report on published measures of validity for each screening questionnaire.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>An exhaustive literature search of EMBASE, PubMed, PsycINFO, and Scopus was performed using search criteria including the concepts “UV” and “Addiction.” The most recent search was performed in March 2024 and included all articles from database inception to the time of the search. Studies were included if they reported on screening questionnaires for UV addiction. Articles were excluded from the study if they did not report primary data or did not report on measures of questionnaire validity. Methodology was created using best practices for scoping reviews.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>After identifying 171 articles, 106 articles underwent full-text review, and 26 were included in data extraction. We identified nine questionnaires for tanning addiction, with the modified Cut-down, Annoyed, Guilty, Eye-opener (mCAGE), and modified Diagnostic and Statistical Manual of Mental Disorders (mDSM) being most frequently reported on, and the Behavioral Addiction Indoor Tanning Screener (BAITS) being the most promising for future use.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusions</h3>\u0000 \u0000 <p>This information should be used to choose questionnaires to be studied against a “gold-standard” of a panel of psychologists. After defining accuracy of diagnostic tests, studies can be designed to examine interventions for treating tanning addiction, so at-risk patients can receive specialized therapy, reducing the overall burden of skin cancers.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"2 7","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.12092","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141488519","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Samer Saadi, Bashar Hasan, Adel Kanaan, Mohamed Abusalih, Zin Tarakji, Mustafa Sadek, Ayla Shamsi Basha, Mohammed Firwana, Zhen Wang, M. Hassan Murad
Background
Risk of bias (RoB) tools are critical in systematic reviews and affect subsequent decision-making. RoB tools should have adequate interrater reliability and interconsensus agreement. We present an approach of post hoc evaluation of RoB tools using duplicated studies that overlap systematic reviews.
Methods
Using a back-citation approach, we identified systematic reviews that used the Risk Of Bias In Nonrandomized Studies-of Interventions (ROBINS-I) tool and retrieved all the included primary studies. We selected studies that were appraised by more than one systematic review and calculated observed agreement and unweighted kappa comparing the different systematic reviews' assessments.
Results
We identified 903 systematic reviews that used the tool with 51,676 cited references, from which we eventually analyzed 171 duplicated studies assessed using ROBINS-I by different systematic reviewers. The observed agreement on ROBINS-I domains ranged from 54.9% (missing data domain) to 70.3% (deviations from intended interventions domain), and was 63.0% for overall RoB assessment of the study. Kappa coefficient ranged from 0.131 (measurement of outcome domain) to 0.396 (domains of confounding and deviations from intended interventions), and was 0.404 for overall RoB assessment of the study.
Conclusion
A post hoc evaluation of RoB tools is feasible by focusing on duplicated studies that overlap systematic review. ROBINS-I assessments demonstrated considerable variation in interconsensus agreement among various systematic reviewes that assessed the same study and outcome, suggesting the need for more intensive upfront work to calibrate systematic reviewers on how to identify context-specific information and agree on how to judge it.
背景偏倚风险（RoB）工具在系统综述中至关重要，并影响后续决策。RoB工具应具有足够的研究者间可靠性和共识性。我们提出了一种利用与系统综述重叠的重复研究对 RoB 工具进行事后评估的方法。 方法 采用反向引用法，我们确定了使用非随机干预研究中的偏倚风险（ROBINS-I）工具的系统综述，并检索了所有纳入的主要研究。我们选择了由一篇以上系统综述评估的研究，并计算了比较不同系统综述评估的观察一致度和非加权卡帕值。 结果 我们确定了 903 篇使用该工具的系统综述，共引用了 51676 篇参考文献，最终从中分析出 171 篇由不同系统综述作者使用 ROBINS-I 评估的重复研究。观察到的 ROBINS-I 领域的一致性从 54.9%（数据缺失领域）到 70.3%（偏离预期干预领域）不等，对研究的整体 RoB 评估的一致性为 63.0%。Kappa 系数从 0.131（结果测量域）到 0.396（混杂域和偏离预期干预域）不等，研究的整体 RoB 评估为 0.404。 结论 通过关注与系统审查重叠的重复研究，对 RoB 工具进行事后评估是可行的。ROBINS-I 评估表明，在评估同一研究和结果的不同系统综述中，共识之间的一致性存在很大差异，这表明需要开展更深入的前期工作，以校准系统综述者如何识别特定背景信息并就如何判断这些信息达成一致。
{"title":"Real-world evaluation of interconsensus agreement of risk of bias tools: A case study using risk of bias in nonrandomized studies-of interventions (ROBINS-I)","authors":"Samer Saadi, Bashar Hasan, Adel Kanaan, Mohamed Abusalih, Zin Tarakji, Mustafa Sadek, Ayla Shamsi Basha, Mohammed Firwana, Zhen Wang, M. Hassan Murad","doi":"10.1002/cesm.12094","DOIUrl":"https://doi.org/10.1002/cesm.12094","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Background</h3>\u0000 \u0000 <p>Risk of bias (RoB) tools are critical in systematic reviews and affect subsequent decision-making. RoB tools should have adequate interrater reliability and interconsensus agreement. We present an approach of post hoc evaluation of RoB tools using duplicated studies that overlap systematic reviews.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>Using a back-citation approach, we identified systematic reviews that used the Risk Of Bias In Nonrandomized Studies-of Interventions (ROBINS-I) tool and retrieved all the included primary studies. We selected studies that were appraised by more than one systematic review and calculated observed agreement and unweighted kappa comparing the different systematic reviews' assessments.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>We identified 903 systematic reviews that used the tool with 51,676 cited references, from which we eventually analyzed 171 duplicated studies assessed using ROBINS-I by different systematic reviewers. The observed agreement on ROBINS-I domains ranged from 54.9% (missing data domain) to 70.3% (deviations from intended interventions domain), and was 63.0% for overall RoB assessment of the study. Kappa coefficient ranged from 0.131 (measurement of outcome domain) to 0.396 (domains of confounding and deviations from intended interventions), and was 0.404 for overall RoB assessment of the study.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusion</h3>\u0000 \u0000 <p>A post hoc evaluation of RoB tools is feasible by focusing on duplicated studies that overlap systematic review. ROBINS-I assessments demonstrated considerable variation in interconsensus agreement among various systematic reviewes that assessed the same study and outcome, suggesting the need for more intensive upfront work to calibrate systematic reviewers on how to identify context-specific information and agree on how to judge it.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"2 7","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.12094","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141488174","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This tutorial provides guidance on creating clear and informative summary of findings tables for systematic reviews of interventions.
本教程指导如何为干预措施的系统综述创建清晰翔实的研究结果摘要表。
{"title":"How to present an informative summary of findings table for systematic reviews of interventions: A tutorial","authors":"Jennifer Hilgart, Clare Miles, Jo-Ana Chase","doi":"10.1002/cesm.12093","DOIUrl":"https://doi.org/10.1002/cesm.12093","url":null,"abstract":"<p>This tutorial provides guidance on creating clear and informative summary of findings tables for systematic reviews of interventions.</p>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"2 6","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.12093","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141435607","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Tamara Lotfi, Vivian Welch, Jordi P. Pardo, Jennifer Petkovic, Shaun Treweek, Andrea J. Darzi, Rebecca Glover, Declan Devane, Meera Viswanathan, Lawrence Mbuagbaw, Kevin Pottie, Elizabeth Kristjansson, Shahab Sayfi, Lara Maxwell, Olivia Magwood, Damian Francis, Dru Riddle, Beverly Shea, Peter Tugwell
In the 2022 Cochrane Lecture [1], Jimmy Volmink, recognized as a pioneer of evidence-based medicine in Africa, challenged Cochrane to enhance its equity efforts and suggested five ways to do so. We, as members of the Campbell and Cochrane Health Equity Thematic group*, fully agree with his suggestions and have developed an actionable plan, described below. We invite the global community to join us in our efforts to meet the equity gaps in research and practice.
Population health and healthcare delivery should be equitable and the research that guides it equity sensitive. By this, we mean that we need to focus on the distribution of health outcomes in the population not just overall health. That is, people should have equal opportunities for health and are not subjected to systemic discrimination or structural barriers to health. It is an ambitious goal and one that many of us who work in healthcare delivery and health research are striving for.
This includes those of us who work in evidence synthesis. Synthesizers of other researchers' evidence may think that our handling of equity cannot be better than the handling of equity in the research we synthesize. We, as members of the Campbell and Cochrane Health Equity Thematic Group, disagree. To truly address inequity, evidence synthesis must take into account equity considerations in a systematic and explicit manner, regardless of how equity was addressed in the original research. We believe that evidence synthesis should lead the way in promoting equity, rather than simply reflecting the approaches taken in the primary research that is included in our reviews (Box 1).
We fully agree that Cochrane cannot succeed in better addressing health equity in systematic reviews without also addressing inequities in its own organization and governance. As members of the Campbell and Cochrane Health Equity Thematic group, we commit to the following actions.
{"title":"Equity in evidence synthesis: You can't play on broken strings","authors":"Tamara Lotfi, Vivian Welch, Jordi P. Pardo, Jennifer Petkovic, Shaun Treweek, Andrea J. Darzi, Rebecca Glover, Declan Devane, Meera Viswanathan, Lawrence Mbuagbaw, Kevin Pottie, Elizabeth Kristjansson, Shahab Sayfi, Lara Maxwell, Olivia Magwood, Damian Francis, Dru Riddle, Beverly Shea, Peter Tugwell","doi":"10.1002/cesm.12091","DOIUrl":"https://doi.org/10.1002/cesm.12091","url":null,"abstract":"<p>In the 2022 Cochrane Lecture [<span>1</span>], Jimmy Volmink, recognized as a pioneer of evidence-based medicine in Africa, challenged Cochrane to enhance its equity efforts and suggested five ways to do so. We, as members of the Campbell and Cochrane Health Equity Thematic group*, fully agree with his suggestions and have developed an actionable plan, described below. We invite the global community to join us in our efforts to meet the equity gaps in research and practice.</p><p>Population health and healthcare delivery should be equitable and the research that guides it equity sensitive. By this, we mean that we need to focus on the distribution of health outcomes in the population not just overall health. That is, people should have equal opportunities for health and are not subjected to systemic discrimination or structural barriers to health. It is an ambitious goal and one that many of us who work in healthcare delivery and health research are striving for.</p><p>This includes those of us who work in evidence synthesis. Synthesizers of other researchers' evidence may think that our handling of equity cannot be better than the handling of equity in the research we synthesize. We, as members of the Campbell and Cochrane Health Equity Thematic Group, disagree. To truly address inequity, evidence synthesis must take into account equity considerations in a systematic and explicit manner, regardless of how equity was addressed in the original research. We believe that evidence synthesis should lead the way in promoting equity, rather than simply reflecting the approaches taken in the primary research that is included in our reviews (Box 1).</p><p>We fully agree that Cochrane cannot succeed in better addressing health equity in systematic reviews without also addressing inequities in its own organization and governance. As members of the Campbell and Cochrane Health Equity Thematic group, we commit to the following actions.</p><p><b>Tamara Lotfi</b>: Conceptualization, writing—original draft, reviewing and editing. <b>Vivian Welch</b>: Conceptualization, writing—original draft, review & editing. <b>Jordi P. Pardo</b>: Conceptualization, writing—original draft, review & editing. <b>Jennifer Petkovic</b>: Conceptualization, writing—original draft, review & editing. <b>Shaun Treweek</b>: Writing—review & editing. <b>Andrea Darzi</b>: Writing—review & editing. <b>Rebecca Glover</b>: Writing—review & editing. <b>Declan Devane</b>: Writing—review & editing. <b>Meera Viswanathan</b>: Writing—review & editing. <b>Lawrence Mbuagbaw</b>: Writing—review & editing. <b>Kevin Pottie</b>: Writing—review & editing. <b>Elizabeth Kristjansson</b>: Writing—review & editing. <b>Shahab Sayfi</b>: Writing—review & editing. <b>Lara Maxwell</b>: Writing—review & editing. <b>Olivia Magwood</b>: Writing—review & editing. <b>Damian Francis</b>: Writing—review & editing. <b>Dru Riddle</b>: Writing—review & ed","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"2 6","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.12091","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141425124","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jessica Power, Sara Dada, Andrew Booth, Aoife De Brún, Brynne Gilmore
Introduction
Realist reviews may involve groups or panels external to the research team who provide external and independent perspectives informing the review based on their experience of the topic area. These panels or groups are termed in this study as an “advisory group.” This study aims to map current practice of advisory groups in realist reviews and provide guidance for planning and reporting.
Methods
A “best-fit” framework synthesis methodology was used by first searching for a best-fit framework and then conducting a systematic search to identify a sample of realist reviews and rapid realist reviews (RRRs) from the most recent year, 2021. Nine databases were searched: CINAHL Complete, Cochrane, Embase, ERIC, MEDLINE, PsycInfo, Social Services Abstracts, Sociological Abstracts, and Web of Science Core Collection. Screening and data extraction was conducted by two researchers. The chosen best-fit framework (ACTIVE framework) informed the data extraction tool.
Results
One hundred and seven reviews (93 realist reviews, 14 RRRs) were identified for inclusion. Of these, 40% (n = 37) of realist reviews and 71.5% (n = 10) of RRRs mentioned use of an advisory group, though there was considerable variation in terminology used. Individuals in advisory groups were involved at varying stages of the review and tended to bring experience in the topic area from the perspective of (i) a lived experience, that is, patients, carers, family members (n = 15 realist reviews; n = 4 RRRs); (ii) professional experience, such as healthcare professionals (n = 20 realist reviews; n = 6 RRRs); or (iii) policy or research experience in the topic area (n = 19 realist reviews; n = 7 RRRs).
Conclusions
This study proposes a definition of advisory groups, considerations for advisory group use, and suggested items for reporting. The purpose of the advisory group should be carefully considered when deciding on their use in a realist review.
{"title":"Advisory groups in realist reviews: Systematically mapping current research and recommendations for practice","authors":"Jessica Power, Sara Dada, Andrew Booth, Aoife De Brún, Brynne Gilmore","doi":"10.1002/cesm.12073","DOIUrl":"https://doi.org/10.1002/cesm.12073","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Introduction</h3>\u0000 \u0000 <p>Realist reviews may involve groups or panels external to the research team who provide external and independent perspectives informing the review based on their experience of the topic area. These panels or groups are termed in this study as an “advisory group.” This study aims to map current practice of advisory groups in realist reviews and provide guidance for planning and reporting.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>A “best-fit” framework synthesis methodology was used by first searching for a best-fit framework and then conducting a systematic search to identify a sample of realist reviews and rapid realist reviews (RRRs) from the most recent year, 2021. Nine databases were searched: CINAHL Complete, Cochrane, Embase, ERIC, MEDLINE, PsycInfo, Social Services Abstracts, Sociological Abstracts, and Web of Science Core Collection. Screening and data extraction was conducted by two researchers. The chosen best-fit framework (ACTIVE framework) informed the data extraction tool.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>One hundred and seven reviews (93 realist reviews, 14 RRRs) were identified for inclusion. Of these, 40% (<i>n</i> = 37) of realist reviews and 71.5% (<i>n</i> = 10) of RRRs mentioned use of an advisory group, though there was considerable variation in terminology used. Individuals in advisory groups were involved at varying stages of the review and tended to bring experience in the topic area from the perspective of (i) a lived experience, that is, patients, carers, family members (<i>n</i> = 15 realist reviews; <i>n</i> = 4 RRRs); (ii) professional experience, such as healthcare professionals (<i>n</i> = 20 realist reviews; <i>n</i> = 6 RRRs); or (iii) policy or research experience in the topic area (<i>n</i> = 19 realist reviews; <i>n</i> = 7 RRRs).</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusions</h3>\u0000 \u0000 <p>This study proposes a definition of advisory groups, considerations for advisory group use, and suggested items for reporting. The purpose of the advisory group should be carefully considered when deciding on their use in a realist review.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"2 6","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.12073","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141304224","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Claudia Kapp, Naomi Fujita-Rohwerder, Jona Lilienthal, Wiebke Sieben, Siw Waffenschmidt, Elke Hausner
Introduction
One of the main tasks in information retrieval is the development of Boolean search strategies for systematic searches in bibliographic databases. This includes the identification of free-text terms and controlled vocabulary. IQWiG has previously implemented its objective approach for search strategy development using a fee-based text analysis software. However, this implementation is not fully automated, due to a lack of technical options. The aim of our project was to develop a text analysis tool for the development of Boolean search strategies using R.
Methods
We adopt an incremental approach to software development, with the first goal being to develop a minimum viable product for the previously defined use cases. To create an interactive user interface, we use the shiny framework.
Results
Our newly developed shiny app searchbuildR is a text analysis tool with a point-and-click user interface, that automatically extracts and ranks terms from titles, abstracts, and MeSH terms of a given test set of PubMed records. It returns searchable, interactive tables of free-text and MeSH terms. Each free-text term can also be viewed within its original context in the full titles and abstracts or in a user-defined word window. In addition, 2-word combinations are extracted and also provided as an interactive table to help the user identify free-text term combinations, that can be searched with proximity operators in Boolean searches. The results can be exported to a CSV file. The new implementation with searchbuildR was evaluated by validating the text analysis results against the results of the previously used fee-based software.
Conclusions
QWiG has developed the shiny app searchbuildR to support the development of search strategies in systematic reviews. It is open source and can be used by researchers and other information specialists without extensive R or programming skills. The package code is openly available on GitHub at www.github.com/IQWiG/searchbuildR.
{"title":"The searchbuildR shiny app: A new implementation of the objective approach for search strategy development in systematic reviews","authors":"Claudia Kapp, Naomi Fujita-Rohwerder, Jona Lilienthal, Wiebke Sieben, Siw Waffenschmidt, Elke Hausner","doi":"10.1002/cesm.12078","DOIUrl":"https://doi.org/10.1002/cesm.12078","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Introduction</h3>\u0000 \u0000 <p>One of the main tasks in information retrieval is the development of Boolean search strategies for systematic searches in bibliographic databases. This includes the identification of free-text terms and controlled vocabulary. IQWiG has previously implemented its objective approach for search strategy development using a fee-based text analysis software. However, this implementation is not fully automated, due to a lack of technical options. The aim of our project was to develop a text analysis tool for the development of Boolean search strategies using R.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>We adopt an incremental approach to software development, with the first goal being to develop a minimum viable product for the previously defined use cases. To create an interactive user interface, we use the shiny framework.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>Our newly developed shiny app searchbuildR is a text analysis tool with a point-and-click user interface, that automatically extracts and ranks terms from titles, abstracts, and MeSH terms of a given test set of PubMed records. It returns searchable, interactive tables of free-text and MeSH terms. Each free-text term can also be viewed within its original context in the full titles and abstracts or in a user-defined word window. In addition, 2-word combinations are extracted and also provided as an interactive table to help the user identify free-text term combinations, that can be searched with proximity operators in Boolean searches. The results can be exported to a CSV file. The new implementation with searchbuildR was evaluated by validating the text analysis results against the results of the previously used fee-based software.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusions</h3>\u0000 \u0000 <p>QWiG has developed the shiny app searchbuildR to support the development of search strategies in systematic reviews. It is open source and can be used by researchers and other information specialists without extensive R or programming skills. The package code is openly available on GitHub at www.github.com/IQWiG/searchbuildR.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"2 6","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.12078","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141304225","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Heather M. Munthe-Kaas, Andrew Booth, Isolde Sommer, Sara Cooper, Ruth Garside, Karin Hannes, Jane Noyes, The CAMELOT Development Group
Introduction
Qualitative evidence is increasingly incorporated into decision-making processes. Assessing the methodological limitations of primary studies is critical to making an overall assessment of confidence in findings from qualitative evidence syntheses (QES) using GRADE-CERQual. Current critical appraisal tools were not developed specifically for use in Cochrane reviews or GRADE-CERQual, and few are evidence-based. The aim of CochrAne qualitative Methodological LimitatiOns Tool (CAMELOT) was to address this gap.
Methods
We undertook this project in four stages: (1) systematic literature search to identify existing tools, (2) identification of evidence to support inclusion of potential CAMELOT domains (3) consensus survey to agree on the inclusion and definition of CAMELOT domains, and (4) human-centered design approach to develop and refine CAMELOT by exploring user experience.
Results
CAMELOT is a new evidence-based tool for assessing the methodological strengths and limitations of primary qualitative research studies in a QES. CAMELOT is comprised of 12 domains: four Meta domains that encourage review authors to consider those characteristics of the primary study that are beyond how the study was carried out, but which inform the conduct and design of the study, and eight Method domains which encourage review authors to consider how the study was designed, planned and/or conducted, and how study conduct and design fits with the information provided in the four meta domains. Review authors make an assessment by identifying any concerns regarding the methods used in the study and considering the appropriateness of fit between the Meta and Method domains.
Conclusion
CAMELOT provides review authors with a transparent and systematic method to assess methodological limitations of primary qualitative studies. CAMELOT incorporates qualitative principles and focuses on appropriateness of fit between Meta and Method domains. In line with iterative tool development approach, CAMELOT will continue to be revised over time following extensive user testing and piloting.
{"title":"Developing CAMELOT for assessing methodological limitations of qualitative research for inclusion in qualitative evidence syntheses","authors":"Heather M. Munthe-Kaas, Andrew Booth, Isolde Sommer, Sara Cooper, Ruth Garside, Karin Hannes, Jane Noyes, The CAMELOT Development Group","doi":"10.1002/cesm.12058","DOIUrl":"https://doi.org/10.1002/cesm.12058","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Introduction</h3>\u0000 \u0000 <p>Qualitative evidence is increasingly incorporated into decision-making processes. Assessing the methodological limitations of primary studies is critical to making an overall assessment of confidence in findings from qualitative evidence syntheses (QES) using GRADE-CERQual. Current critical appraisal tools were not developed specifically for use in Cochrane reviews or GRADE-CERQual, and few are evidence-based. The aim of CochrAne qualitative Methodological LimitatiOns Tool (CAMELOT) was to address this gap.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>We undertook this project in four stages: (1) systematic literature search to identify existing tools, (2) identification of evidence to support inclusion of potential CAMELOT domains (3) consensus survey to agree on the inclusion and definition of CAMELOT domains, and (4) human-centered design approach to develop and refine CAMELOT by exploring user experience.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>CAMELOT is a new evidence-based tool for assessing the methodological strengths and limitations of primary qualitative research studies in a QES. CAMELOT is comprised of 12 domains: four <i>Meta domains</i> that encourage review authors to consider those characteristics of the primary study that are beyond how the study was carried out, but which inform the conduct and design of the study, and eight <i>Method domains</i> which encourage review authors to consider how the study was designed, planned and/or conducted, and how study conduct and design fits with the information provided in the four meta domains. Review authors make an assessment by identifying any concerns regarding the methods used in the study and considering the appropriateness of fit between the <i>Meta</i> and <i>Method domains</i>.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusion</h3>\u0000 \u0000 <p>CAMELOT provides review authors with a transparent and systematic method to assess methodological limitations of primary qualitative studies. CAMELOT incorporates qualitative principles and focuses on appropriateness of fit between <i>Meta</i> and <i>Method domains</i>. In line with iterative tool development approach, CAMELOT will continue to be revised over time following extensive user testing and piloting.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"2 6","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.12058","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141294962","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In Equation 4 (computation of the standard error [SE] of the mean difference [MD] at posttreatment time point), we wrote SD instead of SE, which is the correct one (see attached figure below).
{"title":"Correction to “Standardized mean differences in meta-analysis: A tutorial”","authors":"","doi":"10.1002/cesm.12076","DOIUrl":"https://doi.org/10.1002/cesm.12076","url":null,"abstract":"<p>In Equation 4 (computation of the standard error [SE] of the mean difference [MD] at posttreatment time point), we wrote SD instead of SE, which is the correct one (see attached figure below).</p><p></p><p>We apologize for this error.</p>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"2 6","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.12076","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141245517","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号