Alexandra M Bodnaruc , Hassan Khan , Nicole Shaver , Alexandria Bennett , Yiu Lin Wong , Catherine Gracey , Valentina Ly , Beverley Shea , Julian Little , Melissa Brouwers , Dennis Bier , David Moher
{"title":"为《2020-2025 年美国人膳食指南》提供信息的系统综述的可靠性和再现性：一项试点研究。","authors":"Alexandra M Bodnaruc , Hassan Khan , Nicole Shaver , Alexandria Bennett , Yiu Lin Wong , Catherine Gracey , Valentina Ly , Beverley Shea , Julian Little , Melissa Brouwers , Dennis Bier , David Moher","doi":"10.1016/j.ajcnut.2024.10.013","DOIUrl":null,"url":null,"abstract":"<div><h3>Background</h3><div>Although high-quality nutrition systematic reviews (SRs) are important for clinical decision making, there remains debate on their methodological quality and reporting transparency.</div></div><div><h3>Objectives</h3><div>The objective of this study was to assess the reliability and reproducibility of a sample of SRs produced by the Nutrition Evidence Systematic Review (NESR) team to inform the 2020–2025 Dietary Guidelines for Americans (DGAs).</div></div><div><h3>Methods</h3><div>We evaluated a sample of 8 SRs from the DGA dietary patterns subcommittee for methodological quality using the Assessment of Multiple Systematic Reviews 2 (AMSTAR 2) tool and for reporting transparency using the PRISMA 2020 and PRISMA literature search extension (PRISMA-S) checklists. We assessed the quality and reproducibility of the original search strategy of one selected SR using the Peer Review of Electronic Search Strategies checklist. The reporting transparency of the SR’s narrative data synthesis was assessed using the Synthesis Without Meta-Analysis (SWiM) checklist. Interpretation bias was evaluated using existing spin bias classifications in systematic reviews.</div></div><div><h3>Results</h3><div>The AMSTAR 2 assessment identified critical methodological weaknesses, and all included SRs were judged to be of critically low quality. Overall, 74% of the PRISMA 2020 checklist items and 63% of the PRISMA-S checklist items were satisfactorily fulfilled. We identified several errors and inconsistencies in the search strategy and could not reproduce searches within a 10% margin of the original results. The SWiM assessment identified concerns regarding the reporting transparency of the narrative data synthesis, but the spin bias assessment revealed no evidence of interpretation bias.</div></div><div><h3>Conclusions</h3><div>Several methodological quality and reporting concerns were identified, which could lead to reliability and reproducibility issues should a full reproduction attempt be made. However, additional research is needed to confirm the impact of these findings on conclusions statements and their generalizability across the NESR team SRs.</div><div>This study was registered in the Open Science Framework (<span><span>https://osf.io/ns6a9/</span><svg><path></path></svg></span>).</div></div>","PeriodicalId":50813,"journal":{"name":"American Journal of Clinical Nutrition","volume":"121 1","pages":"Pages 111-124"},"PeriodicalIF":6.5000,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11747194/pdf/","citationCount":"0","resultStr":"{\"title\":\"Reliability and reproducibility of systematic reviews informing the 2020–2025 Dietary Guidelines for Americans: a pilot study\",\"authors\":\"Alexandra M Bodnaruc , Hassan Khan , Nicole Shaver , Alexandria Bennett , Yiu Lin Wong , Catherine Gracey , Valentina Ly , Beverley Shea , Julian Little , Melissa Brouwers , Dennis Bier , David Moher\",\"doi\":\"10.1016/j.ajcnut.2024.10.013\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><h3>Background</h3><div>Although high-quality nutrition systematic reviews (SRs) are important for clinical decision making, there remains debate on their methodological quality and reporting transparency.</div></div><div><h3>Objectives</h3><div>The objective of this study was to assess the reliability and reproducibility of a sample of SRs produced by the Nutrition Evidence Systematic Review (NESR) team to inform the 2020–2025 Dietary Guidelines for Americans (DGAs).</div></div><div><h3>Methods</h3><div>We evaluated a sample of 8 SRs from the DGA dietary patterns subcommittee for methodological quality using the Assessment of Multiple Systematic Reviews 2 (AMSTAR 2) tool and for reporting transparency using the PRISMA 2020 and PRISMA literature search extension (PRISMA-S) checklists. We assessed the quality and reproducibility of the original search strategy of one selected SR using the Peer Review of Electronic Search Strategies checklist. The reporting transparency of the SR’s narrative data synthesis was assessed using the Synthesis Without Meta-Analysis (SWiM) checklist. Interpretation bias was evaluated using existing spin bias classifications in systematic reviews.</div></div><div><h3>Results</h3><div>The AMSTAR 2 assessment identified critical methodological weaknesses, and all included SRs were judged to be of critically low quality. Overall, 74% of the PRISMA 2020 checklist items and 63% of the PRISMA-S checklist items were satisfactorily fulfilled. We identified several errors and inconsistencies in the search strategy and could not reproduce searches within a 10% margin of the original results. The SWiM assessment identified concerns regarding the reporting transparency of the narrative data synthesis, but the spin bias assessment revealed no evidence of interpretation bias.</div></div><div><h3>Conclusions</h3><div>Several methodological quality and reporting concerns were identified, which could lead to reliability and reproducibility issues should a full reproduction attempt be made. However, additional research is needed to confirm the impact of these findings on conclusions statements and their generalizability across the NESR team SRs.</div><div>This study was registered in the Open Science Framework (<span><span>https://osf.io/ns6a9/</span><svg><path></path></svg></span>).</div></div>\",\"PeriodicalId\":50813,\"journal\":{\"name\":\"American Journal of Clinical Nutrition\",\"volume\":\"121 1\",\"pages\":\"Pages 111-124\"},\"PeriodicalIF\":6.5000,\"publicationDate\":\"2025-01-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11747194/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"American Journal of Clinical Nutrition\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0002916524008207\",\"RegionNum\":1,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"NUTRITION & DIETETICS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"American Journal of Clinical Nutrition","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0002916524008207","RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"NUTRITION & DIETETICS","Score":null,"Total":0}
Reliability and reproducibility of systematic reviews informing the 2020–2025 Dietary Guidelines for Americans: a pilot study
Background
Although high-quality nutrition systematic reviews (SRs) are important for clinical decision making, there remains debate on their methodological quality and reporting transparency.
Objectives
The objective of this study was to assess the reliability and reproducibility of a sample of SRs produced by the Nutrition Evidence Systematic Review (NESR) team to inform the 2020–2025 Dietary Guidelines for Americans (DGAs).
Methods
We evaluated a sample of 8 SRs from the DGA dietary patterns subcommittee for methodological quality using the Assessment of Multiple Systematic Reviews 2 (AMSTAR 2) tool and for reporting transparency using the PRISMA 2020 and PRISMA literature search extension (PRISMA-S) checklists. We assessed the quality and reproducibility of the original search strategy of one selected SR using the Peer Review of Electronic Search Strategies checklist. The reporting transparency of the SR’s narrative data synthesis was assessed using the Synthesis Without Meta-Analysis (SWiM) checklist. Interpretation bias was evaluated using existing spin bias classifications in systematic reviews.
Results
The AMSTAR 2 assessment identified critical methodological weaknesses, and all included SRs were judged to be of critically low quality. Overall, 74% of the PRISMA 2020 checklist items and 63% of the PRISMA-S checklist items were satisfactorily fulfilled. We identified several errors and inconsistencies in the search strategy and could not reproduce searches within a 10% margin of the original results. The SWiM assessment identified concerns regarding the reporting transparency of the narrative data synthesis, but the spin bias assessment revealed no evidence of interpretation bias.
Conclusions
Several methodological quality and reporting concerns were identified, which could lead to reliability and reproducibility issues should a full reproduction attempt be made. However, additional research is needed to confirm the impact of these findings on conclusions statements and their generalizability across the NESR team SRs.
This study was registered in the Open Science Framework (https://osf.io/ns6a9/).
期刊介绍：
American Journal of Clinical Nutrition is recognized as the most highly rated peer-reviewed, primary research journal in nutrition and dietetics.It focuses on publishing the latest research on various topics in nutrition, including but not limited to obesity, vitamins and minerals, nutrition and disease, and energy metabolism.
Purpose:
The purpose of AJCN is to:
Publish original research studies relevant to human and clinical nutrition.
Consider well-controlled clinical studies describing scientific mechanisms, efficacy, and safety of dietary interventions in the context of disease prevention or health benefits.
Encourage public health and epidemiologic studies relevant to human nutrition.
Promote innovative investigations of nutritional questions employing epigenetic, genomic, proteomic, and metabolomic approaches.
Include solicited editorials, book reviews, solicited or unsolicited review articles, invited controversy position papers, and letters to the Editor related to prior AJCN articles.
Peer Review Process:
All submitted material with scientific content undergoes peer review by the Editors or their designees before acceptance for publication.
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
