Cathalijn H C Leenaars, Frans R Stafleu, Christine Häger, André Bleich
{"title":"系统性综述的偏倚风险和报告质量评估的信息价值案例研究。","authors":"Cathalijn H C Leenaars, Frans R Stafleu, Christine Häger, André Bleich","doi":"10.1186/s13643-024-02650-w","DOIUrl":null,"url":null,"abstract":"<p><p>While undisputedly important, and part of any systematic review (SR) by definition, evaluation of the risk of bias within the included studies is one of the most time-consuming parts of performing an SR. In this paper, we describe a case study comprising an extensive analysis of risk of bias (RoB) and reporting quality (RQ) assessment from a previously published review (CRD42021236047). It included both animal and human studies, and the included studies compared baseline diseased subjects with controls, assessed the effects of investigational treatments, or both. We compared RoB and RQ between the different types of included primary studies. We also assessed the \"informative value\" of each of the separate elements for meta-researchers, based on the notion that variation in reporting may be more interesting for the meta-researcher than consistently high/low or reported/non-reported scores. In general, reporting of experimental details was low. This resulted in frequent unclear risk-of-bias scores. We observed this both for animal and for human studies and both for disease-control comparisons and investigations of experimental treatments. Plots and explorative chi-square tests showed that reporting was slightly better for human studies of investigational treatments than for the other study types. With the evidence reported as is, risk-of-bias assessments for systematic reviews have low informative value other than repeatedly showing that reporting of experimental details needs to improve in all kinds of in vivo research. Particularly for reviews that do not directly inform treatment decisions, it could be efficient to perform a thorough but partial assessment of the quality of the included studies, either of a random subset of the included publications or of a subset of relatively informative elements, comprising, e.g. ethics evaluation, conflicts of interest statements, study limitations, baseline characteristics, and the unit of analysis. This publication suggests several potential procedures.</p>","PeriodicalId":22162,"journal":{"name":"Systematic Reviews","volume":null,"pages":null},"PeriodicalIF":6.3000,"publicationDate":"2024-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11380326/pdf/","citationCount":"0","resultStr":"{\"title\":\"A case study of the informative value of risk of bias and reporting quality assessments for systematic reviews.\",\"authors\":\"Cathalijn H C Leenaars, Frans R Stafleu, Christine Häger, André Bleich\",\"doi\":\"10.1186/s13643-024-02650-w\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><p>While undisputedly important, and part of any systematic review (SR) by definition, evaluation of the risk of bias within the included studies is one of the most time-consuming parts of performing an SR. In this paper, we describe a case study comprising an extensive analysis of risk of bias (RoB) and reporting quality (RQ) assessment from a previously published review (CRD42021236047). It included both animal and human studies, and the included studies compared baseline diseased subjects with controls, assessed the effects of investigational treatments, or both. We compared RoB and RQ between the different types of included primary studies. We also assessed the \\\"informative value\\\" of each of the separate elements for meta-researchers, based on the notion that variation in reporting may be more interesting for the meta-researcher than consistently high/low or reported/non-reported scores. In general, reporting of experimental details was low. This resulted in frequent unclear risk-of-bias scores. We observed this both for animal and for human studies and both for disease-control comparisons and investigations of experimental treatments. Plots and explorative chi-square tests showed that reporting was slightly better for human studies of investigational treatments than for the other study types. With the evidence reported as is, risk-of-bias assessments for systematic reviews have low informative value other than repeatedly showing that reporting of experimental details needs to improve in all kinds of in vivo research. Particularly for reviews that do not directly inform treatment decisions, it could be efficient to perform a thorough but partial assessment of the quality of the included studies, either of a random subset of the included publications or of a subset of relatively informative elements, comprising, e.g. ethics evaluation, conflicts of interest statements, study limitations, baseline characteristics, and the unit of analysis. This publication suggests several potential procedures.</p>\",\"PeriodicalId\":22162,\"journal\":{\"name\":\"Systematic Reviews\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":6.3000,\"publicationDate\":\"2024-09-07\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11380326/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Systematic Reviews\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1186/s13643-024-02650-w\",\"RegionNum\":4,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"MEDICINE, GENERAL & INTERNAL\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Systematic Reviews","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1186/s13643-024-02650-w","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"MEDICINE, GENERAL & INTERNAL","Score":null,"Total":0}
引用次数: 0
摘要
对纳入研究的偏倚风险进行评估是进行系统综述(SR)最耗时的部分之一,这一点毋庸置疑,而且也是任何系统综述(SR)的一部分。在本文中,我们介绍了一项案例研究,其中包括对之前发表的一篇综述(CRD42021236047)的偏倚风险(RoB)和报告质量(RQ)评估的广泛分析。该研究既包括动物研究,也包括人体研究,所纳入的研究将基线患病对象与对照组进行了比较,或评估了研究性治疗的效果,或两者兼而有之。我们比较了纳入的不同类型主要研究的 RoB 和 RQ。我们还评估了每项独立要素对元研究者的 "信息价值",因为元研究者认为报告中的差异可能比一致的高分/低分或已报告/未报告得分更有意义。一般来说,实验细节的报告较少。这导致偏倚风险评分经常不明确。我们在动物研究和人体研究中,以及在疾病对照比较和实验治疗调查中都发现了这种情况。曲线图和探索性的卡方检验表明,与其他研究类型相比,人类试验性治疗研究的报告情况略好。根据目前报告的证据,对系统综述进行偏倚风险评估的信息价值很低,只能反复表明在所有类型的体内研究中,实验细节的报告都需要改进。特别是对于不直接为治疗决策提供信息的综述,对所纳入研究的质量进行全面但不全面的评估可能会很有效,评估可以是对所纳入出版物的随机子集进行评估,也可以是对相对有参考价值的要素的子集进行评估,这些要素包括伦理评估、利益冲突声明、研究限制、基线特征和分析单位等。本出版物提出了几种可能的程序。
A case study of the informative value of risk of bias and reporting quality assessments for systematic reviews.
While undisputedly important, and part of any systematic review (SR) by definition, evaluation of the risk of bias within the included studies is one of the most time-consuming parts of performing an SR. In this paper, we describe a case study comprising an extensive analysis of risk of bias (RoB) and reporting quality (RQ) assessment from a previously published review (CRD42021236047). It included both animal and human studies, and the included studies compared baseline diseased subjects with controls, assessed the effects of investigational treatments, or both. We compared RoB and RQ between the different types of included primary studies. We also assessed the "informative value" of each of the separate elements for meta-researchers, based on the notion that variation in reporting may be more interesting for the meta-researcher than consistently high/low or reported/non-reported scores. In general, reporting of experimental details was low. This resulted in frequent unclear risk-of-bias scores. We observed this both for animal and for human studies and both for disease-control comparisons and investigations of experimental treatments. Plots and explorative chi-square tests showed that reporting was slightly better for human studies of investigational treatments than for the other study types. With the evidence reported as is, risk-of-bias assessments for systematic reviews have low informative value other than repeatedly showing that reporting of experimental details needs to improve in all kinds of in vivo research. Particularly for reviews that do not directly inform treatment decisions, it could be efficient to perform a thorough but partial assessment of the quality of the included studies, either of a random subset of the included publications or of a subset of relatively informative elements, comprising, e.g. ethics evaluation, conflicts of interest statements, study limitations, baseline characteristics, and the unit of analysis. This publication suggests several potential procedures.
期刊介绍:
Systematic Reviews encompasses all aspects of the design, conduct and reporting of systematic reviews. The journal publishes high quality systematic review products including systematic review protocols, systematic reviews related to a very broad definition of health, rapid reviews, updates of already completed systematic reviews, and methods research related to the science of systematic reviews, such as decision modelling. At this time Systematic Reviews does not accept reviews of in vitro studies. The journal also aims to ensure that the results of all well-conducted systematic reviews are published, regardless of their outcome.