BMJ Evidence-Based Medicine最新文献

Objective: The purpose of this study is to validate the taxonomy and framework using Chinese guidelines and identify actionable statements.

Design and setting: We searched five databases, to identify the health guidelines from 1 January 2020 to 1 May 2023. Five researchers categorised statements into six types: formal recommendations (Type I) with clear direction and strength, with explicit and direct evidence; good practice statements (GPS) (Type II), actionable in isolation with a significant benefit; remarks (Type III), an inseparable unit belonging to a formal recommendation or GPS that provides additional clarification; research only recommendations (Type IV) for specific populations; implementation considerations, tools and tips (Type V), that describe the how, who, where, what and when, in relation to implementing a recommendation and lacking a direct evidence link; and informal recommendations (Type VI), unrelated to evidence and not meeting GPS criteria.

Results: We included 116 guidelines, including 74 Western medicine guidelines, 12 traditional Chinese medicine guidelines and 30 integrated Chinese and Western medicine guidelines. 99 guidelines (85.3%) used the Grading of Recommendations Assessment, Development and Evaluation criteria. Medical specialty societies developed the highest number of guidelines (53.4%). Of all the statements, 4422 statements were extracted from the guidelines. Among them, 2154 (48.7%) were formal recommendations, 197 (4.4%) were GPS, 394 (8.9%) were remarks, 16 (0.4%) were research only recommendations, 1106 (25.0%) were implementation considerations, tools and tips, and 555 (12.6%) were informal recommendations.

Conclusions: Up to date, the Chinese guideline developers tend to overestimate the number of formal recommendations and underestimate the number of GPS, remarks, research only recommendations, implementation considerations, tools and tips, and informal recommendations. Thus the current quality of actionable statements in Chinese health guidelines requires further enhancement.

Objective: To assess the therapeutic quality of exercise interventions delivered in chronic low back pain (cLBP) trials using the international Consensus on Therapeutic Exercise aNd Training (i-CONTENT) tool and its inter-rater agreement.

Methods: We performed a meta-research study, starting from the trials' arms included in the published Cochrane review (2021) 'Exercise therapy for chronic low back pain'. Two pairs of independent reviewers applied the i-CONTENT tool, a standardised tool designed to ensure the quality of exercise therapy intervention, in a random sample of 100 different exercise arms. We assessed the inter-rater agreement of each category calculating the specific agreement. A percentage of 70% was considered satisfactory.

Results: We included 100 arms from 68 randomised controlled trials published between 1991 and 2019. The most assessed exercise types were core strengthening (n=27 arms) and motor control (n=13 arms). Among alternative approaches, yoga (n=11) and Pilates (n=7) were the most representative. Overall, most exercise interventions were rated as having a low risk of ineffectiveness for patient selection (100%), exercise type (92%), outcome type and timing (89%) and qualified supervisor (84%). Conversely, some items showed more uncertainty: the safety of exercise programmes was rated as 'probably low risk' in 58% of cases, exercise dosage in 34% and adherence to exercise in 44%. The items related to exercise dosage (31%) and adherence (29%) had heterogenous judgements, scoring as high risk of ineffectiveness or probably not done. Among all exercise types, Pilates scored best in all domains. A satisfactory specific agreement for 'low risk category' was achieved in all items, except dosage of exercise (60%) and adherence to exercise (54%).

Conclusion: Exercises delivered for patients with cLBP generally demonstrate favourable therapeutic quality, although some exercise modalities may present poor therapeutic quality related to dosage and adherence. While the i-CONTENT judgements generally showed satisfactory specific agreement between raters, disagreements arose in evaluating some crucial items.

Objective: To evaluate the impact of adding librarians and information specialists (LIS) as methodological peer reviewers to the formal journal peer review process on the quality of search reporting and risk of bias in systematic review searches in the medical literature.

Design: Pragmatic two-group parallel randomised controlled trial.

Setting: Three biomedical journals.

Participants: Systematic reviews and related evidence synthesis manuscripts submitted to The BMJ, BMJ Open and BMJ Medicine and sent out for peer review from 3 January 2023 to 1 September 2023. Randomisation (allocation ratio, 1:1) was stratified by journal and used permuted blocks (block size=4). Of 2670 manuscripts sent to peer review during study enrollment, 400 met inclusion criteria and were randomised (62 The BMJ, 334 BMJ Open, 4 BMJ Medicine). 76 manuscripts were revised and resubmitted in the intervention group and 90 in the control group by 2 January 2024.

Interventions: All manuscripts followed usual journal practice for peer review, but those in the intervention group had an additional (LIS) peer reviewer invited.

Main outcome measures: The primary outcomes are the differences in first revision manuscripts between intervention and control groups in the quality of reporting and risk of bias. Quality of reporting was measured using four prespecified PRISMA-S items. Risk of bias was measured using ROBIS Domain 2. Assessments were done in duplicate and assessors were blinded to group allocation. Secondary outcomes included differences between groups for each individual PRISMA-S and ROBIS Domain 2 item. The difference in the proportion of manuscripts rejected as the first decision post-peer review between the intervention and control groups was an additional outcome.

Results: Differences in the proportion of adequately reported searches (4.4% difference, 95% CI: -2.0% to 10.7%) and risk of bias in searches (0.5% difference, 95% CI: -13.7% to 14.6%) showed no statistically significant differences between groups. By 4 months post-study, 98 intervention and 70 control group manuscripts had been rejected after peer review (13.8% difference, 95% CI: 3.9% to 23.8%).

Conclusions: Inviting LIS peer reviewers did not impact adequate reporting or risk of bias of searches in first revision manuscripts of biomedical systematic reviews and related review types, though LIS peer reviewers may have contributed to a higher rate of rejection after peer review.

Trial registration number: Open Science Framework: https://doi.org/10.17605/OSF.IO/W4CK2.

When one initially targets the null effect and the point estimate falls close to the null, two challenges exist in rating certainty of evidence. First, when the point estimate is near the null and the data, therefore, suggests little or no effect, rating certainty in a benefit or harm is misleading. Second, since in general the narrower the confidence interval (CI) the more precise the estimate, if the CI is narrow, rating down for imprecision due simply to crossing the null is inappropriate. This paper addresses these issues and provides a solution: to revise the target of certainty rating from a non-zero effect to a little or no effect. This solution requires estimating a range in which the minimal important difference (MID) for benefit and an MID for harm might lie, and thus establishing a range that represents little or no effect. If GRADE (Grading of Recommendations, Assessment, Development, and Evaluations) users are confident that the point estimate represents an effect less than the smallest plausible MID, they will revise their target and rate certainty to a little or no effect. If the entire CI falls within the range of little or no effect, they will not rate down for imprecision. Otherwise (if the CI includes an important effect), they will rate down. Using the solution provided in this paper GRADE users can make an optimal choice of the target of certainty rating.

Objectives: We aimed to determine to what extent systematic reviewers and post-preprint and post-publication peer review identified methodological and reporting issues in COVID-19 trials that could be easily resolved by the authors.

Design: Qualitative study.

Data sources: COVID-NMA living systematic review (covid-nma.com), PubPeer, medRxiv, Research Square, SSRN.

Methods: We considered randomised controlled trials (RCTs) in COVID-NMA that evaluated pharmacological treatments for COVID-19 and retrieved systematic reviewers' assessments of the risk of bias and outcome reporting bias. We also searched for commentary data on PubPeer and preprint servers up to 6 November 2023. We employed qualitative content analysis to develop themes and domains of methodological and reporting issues identified by commenters.

Results: We identified 500 eligible RCTs. Systematic reviewers identified methodological and reporting issues in 446 (89%) RCTs. In 391 (78%) RCTs, the issues could be easily resolved by the trial authors; issues included incomplete reporting (49%), selection of the reported results (52%) and no access to the pre-specified plan (25%). Alternatively, 74 (15%) RCTs had received at least one comment on PubPeer or preprint servers, totalling 348 comments. In 46 (9%) RCTs, the issues identified by post-preprint and post-publication peer review comments could be easily resolved by the trial authors; the issues were related to incomplete reporting (6%), errors (5%), statistical analysis (3%), inconsistent reporting of methods and analyses (2%), spin (2%), selection of the reported results (1%) and no access to the raw data/pre-specified plan (1%).

Conclusions: Without changing their process, systematic reviewers identified issues in most RCTs that could be easily resolved by the trial authors; however, the lack of an established author feedback mechanism represents a wasted opportunity for facilitating improvement and enhancing the overall manuscript quality. On the other hand, despite the existing feedback loop to authors present in post-publication peer review, it demonstrated limited effectiveness in identifying methodological and reporting issues.