BMJ Evidence-Based Medicine最新文献

Objectives: To evaluate the feasibility of conducting a clinically integrated randomised comparative effectiveness trial using digital clinical trial infrastructure within an electronic patient record (EPR).

Design: A mixed-methods, unblinded, feasibility study of digital clinical trial system incorporating testing of two designs of electronic point-of-care randomisation prompt.

Setting: The study was conducted at University College London Hospitals NHS Trust between March and November 2022. The study used a real clinical research question for context, comparing liberal vs restrictive strategies for magnesium supplementation to prevent new-onset atrial fibrillation in critical care.

Participants: Adult patients undergoing elective, non-cardiac surgical procedures expecting postoperative admission to critical care were recruited.

Interventions: A digital trial system screened participants continuously against eligibility criteria. Participants were automatically randomised (1:1) to (1) magnesium supplementation strategy and (2) one of two electronic randomisation prompt designs (nudge or preference).Electronic point-of-care randomisation prompts displayed to clinicians at regular intervals, inviting them to follow a randomised magnesium supplementation suggestion.

Main outcome measures: The primary outcome measure was a composite determination of study design feasibility (including recruitment, technical performance and concordance between the randomised suggestion and the observed clinician action).

Results: 23 patients were recruited and 11 successfully randomised. The implemented digital systems for automated eligibility screening, randomisation, data collection and follow-up demonstrated technical feasibility. 47 electronic point-of-care randomisation prompts successfully deployed across 11 patients. Clinician actions were concordant with randomised suggestions in 32 prompts (68%).Technical and implementational barriers to delivering the electronic point-of-care randomisation prompts were identified. Patients were followed up to 30 days following discharge from hospital, with no serious adverse events attributable to participation identified.There was insufficient data to make a quantitative determination on the superiority of either prompt design. Clinician feedback suggested the simplified design (nudge) had greater utility.

Conclusions: This study demonstrates that digitally embedding clinical trial infrastructure into a site-level EPR and integrating conduct into clinical care is safe and feasible. Future work will focus on improving and expanding the integrated digital trial design across multiple centres.

Trial registration number: NCT05149820.

The phenomenon of cancer overdiagnosis, the diagnosis of a malignant tumour that, without detection, would never lead to adverse health effects, has been reported for several cancer types in different populations. There has been an increase in studies focused on overdiagnosis, creating an opportunity to synthesise evidence on specific cancer types. However, studies that systematically assess evidence across different research domains remain scarce, with most of them relying on data from studies that already mentioned overdiagnosis as a potential concern. In this review, we consider several evidence categories that are used to systematically assess the presence and magnitude of overdiagnosis, including (1) data from cancer surveillance, (2) studies exploring the 'true' prevalence of cancer in the population, (3) studies that explore the use of diagnostics and its effect on incidence and mortality and (4) studies that explore changes and progress in cancer management and its effect on cancer mortality. This article highlights the strengths and weaknesses of different evidence categories, provides examples of studies on different cancer types and discusses how these categories can help synthesise evidence on cancer overdiagnosis.

Objective: The purpose of this study is to validate the taxonomy and framework using Chinese guidelines and identify actionable statements.

Design and setting: We searched five databases, to identify the health guidelines from 1 January 2020 to 1 May 2023. Five researchers categorised statements into six types: formal recommendations (Type I) with clear direction and strength, with explicit and direct evidence; good practice statements (GPS) (Type II), actionable in isolation with a significant benefit; remarks (Type III), an inseparable unit belonging to a formal recommendation or GPS that provides additional clarification; research only recommendations (Type IV) for specific populations; implementation considerations, tools and tips (Type V), that describe the how, who, where, what and when, in relation to implementing a recommendation and lacking a direct evidence link; and informal recommendations (Type VI), unrelated to evidence and not meeting GPS criteria.

Results: We included 116 guidelines, including 74 Western medicine guidelines, 12 traditional Chinese medicine guidelines and 30 integrated Chinese and Western medicine guidelines. 99 guidelines (85.3%) used the Grading of Recommendations Assessment, Development and Evaluation criteria. Medical specialty societies developed the highest number of guidelines (53.4%). Of all the statements, 4422 statements were extracted from the guidelines. Among them, 2154 (48.7%) were formal recommendations, 197 (4.4%) were GPS, 394 (8.9%) were remarks, 16 (0.4%) were research only recommendations, 1106 (25.0%) were implementation considerations, tools and tips, and 555 (12.6%) were informal recommendations.

Conclusions: Up to date, the Chinese guideline developers tend to overestimate the number of formal recommendations and underestimate the number of GPS, remarks, research only recommendations, implementation considerations, tools and tips, and informal recommendations. Thus the current quality of actionable statements in Chinese health guidelines requires further enhancement.

Objective: To assess custom GPT-4 performance in extracting and evaluating data from medical literature to assist in the systematic review (SR) process.

Design: A proof-of-concept comparative study was conducted to assess the accuracy and precision of custom GPT-4 models against human-performed reviews of randomised controlled trials (RCTs).

Setting: Four custom GPT-4 models were developed, each specialising in one of the following areas: (1) extraction of study characteristics, (2) extraction of outcomes, (3) extraction of bias assessment domains and (4) evaluation of risk of bias using results from the third GPT-4 model. Model outputs were compared against data from four SRs conducted by human authors. The evaluation focused on accuracy in data extraction, precision in replicating outcomes and agreement levels in risk of bias assessments.

Participants: Among four SRs chosen, 43 studies were retrieved for data extraction evaluation. Additionally, 17 RCTs were selected for comparison of risk of bias assessments, where both human comparator SRs and an analogous SR provided assessments for comparison.

Intervention: Custom GPT-4 models were deployed to extract data and evaluate risk of bias from selected studies, and their outputs were compared to those generated by human reviewers.

Main outcome measures: Concordance rates between GPT-4 outputs and human-performed SRs in data extraction, effect size comparability and inter/intra-rater agreement in risk of bias assessments.

Results: When comparing the automatically extracted data to the first table of study characteristics from the published review, GPT-4 showed 88.6% concordance with the original review, with <5% discrepancies due to inaccuracies or omissions. It exceeded human accuracy in 2.5% of instances. Study outcomes were extracted and pooling of results showed comparable effect sizes to comparator SRs. A review of bias assessment using GPT-4 showed fair-moderate but significant intra-rater agreement (ICC=0.518, p<0.001) and inter-rater agreements between human comparator SR (weighted kappa=0.237) and the analogous SR (weighted kappa=0.296). In contrast, there was a poor agreement between the two human-performed SRs (weighted kappa=0.094).

Conclusion: Customized GPT-4 models perform well in extracting precise data from medical literature with potential for utilization in review of bias. While the evaluated tasks are simpler than the broader range of SR methodologies, they provide an important initial assessment of GPT-4's capabilities.