BMC Medical Research Methodology最新文献

Background: Evaluation approaches such as ripple effects mapping (REM) and realist evaluation have emerged as popular methodologies to evidence impact, and the processes of change within public health as part of whole systems approaches. Despite the various examples of their implementation across different evaluation settings, there has been little or no evidence of how they might be effective when combined.

Methods: With REM's potential to pragmatically illustrate impact, and realist evaluation's strength to identify how and why impacts emerge, this paper develops a rationale and process for their amalgamation. Following this, we outline a realist-informed ripple effects mapping (RREM) protocol drawing upon a physical activity based case study in Essex that may be suitable for application within evaluation settings in a range of public health, whole system and physical activity settings.

Discussion: Combining these two approaches has the potential to more effectively illuminate the impacts that we see within public health and whole system approaches and initiatives. What is more, given the complexity often imbued within these approaches and initiatives, they hold capability for also capturing the causal mechanisms that explain these impacts.

Conclusions: It is our conclusion that when combined, this novel approach may help to inspire future research as well as more effective evaluation of public health and whole system approaches. This is crucial if we are to foster a culture for learning, refinement and reflection.

Background: Dichotomisation of statistical significance, rather than interpretation of effect sizes supported by confidence intervals, is a long-standing problem.

Methods: We distributed an online survey to clinical trial statisticians across the UK, Australia and Canada asking about their experiences, perspectives and practices with respect to interpretation of statistical findings from randomised trials. We report a descriptive analysis of the closed-ended questions and a thematic analysis of the open-ended questions.

Results: We obtained 101 responses across a broad range of career stages (24% professors; 51% senior lecturers; 22% junior statisticians) and areas of work (28% early phase trials; 44% drug trials; 38% health service trials). The majority (93%) believed that statistical findings should be interpreted by considering (minimal) clinical importance of treatment effects, but many (61%) said quantifying clinically important effect sizes was difficult, and fewer (54%) followed this approach in practice. Thematic analysis identified several barriers to forming a consensus on the statistical interpretation of the study findings, including: the dynamics within teams, lack of knowledge or difficulties in communicating that knowledge, as well as external pressures. External pressures included the pressure to publish definitive findings and statistical review which can sometimes be unhelpful but can at times be a saving grace. However, the concept of the minimally important difference was identified as a particularly poorly defined, even nebulous, construct which lies at the heart of much disagreement and confusion in the field.

Conclusion: The majority of participating statisticians believed that it is important to interpret statistical findings based on the clinically important effect size, but report this is difficult to operationalise. Reaching a consensus on the interpretation of a study is a social process involving disparate members of the research team along with editors and reviewers, as well as patients who likely have a role in the elicitation of minimally important differences.

Background: The fixed effects model is a useful alternative to the mixed effects model for analyzing stepped-wedge cluster randomized trials (SW-CRTs). It controls for all time-invariant cluster-level confounders and has proper control of type I error when the number of clusters is small. While all clusters in a SW-CRT are typically designed to crossover from the control to receive the intervention, some trials can end with unexposed clusters (clusters that never receive the intervention), such as when a trial is terminated early due to safety concerns. It was previously unclear whether unexposed clusters would contribute to the estimation of the intervention effect in a fixed effects analysis. However, recent work has demonstrated that including an unexposed cluster can improve the precision of the intervention effect estimator in a fixed effects analysis of SW-CRTs with continuous outcomes. Still, SW-CRTs are commonly designed with binary outcomes and it is unknown if those previous results extend to SW-CRTs with non-continuous outcomes.

Methods: In this article, we mathematically prove that the inclusion of unexposed clusters improves the precision of the fixed effects intervention effect estimator for SW-CRTs with binary and count outcomes. We then explore the benefits of including an unexposed cluster in simulated datasets with binary or count outcomes and a real palliative care data example with binary outcomes.

Results: The simulations show that including unexposed clusters leads to tangible improvements in the precision, power, and root mean square error of the intervention effect estimator. The inclusion of the unexposed cluster in the SW-CRT of a novel palliative care intervention with binary outcomes yielded smaller standard errors and narrower 95% Wald Confidence Intervals.

Conclusions: In this article, we demonstrate that the inclusion of unexposed clusters in the fixed effects analysis can lead to the improvements in precision, power, and RMSE of the fixed effects intervention effect estimator for SW-CRTs with binary or count outcomes.

Background: In randomised controlled trials with efficacy-related primary outcomes, adverse events are collected to monitor potential intervention harms. The analysis of adverse event data is challenging, due to the complex nature of the data and the large number of unprespecified outcomes. This is compounded by a lack of guidance on best analysis approaches, resulting in widespread inadequate practices and the use of overly simplistic methods; leading to sub-optimal exploitation of these rich datasets. To address the complexities of adverse events analysis, statistical methods are proposed that leverage existing structures within the data, for instance by considering groupings of adverse events based on biological or clinical relationships.

Methods: We conducted a methodological scoping review of the literature to identify all existing methods using structures within the data to detect signals for adverse reactions in a trial. Embase, MEDLINE, Scopus and Web of Science databases were systematically searched. We reviewed the analysis approaches of each method, extracted methodological characteristics and constructed a narrative summary of the findings.

Results: We identified 18 different methods from 14 sources. These were categorised as either Bayesian approaches (n=11), which flagged events based on posterior estimates of treatment effects, or error controlling procedures (n=7), which flagged events based on adjusted p-values while controlling for some type of error rate. We identified 5 defining methodological characteristics: the type of outcomes considered (e.g. binary outcomes), the nature of the data (e.g. summary data), the timing of the analysis (e.g. final analysis), the restrictions on the events considered (e.g. rare events) and the grouping systems used.

Conclusions: We found a large number of analysis methods that use the group structures of adverse events. Continuous methodological developments in this area highlight the growing awareness that better practices are needed. The use of more adequate analysis methods could help trialists obtain a better picture of the safety-risk profile of an intervention. The results of this review can be used by statisticians to better understand the current methodological landscape and identify suitable methods for data analysis - although further research is needed to determine which methods are best suited and create adequate recommendations.

Background: Accurate reporting of outcomes is crucial for interpreting the results of randomized controlled trials (RCTs). However, selectively reporting outcomes in publications to achieve researchers' anticipated results still occurs frequently. This study aims to investigate the prevalence of selective reporting of outcomes in RCTs on treating acute ischemic stroke (AIS), identify factors contributing to this issue, and assess its potential impact on the degree and direction of intervention effect.

Methods: A search was conducted in MEDLINE, Embase, and the Cochrane Library to collect interventional RCTs on AIS published from 2020 to 2022. Full texts of RCTs were reviewed, and only those reporting International Clinical Trials Registry Platform primary registry numbers were included. Registration information of the RCTs was extracted from the registry platforms and compared with the publications' details to assess the selective reporting of outcomes. Bayesian multilevel logistic regression was used to analyze the reasons behind selective reporting.

Results: Among the total of 159 AIS RCTs identified, 82 (51.6%) were ultimately included, as they reported registration numbers, which encompassed 819 outcomes. Among them, 72 RCTs (87.8%) and 497 outcomes (60.7%) exhibited selective reporting. Omission-type selective reporting (downgrading, omitting, or ambiguously reporting) accounted for 36.4%, while addition-type selective reporting (upgrading, adding, or altering the measurement scope of outcomes) comprised 63.6%. Omission-type selective reporting correlated with negative results (OR: 7.39; 95% CI: 4.08-13.44), whereas addition-type selective reporting correlated with positive results (OR: 2.07; 95% CI: 1.34-3.26) and publication in journals that are not in the top quartile of the Journal Citation Reports (OR: 2.48; 95% CI: 1.15-5.38).

Conclusions: Registered interventional AIS RCTs still face significant issues regarding selective reporting of outcomes. Therefore, it is necessary to further evaluate the influence of selective reporting bias on the positive results obtained from individual AIS RCTs and the systematic reviews based on these RCTs.

Background: Reference intervals, which define an interval in which a specific proportion of measurements from a healthy population are expected to fall, are commonly used in medical practice. Synthesizing information from multiple studies through meta-analysis can provide a more precise and representative reference interval than one derived from a single study. However, the current approaches for estimating the reference interval from a meta-analysis mainly rely on aggregate data and require parametric distributional assumptions that cannot always be checked.

Methods: With the availability of individual participant data (IPD), non-parametric methods can be used to estimate reference intervals without any distributional assumptions. Furthermore, patient-level covariates can be introduced to estimate personalized reference intervals that may be more applicable to specific patients. This paper introduces quantile regression as a method to estimate the reference interval from an IPD meta-analysis under the fixed effects model.

Results: We compared several non-parametric bootstrap methods through simulation studies to account for within-study correlation. Under fixed effects model, we recommend keeping the studies fixed and only randomly sampling subjects with replacement within each study.

Conclusion: We proposed to use the quantile regression in the IPD meta-analysis to estimate the reference interval. Based on the simulation results, we identify an optimal bootstrap strategy for estimating the uncertainty of the estimated reference interval. An example of liver stiffness measurements, a clinically important diagnostic test without explicitly established reference range in children, is provided to demonstrate the use of quantile regression in estimating both overall and subject-specific reference intervals.

Background: Globorisk is one of the prediction tools for 10-year risk assessment of cardiovascular disease, featuring an office-based (non-laboratory-based) version. This version does not require laboratory tests for determining the CVD risk. The present study aims to determine the 10-year CVD risk using the office-based Globorisk model and factors associated with the 10-year CVD risk.

Methods: In this study, baseline data from 6810 individuals participating in the Fasa cohort study, with no history of CVD or stroke, were utilized. The risk equation of the office-based Globorisk model incorporates age, sex, systolic blood pressure (SBP), body mass index (BMI), and smoking status. The Globorisk model categorizes the risk into three groups: low risk (< 10%), moderate risk (10% to < 20%), and high risk (≥ 20%). To identify factors associated with the 10-year CVD risk, the predicted risk was categorized into two groups: <10% and ≥ 10%. Multivariable logistic regression analysis was employed to determine factors associated with an increased CVD risk.

Results: According to the 10-year CVD risk categorization, 78.3%, 16.4%, and 5.3% of men were in the low, moderate, and high risk groups, respectively, while 85.8%, 10.0%, and 4.2%, of women were in the respective risk groups. Multivariable logistic regression results indicated that in men, the 10-year CVD risk decreases with being an opium user, and increases with being illiterate, having abdominal obesity, and low or moderate physical activity compared to high physical activity. In women, being married, and higher fiber consumption decrease the 10-year CVD risk, while being illiterate, low or moderate physical activity compared to high physical activity, having abdominal obesity, opium use, and being in wealth quintiles 1 to 4 compared to quintile 5 increase the risk.

Conclusions: Considering the factors associated with increased CVD risk, there is a need to enhance awareness and modify lifestyle to mitigate and reduce the risk of CVD. Additionally, early identification of individuals at moderate to high risk is essential for preventing disease progression. The use of the office-based Globorisk model can be beneficial in settings where resources are limited for determining the 10-year CVD risk.

Purpose: We aim to thoroughly compare past and current methods that leverage baseline covariate information to estimate the average treatment effect (ATE) using data from of randomized clinical trials (RCTs). We especially focus on their performance, efficiency gain, and power.

Methods: We compared 6 different methods using extensive Monte-Carlo simulation studies: the unadjusted estimator, i.e., analysis of variance (ANOVA), the analysis of covariance (ANCOVA), the analysis of heterogeneous covariance (ANHECOVA), the inverse probability weighting (IPW), the augmented inverse probability weighting (AIPW), and the overlap weighting (OW) as well as the augmented overlap weighting (AOW) estimators. The performance of these methods is assessed using the relative bias (RB), the root mean square error (RMSE), the model-based standard error (SE) estimation, the coverage probability (CP), and the statistical power.

Results: Even with a well-executed randomization, adjusting for baseline covariates by an appropriate method can be a good practice. When the outcome model(s) used in a covariate-adjusted method is closer to the correctly specified model(s), the efficiency and power gained can be substantial. We also found that most covariate-adjusted methods can suffer from the high-dimensional curse, i.e., when the number of covariates is relatively high compared to the sample size, they can have poor performance (along with lower efficiency) in estimating ATE. Among the different methods we compared, the OW performs the best overall with smaller RMSEs and smaller model-based SEs, which also result in higher power when the true effect is non-zero. Furthermore, the OW is more robust when dealing with the high-dimensional issue.

Conclusion: To effectively use covariate adjustment methods, understanding their nature is important for practical investigators. Our study shows that outcome model misspecification and high-dimension are two main burdens in a covariate adjustment method to gain higher efficiency and power. When these factors are appropriately considered, e.g., performing some variable selections if the data dimension is high before adjusting covariate, these methods are expected to be useful.

Introduction: Evaluation of implementation strategies is core to implementation trials, but implementation strategies often deviate from the original plan to adjust to the real-world conditions. The optimal approach to track modifications to implementation strategies is unclear, especially in low-resource settings. Using data from an implementation trial for people who inject drugs (PWID) with HIV in Vietnam, we describe the tracking of implementation strategy modifications and present findings of this process.

Methods: SNaP (Systems Navigation and Psychosocial Counseling) is a hybrid type-III effectiveness-implementation randomized controlled trial aiming to scale up the evidence-based intervention, integrated systems navigation and psychosocial counseling, for PWID with HIV in Vietnam. Forty-two HIV testing sites were randomized 1:1 to a standard or tailored arm. While the standard arm (SA) received a uniform package of strategies, implementation strategies for the tailored arm (TA) were tailored to address specific needs of each site. The central research team also met monthly with the TA to document how their tailored strategies were implemented over time. Five components were involved in the tracking process: describing the planned strategies; tracking strategy use; monitoring barriers and solutions; describing modifications; and identifying and describing any additional strategies.

Results: Our approach allowed us to closely track the modifications to implementation strategies in the tailored arms every month. TA sites originally identified 27 implementation strategies prior to implementation. During implementation, five strategies were dropped by four sites and two new strategies were added to twelve sites. Modifications of five strategies occurred at four sites to accommodate their changing needs and resources. Difficulties related to the COVID-19 pandemic, low number of participants recruited, high workload at the clinic, lack of resources for HIV testing and high staff turnover were among barriers of implementing the strategies. A few challenges to tracking modifications were noted, including the considerable amount of time and efforts needed as well as the lack of motivation from site staff to track and keep written documentations of modifications.

Conclusions: We demonstrated the feasibility of a systematic approach to tracking implementation strategies for a large-scale implementation trial in a low-resource setting. This process could be further enhanced and replicated in similar settings to balance the rigor and feasibility of implementation strategy tracking. Our findings can serve as additional guidelines for future researchers planning to report and track modifications to implementation strategies in large, complex trials.

Trial registration: clinicaltrials.gov ID: NCT03952520 (first posted 2019-05-16).

Background: Causal mediation analysis is widespread in applied medical research, especially in longitudinal settings. However, estimating natural mediational effects in such contexts is often difficult because of the presence of post-treatment confounding. Moreover, many models frequently used in applied research, like multilevel and latent growth models, present an additional difficulty, i.e. the presence of latent variables. In this paper, we propose a causal interpretation of these two classes of models based on a novel type of causal effects called separable, which overcome some of the issues of natural effects.

Methods: We formally derive conditions for the identifiability of separable mediational effects and their analytical expressions based on the g-formula. We carry out a simulation study to investigate how moderate and severe model misspecification, as well as violation of the identfiability assumptions, affect estimates. We also present an application to real data.

Results: The results show how model misspecification impacts the estimates of mediational effects, particularly in the case of severe misspecification, and that the bias worsens over time. The violation of assumptions affects separable effect estimates in a very different way for the mixed effect and the latent growth models.

Conclusion: Our approach allows us to give multilevel and latent growth models an appealing causal interpretation based on separable effects. The simulation study shows that model misspecification can heavily impact effect estimates, highlighting the importance of careful model choice.