BMC Medical Research Methodology最新文献

Background: Methods for matching in longitudinal cohort studies, such as sequential stratification and time-varying propensity scores, facilitate causal inferences in the context of time-dependent treatments that are not randomized where patient eligibility or treatment status changes over time. The tradeoffs in available approaches have not been compared previously, so we compare two methods using simulations based on a retrospective cohort of patients eligible for weight loss surgery, some of whom received it.

Methods: This study compares matching completeness, bias, coverage, and precision among three approaches to longitudinal matching: (1) time-varying propensity scores (tvPS), (2) sequential stratification that matches exactly on all covariates used in tvPS (SS-Full) and (3) sequential stratification that exact matches on a subset of covariates (SS-Selected). These comparisons are made in the context of a deep sampling frame (50:1) and a shallow sampling frame (5:1) of eligible comparators. A simulation study was employed to estimate the relative performance of these approaches.

Results: In 1,000 simulations each, tvPS retained more than 99.9% of treated patients in both the deep and shallow sampling frames, while a smaller proportion of treated patients were retained for SS-Full (91.6%) and SS-Selected (98.2%) in the deep sampling frame. In the shallow sampling frame, sequential stratification retained many fewer treated patients (73.9% SS-Full, 92.0% SS-Selected) than tvPS yet coverage, precision and bias were comparable for tvPS, SS-Full and SS-Selected in the deep and shallow sampling frames.

Conclusion: Time-varying propensity scores have comparable performance to sequential stratification in terms of coverage, bias, and precision, with superior match completeness. While performance was generally comparable across methods, greater match completeness makes tvPS an attractive option for longitudinal matching studies where external validity is highly valued.

Background: The generalized estimating equation (GEE) method is widely used for analyzing longitudinal and clustered data. Although the GEE estimate for regression coefficients and sandwich covariance estimate are consistent regardless of the choice of covariance structure, they are generally biased for small sample sizes. Various researchers have proposed modified GEE methods and covariance estimators to handle small-sample bias.

Results: We briefly present bias-corrected and penalized GEE methods, along with 11 bias-adjusted covariance estimators. In addition, we focus on analyzing longitudinal or clustered data with binary outcomes using the logit link function and introduce package geessbin in R to implement conventional and modified GEE methods with bias-adjusted covariance estimators. Finally, we illustrate the implementation and detail a usage example of the package. The package is available from the Comprehensive R Archive Network (CRAN) at https://cran.r-project.org/web/packages/geessbin/index.html .

Conclusions: The geessbin package provides three GEE estimates with numerous covariance estimates. It is useful for analyzing correlated data such as longitudinal and clustered data. Additionally, the geessbin is designed to be user-friendly, making it accessible to non-statisticians.

Background: Pooling data from complex survey designs is increasingly used in the health and medical sciences. However, current methodological practices are not well documented in the literature while performing the pooling strategy. We aimed to review related pooling studies and evaluate the quality of pooling within the framework of specific methodological guidelines, particularly when combining complex surveys such as Demographic & Health Surveys (DHS) and Multiple Indicator Cluster Surveys (MICS).

Methods: We performed a systematic literature search focusing on studies utilizing the pooling method with DHS and MICS survey data. These studies were selected from those published between 2010 and 2021 and were retrieved from electronic databases (PubMed and Scopus) in accordance with pre-defined inclusion criteria. Then, we extracted 355 studies for the final review and evaluated the reporting quality of the pooling strategy while considering some methodological issues.

Results: The majority of studies (81.4%) reported using a pooled (one-stage) approach, while 11.8% used a separate (two-stage) approach, and 6.8% used both approaches. Approximately 63.3% of studies did not clearly describe their pooling strategy. Only 3.4% of the studies mentioned the variable harmonization process, while 66.9% addressed dealing with heterogeneity between surveys. All studies that used the separate (two-stage) approach conducted a meta-analytic procedure, while 38.1% of studies using the pooled approach employed a multilevel model. More than half of the studies (55.6%) mentioned the use of clustered standard errors. The Delta method, Bootstrap, and Taylor linearization were each applied in 11.1% of the studies for variance estimation. Survey weights, primary sampling unit (PSU) or cluster, and strata were used together in 30.5% of the studies. Survey weights were employed by 69.8%, PSU or cluster by 43.8%, and the strata variable by 31.7%. Sensitivity analysis was conducted in 16% of the studies.

Conclusions: Our study revealed that fundamental methodological issues associated with pooling complex survey databases, such as the selection of pooling procedures, data harmonization, accounting for cycle effects, quality control checks, addressing heterogeneity, selecting model effects, utilizing survey design variables, and dealing with missing values, etc., were inadequately reported in the included studies. We recommend authors, readers, reviewers, and editors examine pooling studies more attentively and utilize the customized checklist developed by our study to assess the quality of future pooling studies.

Background: Measurements, nowcasts, or forecasts ideally should correctly reflect changes in the values of interest. In this article, we focus on how to assess the ability of measurements, nowcasts, or forecasts to correctly predict the direction of changes in values - which we refer to as the ability to track changes (ATC).

Methods: We review and develop visual techniques and quantitative measures to assess ATC. Extensions for noisy data and estimation uncertainty are implemented using bootstrap confidence intervals and exclusion areas.

Results: We exemplarily illustrate the proposed methods to assess the ability to track changes for nowcasting during the COVID-19 pandemic, patient admissions to an emergency department, and non-invasive blood pressure measurements. The proposed methods effectively evaluate ATC across different applications.

Conclusions: The developed ATC assessment methods offer a comprehensive toolkit for evaluating the ATC of measurements, nowcasts, and forecasts. These techniques provide valuable insights into model performance, complementing traditional accuracy measures and enabling more informed decision-making in various fields, including public health, healthcare management, and medical diagnostics.

Background: Key informant interviews (KII) are a widely used method in qualitative health research to gain in-depth insights from individuals with specialized knowledge, experience, or access that is crucial to the research topic. However, there is growing criticism regarding how the selection of key informants is insufficiently described in research. This opacity is problematic as the authority and knowledge of key informants may be given undue weight in research findings, potentially overshadowing other non-expert samples. The resulting imbalance in representation can lead to favoring certain viewpoints while marginalizing others, and thereby reinforcing existing inequities.

Methods: Using our KII study as an example, we demonstrate how we initially composed an ideal sample based on theoretical considerations and subsequently operationalized it in the field. We employed a selective recruitment strategy informed by intersectional theory, targeting physicians with migration backgrounds from Middle Eastern countries for a study on cancer prevention and screening. Our recruitment process combined direct methods, including database searches and email outreach, with indirect methods like snowball sampling and engagement with multipliers. The recruitment strategy was iterative, allowing for ongoing assessment and adaptation to ensure a diverse and representative sample.

Results: The KII study successfully recruited 21 physicians with diverse social categories, including different genders, migration backgrounds, language skills, and medical specialties. Direct recruitment was more effective than indirect methods and allowed for greater control in reaching out to specific subsamples. It highlights the importance of flexible and persistent recruitment strategies to achieve the desired sample.

Conclusions: This KII study underscores the interplay between methodological ideals and the practical realities of recruiting a diverse, carefully composed sample of key informants in health research. Our intersectional approach aimed to ensure equitable representation by considering power dynamics and refining recruitment strategies, while balancing the challenges of real-world fieldwork-such as engaging busy physicians with specific recruitment criteria-with practical adaptability. Our KII study emphasizes the need for ongoing reflexivity to balance ideality and equity with practical feasibility.

Background: The SF-8™ Short Form Health Survey creates physical and mental health scale scores from responses to eight survey questions. These widely used scales demonstrate reasonable reliablity, and some forms of validity but have not been assessed for fusion validity. We assess the fusion validity of the SF-8 physical and mental health scales, and provide comments assisting fusion validity assessment of other scales.

Methods: Checking the fusion validity of a scale requires including the scale and its constituent indicators in a structural equation model that has at least one variable causally downstream from the scale. We assessed fusion validity of the SF-8 physical and mental health scales in the context of work-related variables for care aides working in Canadian long-term care homes. Variables causally downstream from physical and mental health, such as work burnout, permit checking whether the SF-8 indicator items fuse to form cogent physical and mental scales, irrespective of whether those indicators share common-factor foundations.

Results: We found that the SF-8 physical and mental health scales did not function appropriately. The scales inappropriately claimed effects for several items that had no effects and provided biased estimates of other effects. These deficiencies seem grounded in the scales' developmental history, which implicitly bolstered selection of some causally ambiguous items and paid insufficient attention to component factor model testing.

Conclusion: Our observations of causal incongruities question whether the SF-8 can provide valid assessments of physical and mental health. However, it would be imprudent to discontinue SF-8 use on the basis of a single study suggesting invalidity. This uncomfortable conclusion can be rechecked by re-analyzing data from any project that employed the SF-8 and recorded even one causal consequence of physical or mental health. The power of fusion validity assessment comes from connecting the recorded consequences simultaneously to both the scale and the items from which that scale is calculated.

Background: Pragmatic randomized controlled trials are getting more interest to improve trials' external validity. This study aimed to assess how pragmatic the design of the self-labelled pragmatic randomised controlled trials in the manual therapy field is.

Methods: We searched MEDLINE and the Cochrane Central Register of Controlled Trials for self-labelled pragmatic randomised controlled trials in the manual therapy field until January 2024 were included. Two independent reviewers collected and extracted data related to the intention of the trial, the rationale for the intervention, and specific features of the trial and performed an assessment using the PRECIS-2 tool.

Results: Of 39 self-labelled pragmatic trials, the mean PRECIS-2 score was 3.5 (SD: 0.6). Choice of outcome measures, how the interventions were performed, the follow-up of the participants and how all the available data were included in the statistical analysis were the domains rated as most 'pragmatic'. Participants' eligibility, recruitment, and setting obtained lower scores. Less than 25% of the trials claimed that the aim was to investigate an intervention under real-world conditions and to make clinical decisions about its effectiveness. In the 21% of the sample the authors described neither the proof-of-concept of the intervention nor the state of previous studies addressing related research questions.

Conclusions: Self-labelled pragmatic randomised controlled trials showed a moderately pragmatic attitude. Beyond the label 'pragmatic', the description of the intention of the trial and the context of every PRECIS-2 domain is crucial to understanding the real pragmatism of a trial.

Background: A time-dependent analysis, usually by means of Poisson and Cox regression models, can be applied to prevent immortal time bias. However, the use of the Poisson model requires the assumption that the event rate is constant over time. This study aims to assess the potential consequences of using the Poisson model to cope with immortal time bias on estimating the exposure-outcome relationship in the case of time-varying risks.

Methods: A simulation study was carried out. Survival times were assumed to follow a Weibull distribution, and the Weibull parameters were chosen to identify three different scenarios: the hazard of the event is constant, decreases, or increases over time. A dichotomous time-varying exposure in which patients can change at most once from unexposed to exposed was considered. The Poisson model was fitted to estimate the exposure-outcome association.

Results: Small changes in the outcome risk over time (as denoted by the shape parameter of the Weibull distribution) strongly affected the exposure-outcome association estimate. The estimated effect of exposure was always lower and greater than the true exposure effect when the event risk decreases or increases over time, and this was the case irrespective of the true exposure effect. The bias magnitude was positively associated with the prevalence of and time to exposure.

Conclusions: Biased estimates were obtained from the Poisson model to cope with immortal time. In settings with a time-varying outcome risk, the model should adjust for the trend in outcome risk. Otherwise, other models should be considered.

Background: Systematic reviews (SRs) are used to inform clinical practice guidelines and healthcare decision making by synthesising the results of primary studies. Efficiently retrieving as many relevant SRs as possible is challenging with a minimum number of databases, as there is currently no guidance on how to do this optimally. In a previous study, we determined which individual databases contain the most SRs, and which combination of databases retrieved the most SRs. In this study, we aimed to validate those previous results by using a different, larger, and more recent set of SRs.

Methods: We obtained a set of 100 Overviews of Reviews that included a total of 2276 SRs. SR inclusion was assessed in MEDLINE, Embase, and Epistemonikos. The mean inclusion rates (% of included SRs) and corresponding 95% confidence intervals were calculated for each database individually, as well as for combinations of MEDLINE with each other database and reference checking. Features of SRs not identified by the best database combination were reviewed qualitatively.

Results: Inclusion rates of SRs were similar in all three databases (mean inclusion rates in % with 95% confidence intervals: 94.3 [93.9-94.8] for MEDLINE, 94.4 [94.0-94.9] for Embase, and 94.4 [93.9-94.9] for Epistemonikos). Adding reference checking to MEDLINE increased the inclusion rate to 95.5 [95.1-96.0]. The best combination of two databases plus reference checking consisted of MEDLINE and Epistemonikos (98.1 [97.7-98.5]). Among the 44/2276 SRs not identified by this combination, 34 were published in journals from China, four were other journal publications, three were health agency reports, two were dissertations, and one was a preprint. When discounting the journal publications from China, the SR inclusion rate in the recommended combination (MEDLINE, Epistemonikos and reference checking) was even higher than in the previous study (99.6 vs. 99.2%).

Conclusions: A combination of databases and reference checking was the best approach to searching for biomedical SRs. MEDLINE and Epistemonikos, complemented by checking the references of the included studies, was the most efficient and produced the highest recall. However, our results point to the presence of geographical bias, because some publications in journals from China were not identified.

Study registration: https://doi.org/10.17605/OSF.IO/R5EAS (Open Science Framework).

Background: Missing values in datasets present significant challenges for data analysis, particularly in the medical field where data accuracy is crucial for patient diagnosis and treatment. Although MissForest (MF) has demonstrated efficacy in imputation research and recursive feature elimination (RFE) has proven effective in feature selection, the potential for enhancing MF through RFE integration remains unexplored.

Methods: This study introduces a novel imputation method, "recursive feature elimination-MissForest" (RFE-MF), designed to enhance imputation quality by reducing the impact of irrelevant features. A comparative analysis is conducted between RFE-MF and four classical imputation methods: mean/mode, k-nearest neighbors (kNN), multiple imputation by chained equations (MICE), and MF. The comparison is carried out across ten medical datasets containing both numerical and mixed data types. Different missing data rates, ranging from 10 to 50%, are evaluated under the missing completely at random (MCAR) mechanism. The performance of each method is assessed using two evaluation metrics: normalized root mean squared error (NRMSE) and predictive fidelity criterion (PFC). Additionally, paired samples t-tests are employed to analyze the statistical significance of differences among the outcomes.

Results: The findings indicate that RFE-MF demonstrates superior performance across the majority of datasets when compared to four classical imputation methods (mean/mode, kNN, MICE, and MF). Notably, RFE-MF consistently outperforms the original MF, irrespective of variable type (numerical or categorical). Mean/mode imputation exhibits consistent performance across various scenarios. Conversely, the efficacy of kNN imputation fluctuates in relation to varying missing data rates.

Conclusion: This study demonstrates that RFE-MF holds promise as an effective imputation method for medical datasets, providing a novel approach to addressing missing data challenges in medical applications.