BMC Medical Research Methodology最新文献

Background: Systematic reviews within the field of animal research are becoming more common. However, in animal translational research, issues related to methodological quality and quality of reporting continue to arise, potentially leading to underestimation or overestimation of the effects of interventions or prevent studies from being replicated. The various tools and checklists available to ensure good-quality studies and proper reporting include both unique and/or overlapping items and/or simply lack necessary elements or are too situational to certain conditions or diseases. Currently, there is no tool available, which covers all aspects of animal models, from bench-top activities to animal facilities, hence a new tool is needed. This tool should be designed to be able to assess all kinds of animal studies such as old, new, low quality, high quality, interventional and noninterventional on. It should do this on multiple levels through items on quality of reporting, methodological (technical) quality, and risk of bias, for use in assessing the overall quality of studies involving animal research.

Methods: During a systematic review of meningioma models in animals, we developed a novel unifying tool that can assess all types of animal studies from multiple perspectives. The tool was inspired by the Collaborative Approach to Meta Analysis and Review of Animal Data from Experimental Studies (CAMARADES) checklist, the ARRIVE 2.0 guidelines, and SYRCLE's risk of bias tool, while also incorporating unique items. We used the interrater agreement percentage and Cohen's kappa index to test the interrater agreement between two independent reviewers for the items in the tool.

Results: There was high interrater agreement across all items (92.9%, 95% CI 91.0-94.8). Cohen's kappa index showed quality of reporting had the best mean index of 0.86 (95%-CI 0.78-0.94), methodological quality had a mean index of 0.83 (95%-CI 0.78-0.94) and finally the items from SYRCLE's risk of bias had a mean kappa index of 0.68 (95%-CI 0.57-0.79).

Conclusions: The Critical Appraisal of Methodological (technical) Quality, Quality of Reporting and Risk of Bias in Animal Research (CRIME-Q) tool unifies a broad spectrum of information (both unique items and items inspired by other methods) about the quality of reporting and methodological (technical) quality, and contains items from SYRCLE's risk of bias. The tool is intended for use in assessing overall study quality across multiple domains and items and is not, unlike other tools, restricted to any particular model or study design (whether interventional or noninterventional). It is also easy to apply when designing and conducting animal experiments to ensure proper reporting and design in terms of replicability, transparency, and validity.

Background: Physical activity (PA) is often the cornerstone in risk-reduction interventions for the prevention and treatment of many chronic health conditions. PA interventions are inherently multi-dimensional and complex in nature. Thus, study designs used in the evaluation of PA interventions must be adaptive to intervention components and individual capacities. A Sequential Multiple Assignment Randomised Trial (SMART) is a factorial design in a sequential setting used to build effective adaptive interventions. SMARTs represent a relatively new design for PA intervention research. This systematic review aims to examine the state-of-the-art of SMARTs used to develop PA interventions, with a focus on study characteristics, design, and analyses.

Methods: PubMed, Embase, PsychINFO, CENTRAL, and CinAHL were systematically searched through May 2023 for studies wherein PA SMARTs were conducted. Methodological quality was assessed using the Cochrane Risk of Bias 2 Tool.

Results: Twenty studies across a variety of populations - e.g., obesity, chronic pain, and cardiovascular conditions, were included. All PA SMARTs involved two decision stages, with the majority including two initial treatment options. PA interventions most commonly consisted of individual aerobic exercise with strategies such as goal setting, wearable technology, and motivational interviewing also used to promote PA. Variation was observed across tailoring variables and timing of tailoring variables. Non-response strategies primarily involved augmenting and switching treatment options, and for responders to continue with initial treatment options. For analyses, most sample size estimations and outcome analyses accounted for the SMART aims specified. Techniques such as linear mixed models, weighted regressions, and Q-learning regression were frequently used. Risk of bias was high across the majority of included studies.

Conclusions: Individual-based aerobic exercise interventions supported by behaviour change techniques and wearable sensing technology may play a key role in the future development of SMARTs addressing PA intervention development. Clearer rationale for the selection of tailoring variables, timing of tailoring variables, and included measures is essential to advance PA SMART designs. Collaborative efforts from researchers, clinicians, and patients are needed in order to bridge the gap between adaptive research designs and personalised treatment pathways observed in clinical practice.

Background: The $\gamma$ -metric value is generally used as the importance score of a feature (or a set of features) in a classification context. This study aimed to go further by creating a new methodology for multivariate feature selection for classification, whereby the $\gamma$ -metric is associated with a specific search direction (and therefore a specific stopping criterion). As three search directions are used, we effectively created three distinct methods.

Methods: We assessed the performance of our new methodology through a simulation study, comparing them against more conventional methods. Classification performance indicators, number of selected features, stability and execution time were used to evaluate the performance of the methods. We also evaluated how well the proposed methodology selected relevant features for the detection of atrial fibrillation, which is a cardiac arrhythmia.

Results: We found that in the simulation study as well as the detection of AF task, our methods were able to select informative features and maintain a good level of predictive performance; however in a case of strong correlation and large datasets, the $\gamma$ -metric based methods were less efficient to exclude non-informative features.

Conclusions: Results highlighted a good combination of both the forward search direction and the $\gamma$ -metric as an evaluation function. However, using the backward search direction, the feature selection algorithm could fall into a local optima and can be improved.

Background: Electronic medical records (EMR)-trained machine learning models have the potential in CVD risk prediction by integrating a range of medical data from patients, facilitate timely diagnosis and classification of CVDs. We tested the hypothesis that unsupervised ML approach utilizing EMR could be used to develop a new model for detecting prevalent CVD in clinical settings.

Methods: We included 155,894 patients (aged ≥ 18 years) discharged between January 2014 and July 2022, from Xuhui Hospital, Shanghai, China, including 64,916 CVD cases and 90,979 non-CVD cases. K-means clustering was used to generate the clustering models with k = 2, 4, and 8 as predetermined number of clusters k = 2, 4, and 8. Bayesian theorem was used to estimate the models' predictive accuracy.

Results: The overall predictive accuracy of the 2-, 4-, and 8-classification clustering models in the training set was 0.856, 0.8634, and 0.8506, respectively. Similarly, the predictive accuracy of the 2-, 4-, and 8-classification clustering models in the testing set was 0.8598, 0.8659, and 0.8525, respectively. After reducing from 19 dimensions to 2 dimensions by principal component analysis, significant separation was observed for CVD cases and non-CVD cases in both training and testing sets.

Conclusion: Our findings indicate that the utilization of EMR data can support the development of a robust model for CVD detection through an unsupervised ML approach. Further investigation using longitudinal design is needed to refine the model for its applications in clinical settings.

Background: In standard Sequence Analysis, similar trajectories are clustered together to create a typology of trajectories, which is then often used to evaluate the association between sequence patterns and covariates inside regression models. The sampling uncertainty, which affects both the derivation of the typology and the associated regressions, is typically ignored in this analysis, an oversight that may lead to wrong statistical conclusions. We propose utilising sampling variation to derive new estimates that further inform on the association of interest.

Methods: We introduce a novel procedure to assess the robustness of regression results obtained from the standard analysis. Bootstrap samples are drawn from the data, and for each bootstrap, a new typology replicating the original one is constructed, followed by the estimation of the corresponding regression models. The bootstrap estimates are then combined using a multilevel modelling framework that mimics a meta-analysis. The fitted values from this multilevel model allow to account for the sampling uncertainty in the inferential analysis. We illustrate the methodology by applying it to the study of healthcare utilisation trajectories in a Swiss cohort of diabetic patients.

Results: The procedure provides robust estimates for an association of interest, along with 95% prediction intervals, representing the range of expected values if the clustering and associated regressions were performed on a new sample from the same underlying distribution. It also identifies central and borderline trajectories within each cluster. Regarding the illustrative application, while there was evidence of an association between regular lipid testing and subsequent healthcare utilisation patterns in the original analysis, this is not supported in the robustness assessment.

Conclusions: Investigating the relationship between trajectory patterns and covariates is of interest in many situations. However, it is a challenging task with potential pitfalls. Our Robustness Assessment of Regression using Cluster Analysis Typologies (RARCAT) may assist in ensuring the robustness of such association studies. The method is applicable wherever clustering is combined with regression analysis, so its relevance goes beyond State Sequence Analysis.

Background: A barrier to evidence-informed exercise programming is locating studies of exercise training programs. The purpose of this study was to create a search filter for studies of exercise training programs for the PubMed electronic bibliographic database.

Methods: Candidate search terms were identified from three sources: exercise-relevant MeSH terms and their corresponding Entry terms, word frequency analysis of articles in a gold-standard reference set curated from systematic reviews focused on exercise training, and retrospective searching of articles retrieved in the search filter development and testing steps. These terms were assembled into an exercise training search filter, and its performance was assessed against a basic search string applied to six case studies. Search string performance was measured as sensitivity (relative recall), precision, and number needed to read (NNR). We aimed to achieve relative recall ≥ 85%, and a NNR ≥ 2.

Results: The reference set consisted of 71 articles drawn from six systematic reviews. Sixty-one candidate search terms were evaluated for inclusion, 21 of which were included in the finalized exercise-training search filter. The relative recall of the search filter was 96% for the reference set and the precision mean ± SD was 54 ± 16% across the case studies, with the corresponding NNR = ~ 2. The exercise training search filter consistently outperformed the basic search string.

Conclusion: The exercise training search filter fosters more efficient searches for studies of exercise training programs in the PubMed electronic bibliographic database. This search string may therefore support evidence-informed practice in exercise programming.

Background: Traditional clustering techniques are typically restricted to either continuous or categorical variables. However, most real-world clinical data are mixed type. This study aims to introduce a clustering technique specifically designed for datasets containing both continuous and categorical variables to offer better clustering compatibility, adaptability, and interpretability than other mixed type techniques.

Methods: This paper proposed a modified Gower distance incorporating feature importance as weights to maintain equal contributions between continuous and categorical features. The algorithm (DAFI) was evaluated using five simulated datasets with varying proportions of important features and real-world datasets from the 2011-2014 National Health and Nutrition Examination Survey (NHANES). Effectiveness was demonstrated through comparisons with 13 clustering techniques. Clustering performance was assessed using the adjusted Rand index (ARI) for accuracy in simulation studies and the silhouette score for cohesion and separation in NHANES. Additionally, multivariable logistic regression estimated the association between periodontitis (PD) and cardiovascular diseases (CVDs), adjusting for clusters in NHANES.

Results: In simulation studies, the DAFI-Gower algorithm consistently performs better than baseline methods according to the adjusted Rand index in settings investigated, especially on datasets with more redundant features. In NHANES, 3,760 people were analyzed. DAFI-Gower achieves the highest silhouette score (0.79). Four distinct clusters with diverse health profiles were identified. By incorporating feature importance, we found that cluster formations were more strongly influenced by CVD-related factors. The association between periodontitis and cardiovascular diseases, after adjusting for clusters, reveals significant insights (adjusted OR 1.95, 95% CI 1.50 to 2.55, p = 0.012), highlighting severe periodontitis as a potential risk factor for cardiovascular diseases.

Conclusions: DAFI performed better than classic clustering baselines on both simulated and real-world datasets. It effectively captures cluster characteristics by considering feature importance, which is crucial in clinical settings where many variables may be similar or irrelevant. We envisage that DAFI offers an effective solution for mixed type clustering.

Background: Medical record abstraction (MRA) is a commonly used method for data collection in clinical research, but is prone to error, and the influence of quality control (QC) measures is seldom and inconsistently assessed during the course of a study. We employed a novel, standardized MRA-QC framework as part of an ongoing observational study in an effort to control MRA error rates. In order to assess the effectiveness of our framework, we compared our error rates against traditional MRA studies that had not reported using formalized MRA-QC methods. Thus, the objective of this study was to compare the MRA error rates derived from the literature with the error rates found in a study using MRA as the sole method of data collection that employed an MRA-QC framework.

Methods: A comparison of the error rates derived from MRA-centric studies identified as part of a systematic literature review was conducted against those derived from an MRA-centric study that employed an MRA-QC framework to evaluate the effectiveness of the MRA-QC framework. An inverse variance-weighted meta-analytical method with Freeman-Tukey transformation was used to compute pooled effect size for both the MRA studies identified in the literature and the study that implemented the MRA-QC framework. The level of heterogeneity was assessed using the Q-statistic and Higgins and Thompson's I² statistic.

Results: The overall error rate from the MRA literature was 6.57%. Error rates for the study using our MRA-QC framework were between 1.04% (optimistic, all-field rate) and 2.57% (conservative, populated-field rate), 4.00-5.53% points less than the observed rate from the literature (p < 0.0001).

Conclusions: Review of the literature indicated that the accuracy associated with MRA varied widely across studies. However, our results demonstrate that, with appropriate training and continuous QC, MRA error rates can be significantly controlled during the course of a clinical research study.

Background: Transitivity assumption is the cornerstone of network meta-analysis (NMA). Investigating the plausibility of transitivity can unveil the credibility of NMA results. The commonness of transitivity was examined based on study dissimilarities regarding several study-level aggregate clinical and methodological characteristics reported in the systematic reviews. The present study also demonstrated the disadvantages of using multiple statistical tests to assess transitivity and compared the conclusions drawn from multiple statistical tests with those from the approach of study dissimilarities for transitivity assessment.

Methods: An empirical study was conducted using 209 published systematic reviews with NMA to create a database of study-level aggregate clinical and methodological characteristics found in the tracenma R package. For each systematic review, the network of the primary outcome was considered to create a dataset with extracted study-level aggregate clinical and methodological characteristics reported in the systematic review that may act as effect modifiers. Transitivity was evaluated by calculating study dissimilarities based on the extracted characteristics to provide a measure of overall dissimilarity within and between the observed treatment comparisons. Empirically driven thresholds of low dissimilarity were employed to determine the proportion of datasets with evidence of likely intransitivity. One-way ANOVA and chi-squared test were employed for each characteristic to investigate comparison dissimilarity at a significance level of 5%.

Results: Study dissimilarities covered a wide range of possible values across the datasets. A 'likely concerning' extent of study dissimilarities, both intra-comparison and inter-comparison, dominated the analysed datasets. Using a higher dissimilarity threshold, a 'likely concerning' extent of study dissimilarities persisted for objective outcomes but decreased substantially for subjective outcomes. A likely intransitivity prevailed in all datasets; however, using a higher dissimilarity threshold resulted in few networks with transitivity for semi-objective and subjective outcomes. Statistical tests were feasible in 127 (61%) datasets, yielding conflicting conclusions with the approach of study dissimilarities in many datasets.

Conclusions: Study dissimilarity, manifested from variations in the effect modifiers' distribution across the studies, should be expected and properly quantified. Measuring the overall study dissimilarity between observed comparisons and comparing it with a proper threshold can aid in determining whether concerns of likely intransitivity are warranted.