BMC Medical Research Methodology最新文献

Background: Incorporating backfill cohorts in phase I oncology trials is a recently developed strategy for dose optimization. However, the efficacy assessment window is long in general, causing a lag in identifying ineffective doses and more patients being backfilled to those doses. There is necessity to investigate how to use patient-reported outcomes (PRO) to determine doses for backfill cohorts.

Methods: We propose a unified Bayesian design framework, called 'Backfill-QoL', to utilize patient-reported quality of life (QoL) data into phase I oncology trials with backfill cohorts, including methods for trial monitoring, algorithm for dose-finding, and criteria for dose selection. Simulation studies and sensitivity analyses are conducted to evaluate the proposed Backfill-QoL design.

Results: The simulation studies demonstrate that the Backfill-QoL design is more efficient than traditional dose-expansion strategy, and fewer patients would be allocated to doses with unacceptable QoL profiles. A user-friendly Windows desktop application is developed and freely available for implementing the proposed design.

Conclusions: The Backfill-QoL design enables continuous monitoring of safety, efficacy and QoL outcomes, and the recommended phase II dose (RP2D) can be identified in a more patient-centered perspective.

Background: Adolescent mental health surveys in public health are sometimes questioned because of their main focus on self-reported symptoms, lacking data on impairment, e.g. the consequences on everyday life of the mental health problems. While public health studies typically reveal higher prevalence rates of internalising problems for girls than boys, there are indications that the gender pattern may change when self-reported data on symptoms are analysed simultaneously with impairment. The purpose is to determine how gender patterns of adolescent mental health solely based on symptoms are affected when level of symptoms is analysed simultaneously with impairment.

Methods: Questionnaire data on adolescent mental health were collected in schools by Statistics Sweden in the autumn of 2009 as part of a national total population study in grades 6 and 9 in Sweden. In this study only data from grade 9 students are used (n = 91 627; response rate = 80 per cent). Psychosomatic symptoms were measured with the Psychosomatic Problems scale including eight items. Impairment was measured with four items included in the SDQ impact supplement. The associations between these key constructs were analysed with logistic regression and contingency tables.

Results: When analysing variables on psychosomatic symptoms and impairment independently, the results are consistent with typical findings of gender patterns in adolescent internalising mental health. Girls report both more psychosomatic symptoms, and more negative consequences in everyday life, than boys. The gender patterns are, however, strongly affected when impairment is conditioned on level of psychosomatic symptoms. Except for the Home Life setting, in the settings of Friendships, Classroom Learning and Leisure Activities, the previously reported gender pattern favoring higher disturbances among girls becomes partly reversed implying that boys report more negative consequences than girls. Hence, while girls report a higher prevalence of psychosomatic symptoms, boys appear to suffer from such symptoms more than, or as much as, girls in three out of four everyday life settings.

Conclusions: The study confirms the insufficiency of solely including data on symptoms in the measurement of adolescent mental health. Regardless of the causes of the complex gender pattern shown in this study, the results highlight the importance of simultaneous inclusion of indicators of impairment as well as symptom counts and frequency in the measurement of adolescent mental health.

Background: Ring-based studies are a novel research design commonly used for research involving infectious diseases: contacts of newly infected individuals form a ring that is targeted for interventions (e.g., vaccine, post-exposure prophylaxis). Given the novelty of the research design, it is critical to obtain feedback from participants on their experiences with ring-based studies to help with the development of future trials.

Methods: In 2021, we conducted 26 semi-structured interviews with adult participants of a COVID-19 ring-based post-exposure prophylaxis trial based in Canada. We applied a purposive sampling approach and electronically recruited participants who tested positive for COVID-19 (Index Cases) and either agreed or declined for the study team to contact their potentially exposed contacts. We also included individuals who participated in the trial after being potentially exposed to an Index Case (known as Ring Members), and those who declined to participate after potential exposure. The methodological design of semi-structured interviews allowed participants to share their opinions and experiences in the trial (e.g., elements they enjoyed and disliked regarding their participation in the study).

Results: The majority of participants in our study were women (62%) and the average age was 37.3 years (SD = 13.2). Overall, participants reported being highly satisfied with partaking in the ring-based trial. Notably, no substantial complaints were voiced about the trial's design involving contact after exposure. The most common reason of satisfaction was the knowledge of potentially helping others by advancing knowledge for a greater cause (e.g., development of potential treatment to prevent SARS-CoV-2 infection). Other reasons were curiosity about participating in a trial, and an activity to fill free time during the pandemic. A central element of dislike was confusion about instructions with the trial (e.g., independent at home SARS-CoV-2 testing). Additionally, maintaining confidentiality was a crucial concern for participants, who sought assurance that their data would not be shared beyond the scope of the study.

Conclusions: Our results have the potential to inform future research, including clinical trials such as ring-based studies, by incorporating insights from participants' experiences into the development of study protocols. Despite some protocol-related challenges, participants expressed high satisfaction, driven by the desire to advance science and potentially aid others.

Background: Assessing the methodological quality of case reports and case series is challenging due to human judgment variability and time constraints. We evaluated the agreement in judgments between human reviewers and GPT-4 when applying a standard methodological quality assessment tool designed for case reports and series.

Methods: We searched Scopus for systematic reviews published in 2023-2024 that cited the appraisal tool by Murad et al. A GPT-4 based agent was developed to assess the methodological quality using the 8 signaling questions of the tool. Observed agreement and agreement coefficient were estimated comparing published judgments of human reviewers to GPT-4 assessment.

Results: We included 797 case reports and series. The observed agreement ranged between 41.91% and 80.93% across the eight questions (agreement coefficient ranged from 25.39 to 79.72%). The lowest agreement was noted in the first signaling question about selection of cases. The agreement was similar in articles published in journals with impact factor < 5 vs. ≥ 5, and when excluding systematic reviews that did not use 3 causality questions. Repeating the analysis using the same prompts demonstrated high agreement between the two GPT-4 attempts except for the first question about selection of cases.

Conclusions: The study demonstrates a moderate agreement between GPT-4 and human reviewers in assessing the methodological quality of case series and reports using the Murad tool. The current performance of GPT-4 seems promising but unlikely to be sufficient for the rigor of a systematic review and pairing the model with a human reviewer is required.

Background: The use of historical external control data in clinical trials has grown in interest and needs when considering the design of future trials. Hybrid control designs can be more efficient to achieve the same power with fewer patients and limited resources. The literature is sparse on appropriate statistical methods which can account for the differences between historical external controls and the control patients in a study. In this article, we illustrate the analysis framework of a clinical trial if a hybrid control design was used after determining an RCT may not be feasible.

Methods: We utilize two previously completed RCTs in nonsquamous NSCLC and a nationwide electronic health record derived de-identified database as examples and compare 5 analysis methods on each trial, as well as a set of simulations to determine operating characteristics of such designs.

Results: In single trial estimation, the Case Weighted Adaptive Power Prior provided estimated treatment hazard ratios consistent with the original trial's conclusions with narrower confidence intervals. The simulation studies showed that the Case Weighted Adaptive Power Prior achieved the highest power (and well controlled type-1 error) across all 5 methods with consistent study sample size.

Conclusions: By following the proposed hybrid control framework, one can design a hybrid control trial transparently and accounting for differences between control groups while controlling type-1 error and still achieving efficiency gains from the additional contribution from external controls.

Background: Rapid timescales for the design and delivery of research were common during the COVID-19 pandemic. The recruitment and retention of healthcare workers (HCWs) as participants in research studies are notoriously challenging, but this was exacerbated during the pandemic by the unprecedented demand placed on the workforce. The SARS-CoV-2 Immunity and Reinfection Evaluation (SIREN study) is a prospective multicentre cohort study following HCWs in the UK. This paper discusses the strategies and challenges associated with recruitment and retention of HCW participants in Scotland.

Methods: There were 44,546 HCWs recruited to the SIREN study, of whom 6,285 were recruited by research teams at ten different research sites in Scotland between October 2020 and March 2021. Information on target and actual sample size, availability of resource, recruitment rate, and recruitment and engagement strategies by site was collated from SIREN study documentation and discussions with local key SIREN site staff. Individual-level data from 6,153 HCW participants with ongoing consent for all data usage were also collated, including socio-demographic data and information on withdrawal (in first year) and opt-in to a study extension after one year. Factors associated with these outcomes were explored in logistic regression analyses.

Results: Different recruitment strategies were used in each site according to local agreements, protocol and staff capacity, with the recruitment period ranging from 13 to 160 days. The locally-agreed recruitment target was met in four sites. The proportion of participants who withdrew in the first year ranged from 3.1 to 24.8% by site, while subsequent opt-in to a 12-month study extension ranged from 28.6 to 74.8%. The sites with the highest proportions of withdrawals were the same four sites with lowest proportions of opt-in. On an individual level, there was a lower level of retention among younger participants, and those from lower socio-economic backgrounds and minority ethnic groups.

Conclusions: Site-specific factors including research-readiness likely had a significant influence on recruitment and retention, more so than the specific recruitment or retention strategies employed. Independent of site factors, individual-level variables influenced recruitment and retention, suggesting targeted strategies may be needed to promote research engagement among particular socio-demographic groups.

Background: The current practice in guideline development is to use the control group event rate (CR) as a surrogate of baseline risk and to assume portability of the relative treatment effect across populations with low, moderate and high baseline risk. We sought to emulate this practice in a very large sample of meta-analyses.

Methods: We retrieved data from all meta-analyses published in the Cochrane Database of Systematic Reviews (2003-2020) that evaluated a binary outcome, reported 2 × 2 data for each individual study and included at least 4 studies. We excluded studies with no events. We conducted meta-analyses with odds ratios and relative risks and performed subgroup analyses based on tertiles of CR. In sensitivity analyses, we evaluated the use of total event rate (TR) instead of CR and using quartiles instead of tertiles.

Results: The analysis included 2,531 systematic reviews (27,692 meta-analyses, 226,975 studies, 25,669,783 patients).The percentages of meta-analyses with statistically significant interaction (P < 0.05) based on CR tertile or quartile ranged 12-18% across various sensitivity analyses. This percentage increased as the number of studies or range of CR per meta-analysis increased, reflecting increased power of the subgroup test. The percentages of meta-analyses with statistically significant interaction (P < 0.05) with TR quantiles were lower than those with CR but remained higher than expected by chance.

Conclusion: This analysis suggests that when CR or TR are used as surrogates for baseline risk, relative treatment effects may not be portable across populations with varying baseline risks in many meta-analyses. Categroization of the continuous CR variable and not addressing measurement error limit inferences from such analyses and imply that CR is an undesirable source for baseline risk. Guideline developers and decision-makers should be provided with relative and absolute treatment effects that are conditioned on the baseline risk or derived from studies with similar baseline risk to their target populations.

Background: Asthma is a heterogeneous disease that affects millions of children and adults. There is a lack of objective gold standard diagnosis that spans the ages; instead, diagnoses are made by clinician assessment based on a cluster of signs, symptoms and objective tests dependent on age. Yet, there is a clear morbidity associated with chronic asthma symptoms. Machine learning has become a popular tool to improve asthma diagnosis and classification. There is a paucity of literature on the use of Bayesian machine learning algorithms to predict asthma diagnosis in children. This paper develops a prediction model using the Bayesian additive regression trees (BART) and compares its performance to various machine learning algorithms in predicting the diagnosis of childhood asthma.

Methods: Clinically relevant variables collected at or before 3 years of age from 2794 participants in the CHILD Cohort Study were used to predict physician-diagnosed asthma at age 5. BART and six other commonly used machine learning algorithms, namely adaptive boosting, logistic regression, decision tree, neural network, random forest, and support vector machine were trained. Measures of performance including sensitivity, specificity, and area under the receiver operating characteristic (ROC) curve were calculated. The confidence intervals were calculated using Bootstrapping samples. Important predictors and interaction effects associated with asthma were also identified using BART.

Results: BART, logistic regression and random forest showed the highest area under the ROC curve compared to other machine learning algorithms. Based on BART, recurrent wheeze, respiratory infection and food sensitization at 3 years of age were the most important predictors. The three most important interaction effects were found to be interaction terms of respiratory infection at 3 years and recurrent wheezing at 3 years, maternal asthma and paternal asthma, and maternal wheezing and inhalant sensitization of child at 3 years.

Conclusions: BART demonstrated promising prediction performance when compared to other machine learning algorithms. Future research could validate the BART in an external cohort to evaluate its reliability and generalizability.

Background: Medical outcomes of interest to clinicians may have multiple categories. Researchers face several options for risk prediction of such outcomes, including dichotomized logistic regression and multinomial logit regression modeling. We aimed to compare these methods and provide guidance needed for practice.

Methods: We described dichotomized logistic regression, multinomial continuation-ratio logit regression, which is an alternative to standard multinomial logit regression for ordinal outcomes, and logistic competing risks regression. We then applied these methods to develop prediction models of survival and neurodevelopmental outcomes based on the NICHD Extremely Preterm Birth Outcome Tool model. The statistical and practical advantages and flaws of these methods were examined. Both discrimination and calibration of the estimated logistic models of dichotomized outcomes and continuation-ratio logit model were assessed.

Results: The dichotomized logistic models and multinomial continuation-ratio logit model had similar discrimination and calibration in predicting death and survival without neurodevelopmental impairment. But the continuation-ratio logit model had better discrimination and calibration in predicting neurodevelopmental impairment. The sum of predicted probabilities of outcome categories from the dichotomized logistic models could deviate from 100% substantially, ranging from 87.7 to 124.0%, and the dichotomized logistic model of neurodevelopmental impairment greatly overpredicted low risks and underpredicted high risks.

Conclusions: Estimating multiple logistic regression models of dichotomized outcomes may result in poorly calibrated predictions for an outcome with multiple ordinal categories. Multinomial continuation-ratio logit regression produces better calibrated predictions, constrains the sum of predicted probabilities to 100%, and has the advantages of simplicity in model interpretation, flexibility to include outcome category-specific predictors and random-effect terms for patient heterogeneity by hospital. It also accounts for mutual dependence among multiple categories and accommodates competing risks.

Background: Randomised controlled trials (RCTs) are considered the gold standard for generating clinical evidence. The focus on high internal validity in RCTs challenges the external validity and generalisability of findings, potentially hindering their application in routine care. In neurorehabilitation, limited literature addresses conducting RCTs feasibly and efficiently. We investigated barriers and facilitators to conducting RCTs within routine care of neurorehabilitation centres from the perspective of stakeholders in neurorehabilitation in Germany and Austria.

Methods: We conducted semi-structured interviews with stakeholders in neurorehabilitation from four centres in Germany and Austria, informed by the Theoretical Domains Framework (TDF) and the Capability, Opportunity, Motivation and Behaviour model (COM-B). Employing a hybrid approach, the interview analysis integrated both deductive, theory-driven analysis based on the TDF domains and COM-B model and inductive, reflexive thematic analysis.

Results: Twelve stakeholders (4 physicians, 4 therapy managers, 4 therapists; 5 females, 7 males; with research experience spanning 0-40 years) were interviewed. Key barriers to conducting RCTs in neurological rehabilitation centres include limited financial, human, and time resources, high clinical workloads, and a lack of interest of some therapists. Ineffective leadership, perceived lack of research expertise, and communication issues were also significant barriers. Social influence factors such as lack of employer support and inadequate training access further contributed to the challenges. Additionally, barriers included insufficient research infrastructure, limited space, internal power struggles, and rigid cost bearer specifications. Key facilitators included physicians' and therapists' motivation to advance the field, contribute to knowledge, and to prioritise patient health. Support from supervisors, joint decision-making, and efficient organisation were crucial facilitators. Flexible therapy planning, mutual support, and interdisciplinary collaboration also played important roles.

Conclusion: Our results suggest that increasing professional development and understanding, along with providing adequate financial, human, time, and spatial resources to support research endeavours, implementing effective communication strategies to enhance interdisciplinary collaboration and coordination among team members may contribute to increased motivation and facilitate RCTs within the setting of neurorehabilitation centres.

Trial registration: This study was prospectively registered with the German Clinical Trials Register (08.04.2021 DRKSID DRKS00024982).