Diagnostic and prognostic research最新文献

Background: Pressure injuries (PIs) place a substantial burden on healthcare systems worldwide. Risk stratification of those who are at risk of developing PIs allows preventive interventions to be focused on patients who are at the highest risk. The considerable number of risk assessment scales and prediction models available underscores the need for a thorough evaluation of their development, validation, and clinical utility. Our objectives were to identify and describe available risk prediction tools for PI occurrence, their content and the development and validation methods used.

Methods: The umbrella review was conducted according to Cochrane guidance. MEDLINE, Embase, CINAHL, EPISTEMONIKOS, Google Scholar, and reference lists were searched to identify relevant systematic reviews. The risk of bias was assessed using adapted AMSTAR-2 criteria. Results were described narratively. All included reviews contributed to building a comprehensive list of risk prediction tools.

Results: We identified 32 eligible systematic reviews only seven of which described the development and validation of risk prediction tools for PI. Nineteen reviews assessed the prognostic accuracy of the tools and 11 assessed clinical effectiveness. Of the seven reviews reporting model development and validation, six included only machine learning models. Two reviews included external validations of models, although only one review reported any details on external validation methods or results. This was also the only review to report measures of both discrimination and calibration. Five reviews presented measures of discrimination, such as the area under the curve (AUC), sensitivities, specificities, F1 scores, and G-means. For the four reviews that assessed the risk of bias assessment using the PROBAST tool, all models but one were found to be at high or unclear risk of bias.

Conclusions: Available tools do not meet current standards for the development or reporting of risk prediction models. The majority of tools have not been externally validated. Standardised and rigorous approaches to risk prediction model development and validation are needed.

Trial registration: The protocol was registered on the Open Science Framework ( https://osf.io/tepyk ).

Background: The initial theme of the PROGRESS framework for prognosis research is termed overall prognosis research. Its aim is to describe the most likely course of health conditions in the context of current care. These average group-level prognoses may be used to inform patients, health policies, trial designs, or further prognosis research. Acquired brain injury, such as stroke, traumatic brain injury or encephalopathy, is a major cause of disability and functional limitations, worldwide. Rehabilitation aims to maximize independent functioning and meaningful participation in society post-injury. While some observational studies can allow for an inference of the overall prognosis of the level of independent functioning, the context for the provision of rehabilitation is rarely described. The aim of this protocol is to provide a detailed account of the clinical context to aid the interpretation of our upcoming overall prognosis study.

Methods: The study will occur at a Danish post-acute inpatient rehabilitation facility providing specialised inpatient rehabilitation for individuals with moderate to severe acquired brain injury. Routinely collected electronic health data will be extracted from the healthcare provider's database and deterministically linked on an individual level to construct the study cohort. The study period spans from March 2011 to December 2022. Four outcomes will measure the level of functioning. Rehabilitation needs will also be described. Outcomes and rehabilitation needs will be described for the entire cohort, across rehabilitation complexity levels and stratified for relevant demographic and clinical parameters. Descriptive statistics will be used to estimate average prognoses for the level of functioning at discharge from post-acute rehabilitation. The patterns of missing data will be investigated.

Discussion: This protocol is intended to provide transparency in our upcoming study based on routinely collected clinical data. It will aid in the interpretation of the overall prognosis estimates within the context of our current clinical practice and the assessment of potential sources of bias independently.

Background: We evaluated the performance of prognostic models for predicting mortality or ICU admission in hospitalized patients with COVID-19 in the World Health Organization (WHO) Global Clinical Platform, a repository of individual-level clinical data of patients hospitalized with COVID-19, including in low- and middle-income countries (LMICs).

Methods: We identified eligible multivariable prognostic models for predicting overall mortality and ICU admission during hospital stay in patients with confirmed or suspected COVID-19 from a living review of COVID-19 prediction models. These models were evaluated using data contributed to the WHO Global Clinical Platform for COVID-19 from nine LMICs (Burkina Faso, Cameroon, Democratic Republic of Congo, Guinea, India, Niger, Nigeria, Zambia, and Zimbabwe). Model performance was assessed in terms of discrimination and calibration.

Results: Out of 144 eligible models, 140 were excluded due to a high risk of bias, predictors unavailable in LIMCs, or insufficient model description. Among 11,338 participants, the remaining models showed good discrimination for predicting in-hospital mortality (3 models), with areas under the curve (AUCs) ranging between 0.76 (95% CI 0.71-0.81) and 0.84 (95% CI 0.77-0.89). An AUC of 0.74 (95% CI 0.70-0.78) was found for predicting ICU admission risk (one model). All models showed signs of miscalibration and overfitting, with extensive heterogeneity between countries.

Conclusions: Among the available COVID-19 prognostic models, only a few could be validated on data collected from LMICs, mainly due to limited predictor availability. Despite their discriminative ability, selected models for mortality prediction or ICU admission showed varying and suboptimal calibration.

This systematic review and meta-analysis evaluated reported prevalence and diagnostic methods for identifying Candida africana, an opportunistic yeast associated with vaginal and oral candidiasis. A comprehensive literature search yielded 53 studies meeting the inclusion criteria, 2 of which were case studies. The pooled prevalence of C. africana among 20,571 participants was 0.9% (95% CI: 0.7-1.3%), with significant heterogeneity observed (I² = 79%, p < 0.01). Subgroup analyses revealed regional variations, with North America showing the highest prevalence (4.6%, 95% CI: 1.8-11.2%). The majority 84.52% of the C. africana have been isolated from vaginal samples, 8.37% from oral samples, 3.77% from urine, 2.09% from glans penis swabs, and 0.42% from rectal swabs, nasal swabs, and respiratory tract expectorations respectively. No C. africana has been isolated from nail samples. Hyphal wall protein 1 gene PCR was the most used diagnostic method for identifying C. africana. It has been used to identify 70% of the isolates. A comparison of methods revealed that the Vitek-2 system consistently failed to differentiate C. africana from Candida albicans, whereas MALDI-TOF misidentified several isolates compared with HWP1 PCR. Factors beyond diagnostic methodology may influence C. africana detection rates. We highlight the importance of adapting molecular methods for resource-limited settings or developing equally accurate but more accessible alternatives for the identification and differentiation of highly similar and cryptic Candida species such as C. africana.

Background: Machine learning methods are increasingly being used to predict clinical outcomes. Optimism is the difference in model performance between derivation and validation samples. The term "data hungriness" refers to the sample size needed for a modelling technique to generate a prediction model with minimal optimism. Our objective was to compare the relative data hungriness of different statistical and machine learning methods when assessed using calibration.

Methods: We used Monte Carlo simulations to assess the effect of number of events per variable (EPV) on the optimism of six learning methods when assessing model calibration: unpenalized logistic regression, ridge regression, lasso regression, bagged classification trees, random forests, and stochastic gradient boosting machines using trees as the base learners. We performed simulations in two large cardiovascular datasets each of which comprised an independent derivation and validation sample: patients hospitalized with acute myocardial infarction and patients hospitalized with heart failure. We used six data-generating processes, each based on one of the six learning methods. We allowed the sample sizes to be such that the number of EPV ranged from 10 to 200 in increments of 10. We applied six prediction methods in each of the simulated derivation samples and evaluated calibration in the simulated validation samples using the integrated calibration index, the calibration intercept, and the calibration slope. We also examined Nagelkerke's R², the scaled Brier score, and the c-statistic.

Results: Across all 12 scenarios (2 diseases × 6 data-generating processes), penalized logistic regression displayed very low optimism even when the number of EPV was very low. Random forests and bagged trees tended to be the most data hungry and displayed the greatest optimism.

Conclusions: When assessed using calibration, penalized logistic regression was substantially less data hungry than methods from the machine learning literature.

Background: Random forests have become popular for clinical risk prediction modeling. In a case study on predicting ovarian malignancy, we observed training AUCs close to 1. Although this suggests overfitting, performance was competitive on test data. We aimed to understand the behavior of random forests for probability estimation by (1) visualizing data space in three real-world case studies and (2) a simulation study.

Methods: For the case studies, multinomial risk estimates were visualized using heatmaps in a 2-dimensional subspace. The simulation study included 48 logistic data-generating mechanisms (DGM), varying the predictor distribution, the number of predictors, the correlation between predictors, the true AUC, and the strength of true predictors. For each DGM, 1000 training datasets of size 200 or 4000 with binary outcomes were simulated, and random forest models were trained with minimum node size 2 or 20 using the ranger R package, resulting in 192 scenarios in total. Model performance was evaluated on large test datasets (N = 100,000).

Results: The visualizations suggested that the model learned "spikes of probability" around events in the training set. A cluster of events created a bigger peak or plateau (signal), isolated events local peaks (noise). In the simulation study, median training AUCs were between 0.97 and 1 unless there were 4 binary predictors or 16 binary predictors with a minimum node size of 20. The median discrimination loss, i.e., the difference between the median test AUC and the true AUC, was 0.025 (range 0.00 to 0.13). Median training AUCs had Spearman correlations of around 0.70 with discrimination loss. Median test AUCs were higher with higher events per variable, higher minimum node size, and binary predictors. Median training calibration slopes were always above 1 and were not correlated with median test slopes across scenarios (Spearman correlation - 0.11). Median test slopes were higher with higher true AUC, higher minimum node size, and higher sample size.

Conclusions: Random forests learn local probability peaks that often yield near perfect training AUCs without strongly affecting AUCs on test data. When the aim is probability estimation, the simulation results go against the common recommendation to use fully grown trees in random forest models.

Background: Many clinical pathways for the diagnosis of disease are based on diagnostic tests that are performed in sequence. The performance of the full diagnostic sequence is dictated by the diagnostic performance of each test in the sequence as well as the conditional dependence between them, given true disease status. Resulting estimates of performance, such as the sensitivity and specificity of the test sequence, are key parameters in health-economic evaluations. We conducted a methodological review of statistical methods for assessing the performance of diagnostic tests performed in sequence, with the aim of guiding data analysts towards classes of methods that may be suitable given the design and objectives of the testing sequence.

Methods: We searched PubMed, Scopus and Web of Science for relevant papers describing methodology for analysing sequences of diagnostic tests. Papers were classified by the characteristics of the method used, and these were used to group methods into themes. We illustrate some of the methods using data from a cohort study of repeat faecal immunochemical testing for colorectal cancer in symptomatic patients, to highlight the importance of allowing for conditional dependence in test sequences and adjustment for an imperfect reference standard.

Results: Five overall themes were identified, detailing methods for combining multiple tests in sequence, estimating conditional dependence, analysing sequences of diagnostic tests used for risk assessment, analysing test sequences in conjunction with an imperfect or incomplete reference standard, and meta-analysis of test sequences.

Conclusions: This methodological review can be used to help researchers identify suitable analytic methods for studies that use diagnostic tests performed in sequence.

Interim analysis is a common methodology in randomised clinical trials but has received less attention in studies of diagnostic test accuracy. In such studies, early termination for futility may be beneficial if early evidence indicates that a diagnostic test is unlikely to achieve a clinically useful level of diagnostic performance, as measured by the sensitivity and specificity. In this paper, we describe relevant practical and analytical considerations when planning and performing interim analysis in diagnostic accuracy studies, focusing on stopping rules for futility. We present an adaptation of the exact group sequential method for diagnostic testing, with R code provided for implementing this method in practice. The method is illustrated using two simulated data sets and data from a published diagnostic accuracy study for point-of-care testing for SARS-CoV-2. The considerations described in this paper can be used to guide decisions as to when an interim analysis in a diagnostic accuracy study is suitable and highlight areas for further methodological development.

Background: Patients who suffer from chronic conditions or diseases are susceptible to experiencing repeated events of the same type (e.g. seizures), termed 'recurrent events'. Prediction models can be used to predict the risk of recurrence so that intervention or management can be tailored accordingly, but statistical methodology can vary. The objective of this systematic review was to identify and describe statistical approaches that have been applied for the development and validation of multivariable prediction models with recurrent event data. A secondary objective was to informally assess the characteristics and quality of analysis approaches used in the development and validation of prediction models of recurrent event data.

Methods: Searches were run in MEDLINE using a search strategy in 2019 which included index terms and phrases related to recurrent events and prediction models. For studies to be included in the review they must have developed or validated a multivariable clinical prediction model for recurrent event outcome data, specifically modelling the recurrent events and the timing between them. The statistical analysis methods used to analyse the recurrent event data in the clinical prediction model were extracted to answer the primary aim of the systematic review. In addition, items such as the event rate as well as any discrimination and calibration statistics that were used to assess the model performance were extracted for the secondary aim of the review.

Results: A total of 855 publications were identified using the developed search strategy and 301 of these are included in our systematic review. The Andersen-Gill method was identified as the most commonly applied method in the analysis of recurrent events, which was used in 152 (50.5%) studies. This was closely followed by frailty models which were used in 116 (38.5%) included studies. Of the 301 included studies, only 75 (24.9%) internally validated their model(s) and three (1.0%) validated their model(s) in an external dataset.

Conclusions: This review identified a variety of methods which are used in practice when developing or validating prediction models for recurrent events. The variability of the approaches identified is cause for concern as it indicates possible immaturity in the field and highlights the need for more methodological research to bring greater consistency in approach of recurrent event analysis. Further work is required to ensure publications report all required information and use robust statistical methods for model development and validation.

Prospero registration: CRD42019116031.

Background: In recent years, significant efforts have been directed towards the research and development of disease-modifying therapies for dementia. These drugs focus on prodromal (mild cognitive impairment, MCI) and/or early stages of Alzheimer's disease (AD). Literature evidence indicates that a considerable proportion of individuals with MCI do not progress to dementia. Identifying individuals at higher risk of developing dementia is essential for appropriate management, including the prescription of new disease-modifying therapies expected to become available in clinical practice in the near future.

Methods: The ongoing INTERCEPTOR study is a multicenter, longitudinal, interventional, non-therapeutic cohort study designed to enroll 500 individuals with MCI aged 50-85 years. The primary aim is to identify a biomarker or a set of biomarkers able to accurately predict the conversion from MCI to AD dementia within 3 years of follow-up. The biomarkers investigated in this study are neuropsychological tests (mini-mental state examination (MMSE) and delayed free recall), brain glucose metabolism ([¹⁸F]FDG-PET), MRI volumetry of the hippocampus, EEG brain connectivity, cerebrospinal fluid (CSF) markers (p-tau, t-tau, Aβ1-42, Aβ1-42/1-40 ratio, Aβ1-42/p-Tau ratio) and APOE genotype. The baseline visit includes a full cognitive and neuropsychological evaluation, as well as the collection of clinical and socio-demographic information. Prognostic models will be developed using Cox regression, incorporating individual characteristics and biomarkers through stepwise selection. Model performance will be evaluated in terms of discrimination and calibration and subjected to internal validation using the bootstrapping procedure. The final model will be visually represented as a nomogram.

Discussion: This paper contains a detailed description of the statistical analysis plan to ensure the reproducibility and transparency of the analysis. The prognostic model developed in this study aims to identify the population with MCI at higher risk of developing AD dementia, potentially eligible for drug prescriptions. The nomogram could provide a valuable tool for clinicians for risk stratification and early treatment decisions.

Trial registration: ClinicalTrials.gov NCT03834402. Registered on February 8, 2019.