Diagnostic and prognostic research最新文献

Background: Some prognostic factors (PF) are "versatile" insofar as they predict diverse health outcomes (age is an exemplar par excellence). In this study, we sought to quantify the versatility of commonly measured PFs.

Methods: Participants from the UKB (UK biobank) study were followed from enrolment until the date of outcome or censoring. Over 800 incident adverse outcomes were considered, based on a unique 3-digit ICD code (A00, A01, A02, etc.). Twenty-four routine PFs-including renal, liver function and blood count biomarkers-featured in this analysis. Cox regression was used to determine the association between each PF with time to each outcome event. The number of statistically significant associations, direction of the association (positive/negative) and the median log hazard ratio (LHR) were determined for each PF. Data were visualised using Volcano plots.

Results: The analysis included up to 502,408 UKB participants followed for 12.4 years. PFs with the greatest number of statistically significant associations were age (563/836; median LHR: 0.47), waist-hip ratio (530/836; LHR: 0.37) and hand-grip strength (416/836; LHR: 0.27). Conversely, PFs with the lowest number of significant associations were diastolic blood pressure (138/835; LHR: 0.11) and total protein (138/835; LHR: 0.11). Positive correlation was observed between the number of events a PF was associated with and the average effect size for those associations.

Conclusion: A wide spectrum exists between the most and least versatile PFs. In addition to age, waist-hip ratio and handgrip strength exhibit high versatility. Understanding PF versatility has implications for optimising the development/performance of prognostic models.

The recent uptake in certified Artificial Intelligence (AI) tools for healthcare applications has renewed the debate around their adoption. Explainable AI, the sub-discipline promising to render AI devices more transparent and trustworthy, has also come under scrutiny as part of this discussion. Some experts in the medical AI space debate the reliability of Explainable AI techniques, expressing concerns on their use and inclusion in guidelines and standards. Revisiting such criticisms, this article offers a balanced perspective on the utility of Explainable AI, focusing on the specificity of clinical applications of AI and placing them in the context of healthcare interventions. Against its detractors and despite valid concerns, we argue that the Explainable AI research program is still central to human-machine interaction and ultimately a useful tool against loss of control, a danger that cannot be prevented by rigorous clinical validation alone.

Background: Several clinical prediction models that predict the risk of chronic kidney disease (CKD) in people with diabetes have been developed; however, these models lack external validation demonstrating accurate predictions in Canadian primary care. We externally validated existing clinical prediction models for CKD in Canadian primary care data, overall and across subgroups defined by sex/gender, age, comorbidities, and neighbourhood-level deprivation.

Methods: We conducted a retrospective cohort study using data from the Canadian Primary Care Sentinel Surveillance Network (CPCSSN) electronic medical record database (2014-2019). We identified models that use demographic, health behaviour, clinical and diabetes-related characteristics to predict incident CKD based on two recent systematic reviews and included models with sufficient predictors in CPCSSN (≤ 1 unavailable) and eGFR-based CKD definitions. We included adult patients (18 +) with diabetes without an existing diagnosis of CKD. We identified incident cases of CKD within 5 years based on ≥ 2 laboratory values corresponding to eGFR < 60 mL/min/1.73 m² separated by ≥ 90 days and ≤ 1 year. For each model, we estimated the discrimination, precision, recall, and calibration within CPCSSN.

Results: Among 37,604 patients with diabetes, 14.6% met diagnostic criteria for CKD within 5 years. Overall performance of the 13 included CKD prediction models in CPCCSN was mixed: three models displayed moderate to strong discrimination (areas under the receiver-operating characteristic curves [AUROCs] > 0.70), whereas other AUROCs were as Low as 0.508. After model updating, calibrations were heterogeneous with most models displaying some miscalibration. Some subgroups displayed considerable differences in performance: discriminative performance (AUROC) declined with increasing age and number of comorbidities, whereas the precision and recall improved with increasing age and number of comorbidities. We observed no difference in performance according to sex/gender or deprivation quintile.

Conclusions: Three models displayed moderate to strong performance predicting CKD among CPCSSN patients. Next, these models should be evaluated for their impact on practitioner and patient outcomes when implemented in clinical practice. If successful, these models hold promise in achieving widespread adoption to help identify those at highest risk of CKD and guide therapies that may prevent or delay CKD and related sequelae (e.g., end-stage renal disease) among people with diabetes.

Objective: Large language models (LLMs) are attracting increasing interest in healthcare. This commentary evaluates the potential of LLMs to improve clinical prediction models (CPMs) for diagnostic and prognostic tasks, with a focus on their ability to process longitudinal electronic health record (EHR) data.

Findings: LLMs show promise in handling multimodal and longitudinal EHR data and can support multi-outcome predictions for diverse health conditions. However, methodological, validation, infrastructural, and regulatory challenges remain. These include inadequate methods for time-to-event modelling, poor calibration of predictions, limited external validation, and bias affecting underrepresented groups. High infrastructure costs and the absence of clear regulatory frameworks further prevent adoption.

Implications: Further work and interdisciplinary collaboration are needed to support equitable and effective integration into the clinical prediction. Developing temporally aware, fair, and explainable models should be a priority focus for transforming clinical prediction workflow.

Five years ago, Korevaar and colleagues proposed a framework for designing diagnostic accuracy studies, focusing on the definition of clear study hypotheses. This proposal filled a gap and was well received by the scientific community. In this commentary, I suggest five potential refinements. They aim at increasing the flexibility of the framework while pertaining its logical consistency. The refinements address the following five topics: (1) The relationship between minimal criteria and the choice of the null hypothesis region; (2) The potential to allow compensation between sensitivity and specificity; (3) The possibility to use other pairs than sensitivity and specificity; (4) The potential phrasing as an estimation problem; (5) The advantages of directly moving to a comparative accuracy study.

Background: When developing/validating prognostic models, it is typical to assess calibration between predicted and observed risks - either in the development dataset or in an external sample. For competing risks data, correct specification of more than one model may be required to ensure well-calibrated predicted risks for the event of interest. Furthermore, interest may be in the predicted risks of the event of interest, competing events and all-causes. Therefore, calibration must be assessed simultaneously using various measures.

Methods: We focus on the calibration of prediction models for external validation using a cause-specific hazards approach. We propose that miscalibration for cause-specific hazard models be assessed using components specific to each model through the complement of the cause-specific survival alongside the assessment of the calibration of the cause-specific absolute risks. We simulated a range of scenarios to illustrate how to identify which model(s) are mis-specified in an external validation setting. Calibration plots and calibration statistics (calibration slope, calibration-in-the-large) are presented alongside performance measures such as the Brier score and Index of Prediction Accuracy. We use pseudo-observations to calculate observed risks and generate a smooth calibration curve with restricted cubic splines. We fitted flexible parametric survival models to the simulated data to flexibly estimate baseline cause-specific hazards for the prediction of individual cause-specific absolute risks.

Results: Our simulations illustrate that miscalibration due to changes in the baseline cause-specific hazards in external validation data is better identified using components from each cause-specific model. A mis-calibrated model on one cause could lead to poor calibration of the predicted absolute risks for each cause of interest, including the all-cause absolute risk. This is because prediction of a single cause-specific absolute risk is impacted by effects of variables on the cause of interest and competing events.

Conclusions: If accurate predictions for both all-cause and each cause-specific absolute risks are of interest, this is best achieved by developing and validating models via the cause-specific hazards approach. For each cause-specific model, researchers should evaluate calibration plots separately using the complement of the cause-specific survival function to reveal the cause of any miscalibration. However, this also requires careful consideration of dependent censoring which must be sufficiently accounted for.

Background: When a machine learning model is developed and evaluated in a setting where the treatment assignment process differs from the setting of intended model deployment, failure to account for this difference can lead to suboptimal model development and biased estimates of model performance.

Methods: We consider the setting where data from a randomized trial and an observational study emulating the trial are available for machine learning model development and evaluation. We provide two approaches for estimating the model and assessing model performance under a hypothetical treatment strategy in the target population underlying the observational study. The first approach uses counterfactual predictions from the observational study only and relies on the assumption of conditional exchangeability between treated and untreated individuals (no unmeasured confounding). The second approach leverages the exchangeability between treatment groups in the trial (supported by study design) to "transport" estimates from the trial to the population underlying the observational study, relying on an additional assumption of conditional exchangeability between the populations underlying the observational study and the randomized trial.

Results: We examine the assumptions underlying both approaches for fitting the model and estimating performance in the target population and provide estimators for both objectives. We then develop a joint estimation strategy that combines data from the trial and the observational study, and discuss benchmarking of the trial and observational results.

Conclusions: Both the observational and transportability analyses can be used to fit a model and estimate performance under a counterfactual treatment strategy in the population underlying the observational data, but they rely on different assumptions. In either case, the assumptions are untestable, and deciding which method is more appropriate requires careful contextual consideration. If all assumptions hold, then combining the data from the observational study and the randomized trial can be used for more efficient estimation.

Background: Hospital-acquired venous thromboembolism (HA-VTE) is a leading cause of morbidity and mortality among hospitalized adults. Numerous prognostic models have been developed to identify those patients with elevated risk of HA-VTE. None, however, has met the necessary criteria to guide clinical decision-making. This study outlines a protocol for refining and validating a general-purpose prognostic model for HA-VTE, designed for real-time automation within the electronic health record (EHR) system.

Methods: A retrospective cohort of 132,561 inpatient encounters (89,586 individual patients) at a large academic medical center will be collected, along with clinical and demographic data available as part of routine care. Data for temporal, geographic, and domain external validation cohorts will also be collected. Logistic regression will be used to predict occurrence of HA-VTE during an inpatient encounter. Variables considered for model inclusion will be based on prior demonstrated association with HA-VTE and their availability in both retrospective EHR data and routine clinical care. Least absolute shrinkage and selection operator (LASSO) with tenfold cross-validation will be used for initial variable selection. Variables selected by the LASSO procedure, along with those deemed necessary by clinicians, will be used in an unpenalized multivariable logistic regression model. Discrimination and calibration will be reported for the derivation and validation cohorts. Discrimination will be measured using Harrell's C statistic. Calibration will be measured using calibration intercept, calibration slope, Brier score, integrated calibration index, and visual examination of non-linear calibration curve. Model reporting will adhere to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis guidelines for clinical prediction models using machine learning methods (TRIPOD + AI).

Discussion: We describe methods for developing, evaluating, and validating a prognostic model for HA-VTE using routinely collected EHR data. By combining best practices in statistical development and validation, knowledge engineering, and clinical domain knowledge, the resulting model should be well suited for real-time clinical implementation. Although this protocol describes our development of a model for HA-VTE, the general approach can be applied to other clinical outcomes.

Background: A subset of COVID-19 patients develops post-COVID-19 condition (PCC). This condition results in disability in numerous areas of patients' lives and a reduced health-related quality of life, with societal impact including work absences and increased healthcare utilization. There is a scarcity of models predicting PCC, especially those considering the severity of the initial severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection and incorporating long-term follow-up data. Therefore, we developed and internally validated a prediction model for PCC 2 years after SARS-CoV-2 infection in a cohort of COVID-19 patients.

Methods: Data from the CORona Follow-Up (CORFU) study were used. This research initiative integrated data from multiple Dutch COVID-19 cohort studies. We utilized 2-year follow-up data collected via the questionnaires between October 1st of 2021 and December 31st of 2022. Participants were former COVID-19 patients, approximately 2-year post-SARS-CoV-2 infection. Candidate predictors were selected based on literature and availability across cohorts. The outcome of interest was the prevalence of PCC at 2 years after the initial infection. Logistic regression with backward stepwise elimination identified significant predictors such as sex, BMI and initial disease severity. The model was internally validated using bootstrapping. Model performance was quantified as model fit, discrimination and calibration.

Results: In total 904 former COVID-19 patients were included in the analysis. The cohort included 146 (16.2%) non-hospitalized patients, 511 (56.5%) ward admitted patients, and 247 (27.3%) intensive care unit (ICU) admitted patients. Of all participants, 551 (61.0%) participants suffered from PCC. We included 20 candidate predictors in the multivariable analysis. The final model, after backward elimination, identified sex, body mass index (BMI), ward admission, ICU admission, and comorbidities such as arrhythmia, asthma, angina pectoris, previous stroke, hernia, osteoarthritis, and rheumatoid arthritis as predictors of post-COVID-19 condition. Nagelkerke's R-squared value for the model was 0.19. The optimism-adjusted AUC was 71.2%, and calibration was good across predicted probabilities.

Conclusions: This internally validated prediction model demonstrated moderate discriminative ability to predict PCC 2 years after COVID-19 based on sex, BMI, initial disease severity, and a collection of comorbidities.

Background: Helicobacter pylori (H. pylori) is a bacterium that colonizes the stomach and is a major risk factor for gastric cancer, with an estimated 89% of non-cardia gastric cancer cases worldwide attributable to H. pylori. Prospective studies provide reliable evidence for quantifying the association between gastric cancer and H. pylori, as they circumvent the risk of a false negative due to possible reduction in antibody levels before cancer development.

Methods: In a large-scale prospective study within the China Kadoorie Biobank, H. pylori infection is being analysed as a risk factor for gastric cancer. The presence of infection is typically determined by serological tests. The immunoblot test, although well established, is more labour intensive and uses a larger amount of plasma than the alternative high-throughput multiplex serology test. Immunoblot outputs a binary positive/negative serostatus classification, while multiplex outputs a vector of continuous antigen measurements. When mapping such multidimensional continuous measurements onto a binary classification, statistical challenges arise in defining classification cut-offs and accounting for the differences in infection evidence provided by different antigens. We discuss these challenges and propose a novel solution to optimize the translation of the continuous measurements from multiplex serology into probabilities of H. pylori infection, using classification algorithms (Bayesian additive regressive trees (BART), multidimensional monotone BART, logistic regression, random forest and elastic net). We (i) calibrate and apply classification models to predict probabilities of H. pylori infection given multiplex measurements, (ii) compare the predictive performance of the models using immunoblot as reference, (iii) discuss reasons for the differences in predictive performance and (iv) apply the calibrated models to gain insights on the relative strengths of infection evidence provided by the various antigens.

Results: All models showed high discriminative ability with at least 95% area under the curve (AUC) estimates on the training and test data. There was no substantial difference between the performance of models on the training and test data.

Conclusions: Classification algorithms can be used to calibrate the H. pylori multiplex serology test to the immunoblot test in the China Kadoorie Biobank. This study furthers our understanding of the applicability of classification algorithms to the context of serologic tests.