Diagnostic and prognostic research最新文献

Background: Helicobacter pylori (H. pylori) is a bacterium that colonizes the stomach and is a major risk factor for gastric cancer, with an estimated 89% of non-cardia gastric cancer cases worldwide attributable to H. pylori. Prospective studies provide reliable evidence for quantifying the association between gastric cancer and H. pylori, as they circumvent the risk of a false negative due to possible reduction in antibody levels before cancer development.

Methods: In a large-scale prospective study within the China Kadoorie Biobank, H. pylori infection is being analysed as a risk factor for gastric cancer. The presence of infection is typically determined by serological tests. The immunoblot test, although well established, is more labour intensive and uses a larger amount of plasma than the alternative high-throughput multiplex serology test. Immunoblot outputs a binary positive/negative serostatus classification, while multiplex outputs a vector of continuous antigen measurements. When mapping such multidimensional continuous measurements onto a binary classification, statistical challenges arise in defining classification cut-offs and accounting for the differences in infection evidence provided by different antigens. We discuss these challenges and propose a novel solution to optimize the translation of the continuous measurements from multiplex serology into probabilities of H. pylori infection, using classification algorithms (Bayesian additive regressive trees (BART), multidimensional monotone BART, logistic regression, random forest and elastic net). We (i) calibrate and apply classification models to predict probabilities of H. pylori infection given multiplex measurements, (ii) compare the predictive performance of the models using immunoblot as reference, (iii) discuss reasons for the differences in predictive performance and (iv) apply the calibrated models to gain insights on the relative strengths of infection evidence provided by the various antigens.

Results: All models showed high discriminative ability with at least 95% area under the curve (AUC) estimates on the training and test data. There was no substantial difference between the performance of models on the training and test data.

Conclusions: Classification algorithms can be used to calibrate the H. pylori multiplex serology test to the immunoblot test in the China Kadoorie Biobank. This study furthers our understanding of the applicability of classification algorithms to the context of serologic tests.

Background: Prediction of postoperative patient-reported outcomes and risk for revision surgery after total hip arthroplasty (THA) or total knee arthroplasty (TKA) can inform clinical decision-making, health resource allocation, and care planning. Machine learning (ML) algorithms are increasingly used as an alternative to traditional logistic regression (LR) prediction, but there is uncertainty about their superiority in overall model performance. The aim of this study is to compare the predictive performance of LR with different ML approaches for predicting patient outcomes and risk for revision surgery after THA and TKA.

Methods: A population-based historical cohort study will be developed using routinely collected data from all primary and revision THA and TKA procedures performed in Switzerland and registered in the Swiss National Joint Registry (SIRIS). Patients of age ≥ 18 years with surgery for primary osteoarthritis from 01 January 2015 up to 31 December 2023 will be included. Outcomes of interest will be (1) 12-month postoperative poor pain outcome (defined as < 50% improvement of pain or < 3 absolute reduction in pain on a 11-point (0 to 10) numeric rating scale) and poor satisfaction outcome, and (2) early revision within 5 years after primary surgery. Prespecified predictor variables will include demographic characteristics, comorbidity score, patient-reported health status measures, and surgical variables. Measures of overall predictive accuracy, discrimination, and calibration will be used to compare predictive performance, and decision curve analysis performed to evaluate the clinical usefulness of models. The models will be internally validated using cross-validation and externally validated using geographical validation. Development of the models will be informed by the updated Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD + AI) statement.

Discussion: This study will develop, validate, and compare prediction models for postoperative patient-reported outcomes and risk for revision surgery after THA and TKA using SIRIS data.

Objective: Prognostic outcomes related to hospital admissions typically do not suffer from censoring, and can be modeled either categorically or as time-to-event. Competing events are common but often ignored. We compared the performance of static and dynamic random forest (RF) models to predict the risk of central line-associated bloodstream infections (CLABSI) using different outcome operationalizations.

Methods: We included data from 27,478 admissions to the University Hospitals Leuven, covering 30,862 catheter episodes (970 CLABSI, 1466 deaths and 28,426 discharges) to build static and dynamic RF models for binary (CLABSI vs no CLABSI), multinomial (CLABSI, discharge, death or no event), survival (time to CLABSI) and competing risks (time to CLABSI, discharge or death) outcomes to predict the 7-day CLABSI risk. Static models used information at the onset of the catheter episode, while dynamic models updated predictions daily for 30 days (landmark 0-30). We evaluated model performance across 100 train/test splits.

Results: Performance of binary, multinomial and competing risks models was similar: AUROC was 0.74 for predictions at catheter onset, rose to 0.77 for predictions at landmark 5, and decreased thereafter. Survival models overestimated the risk of CLABSI (E:O ratios between 1.2 and 1.6), and had AUROCs about 0.01 lower than other models. Binary and multinomial models had lowest computation times. Models including multiple outcome events (multinomial and competing risks) display a different internal structure compared to binary and survival models, choosing different variables for early splits in trees.

Discussion and conclusion: In the absence of censoring, complex modelling choices do not considerably improve the predictive performance compared to a binary model for CLABSI prediction in our studied settings. Survival models censoring the competing events at their time of occurrence should be avoided.

Background: Hospitals register information in the electronic health records (EHRs) continuously until discharge or death. As such, there is no censoring for in-hospital outcomes. We aimed to compare different static and dynamic regression modeling approaches to predict central line-associated bloodstream infections (CLABSIs) in EHR while accounting for competing events precluding CLABSI.

Methods: We analyzed data from 30,862 catheter episodes at University Hospitals Leuven from 2012 and 2013 to predict 7-day risk of CLABSI. Competing events are discharge and death. Static models using information at catheter onset included logistic, multinomial logistic, Cox, cause-specific hazard, and Fine-Gray regression. Dynamic models updated predictions daily up to 30 days after catheter onset (i.e., landmarks 0 to 30 days) and included landmark supermodel extensions of the static models, separate Fine-Gray models per landmark time, and regularized multi-task learning (RMTL). Model performance was assessed using 100 random 2:1 train-test splits.

Results: The Cox model performed worst of all static models in terms of area under the receiver operating characteristic curve (AUROC) and calibration. Dynamic landmark supermodels reached peak AUROCs between 0.741 and 0.747 at landmark 5. The Cox landmark supermodel had the worst AUROCs (≤ 0.731) and calibration up to landmark 7. Separate Fine-Gray models per landmark performed worst for later landmarks, when the number of patients at risk was low.

Conclusions: Categorical and time-to-event approaches had similar performance in the static and dynamic settings, except Cox models. Ignoring competing risks caused problems for risk prediction in the time-to-event framework (Cox), but not in the categorical framework (logistic regression).

Background: When using a dataset to develop or update a clinical prediction model, small sample sizes increase concerns of overfitting, instability, poor predictive performance and a lack of fairness. For models estimating the risk of a binary outcome, previous research has outlined sample size calculations that target low overfitting and a precise overall risk estimate. However, more guidance is needed for targeting precise and fair individual-level risk estimates.

Methods: We propose a decomposition of Fisher's information matrix to help examine sample sizes required for developing or updating a model, aiming for precise and fair individual-level risk estimates. We outline a five-step process for use before data collection or when an existing dataset or pilot study is available. It requires researchers to specify the overall risk in the target population, the (anticipated) distribution of key predictors in the model and an assumed 'core model' either specified directly (i.e. a logistic regression equation is provided) or based on a specified C-statistic and relative effects of (standardised) predictors.

Results: We produce closed-form solutions that decompose the variance of an individual's risk estimate into the Fisher's unit information matrix, predictor values and the total sample size. This allows researchers to quickly calculate and examine the anticipated precision of individual-level predictions and classifications for specified sample sizes. The information can be presented to key stakeholders (e.g. health professionals, patients, grant funders) to inform target sample sizes for prospective data collection or whether an existing dataset is sufficient. Our proposal is implemented in our new software module pmstabilityss. We provide two real examples and emphasise the importance of clinical context, including any risk thresholds for decision making and fairness checks.

Conclusions: Our approach helps researchers examine potential sample sizes required to target precise and fair individual-level predictions when developing or updating prediction models for binary outcomes.

Background: To improve screening for atherosclerotic cardiovascular disease (ASCVD), we aimed to develop and temporally evaluate sex-specific models to predict 4-year ASCVD risk in South Limburg based on age and neighbourhood characteristics concerning home address.

Methods: We included 40- to 70-year-olds living in South Limburg on 1 January 2015 for model development, and 40- to 70-year-olds living in South Limburg on 1 January 2016 for model evaluation. We randomly sampled people selected in 1 year and in both years to create development and evaluation data sets. Follow-up of ASCVD and competing events (overall mortality excluding ASCVD) lasted until 31 December 2020. Candidate predictors were the individual's age, the neighbourhood's socio-economic status, and the neighbourhood's particulate matter concentration. Using the evaluation data sets, we compared two model types, subdistribution and cause-specific hazard models, and eight model structures. Discrimination was assessed by the area under the receiver operating characteristic curve (AUROC). Calibration was assessed by calculating overall expected-observed ratios (E/O). For the final models, calibration plots were made additionally.

Results: The development data sets consisted of 67,549 males (4-year cumulative ASCVD incidence: 3.08%) and 67,947 females (4-year cumulative ASCVD incidence: 1.50%). The evaluation data sets consisted of 66,068 males (4-year cumulative ASCVD incidence: 3.22%) and 66,231 females (4-year cumulative ASCVD incidence: 1.49%). For males, the AUROC of the final model equalled 0.6548. The E/O equalled 0.9466. For females, the AUROC equalled 0.6744. The E/O equalled 0.9838.

Conclusions: The resulting model shows promise for further research. These models may be used for ASCVD screening in the future.

Background: The COVID-19 pandemic has imposed a significant global health burden. Identifying prognostic markers for COVID-19 and its severity could contribute to improved patient outcomes by reducing morbidity and mortality. This systematic review and meta-analysis aimed to evaluate the relationship between ischemic-modified albumin (IMA) and thiol levels, both indicators of oxidative stress, in patients diagnosed with COVID-19.

Method: We conducted a comprehensive search across PubMed, Scopus, Embase, and Web of Science for eligible original studies. The study assessed IMA and thiol levels in COVID-19 patients, examining their association with both disease severity and mortality. A random effect analysis was conducted to estimate the standardized mean difference (SMD) and confidence intervals (CI).

Results: Sixteen studies comprising 2010 COVID-19 patients and 982 controls were included. A diagnosis of COVID-19 was associated with significantly elevated IMA levels (Hedges's g = 1.02, 95% CI: 0.45 to 1.60) and reduced total thiol levels (Hedges's g = -1.08, 95% CI: -2.10 to -0.07). However, native thiol levels did not reveal a significant difference between infected patients and healthy participants. Subgroup analysis showed significantly lower total thiol levels in patients with critical and severe COVID-19, as well as lower native thiol levels specifically in critical COVID-19 patients. IMA levels were significantly higher across the critical, severe, and moderate COVID-19 groups.

Conclusion: Elevated IMA and reduced thiol levels may serve as novel markers for predicting COVID-19 severity and prognosis. Further research is needed to explore therapeutic interventions that target oxidative imbalance in COVID-19 patients.

Background: In diagnostic accuracy studies, when no reference standard test is available, a group of experts, combined in an expert panel, is often used to assess the presence of the target condition using multiple relevant pieces of patient information. Based on the expert panel's judgment, the accuracy of a test or model can be determined. Methodological choices in design and analysis of the expert panel procedure have been shown to vary considerably between studies as well as the quality of reporting. This review maps the current landscape of expert panels used as reference standard in diagnostic accuracy or model studies.

Methods: PubMed was systematically searched for eligible studies published between June 1, 2012, and October 1, 2022. Data extraction was performed by one author and, in cases of doubt, checked by another author. Study characteristics, expert panel characteristics, and expert panel methodology were extracted. Articles were included if the diagnostic accuracy of an index test or diagnostic model was assessed using an expert panel as reference standard and the study was reported in English, Dutch, or German.

Results: After initial identification of 4078 studies, 318 were included for data extraction. Expert panels were used across numerous medical domains, of which oncology was the most common (20%). The number of experts judging the presence of the target condition in each patient was 2 or fewer in 29%, 3 or 4 in 55%, and 5 or more in 16% of the 318 studies. Expert panel types used were an independent panel (i.e., each expert returns a judgement without conferring with other experts in the panel) in 33% of studies, a panel using a consensus method (i.e., each case was discussed by the expert panel) in 27%, a staged (i.e., each expert independently returns a judgement and discordant cases were discussed in a consensus meeting) target condition assessment approach in 11%, and a tiebreaker (i.e., each expert independently returns a judgement and discordant cases were assessed by another expert) in 8%. The exact expert panel decision approach was unclear or not reported in 21% of studies. In 5% of studies, information about remaining uncertainty in experts about the target condition presence or absence was collected for each participant.

Conclusions: There is large heterogeneity in the composition of expert panels and the way that expert panels are used as reference standard in diagnostic research. Key methodological characteristics of expert panels are frequently not reported, making it difficult to replicate or reproduce results, and potentially masking biasing factors. There is a clear need for more guidance on how to perform an expert panel procedure and specific extensions of the STARD and TRIPOD reporting guidelines when using an expert panel.

Objectives: This scoping review assesses machine learning (ML) tools that predicted falls, relying on information in health records without using any sensor data. The aim was to assess the available evidence on innovative techniques to improve fall prevention management.

Methods: Studies were included if they focused on predicting fall risk with machine learning in elderly populations and were written in English. There were 13 different extracted variables, including population characteristics (community dwelling, inpatients, age range, main pathology, ethnicity/race). Furthermore, the number of variables used in the final models, as well as their type, was extracted.

Results: A total of 6331 studies were retrieved, and 19 articles met criteria for data extraction. Metric performances reported by authors were commonly high in terms of accuracy (e.g., greater than 0.70). The most represented features included cardiovascular status and mobility assessments. Common gaps identified included a lack of transparent reporting and insufficient fairness assessments.

Conclusions: This review provides evidence that falls can be predicted using ML without using sensors if the amount of data and its quality is adequate. However, further studies are needed to validate these models in diverse groups and populations.

Background: Large observational healthcare databases are frequently used to develop models to be implemented in real-world clinical practice populations. For example, these databases were used to develop COVID severity models that guided interventions such as who to prioritize vaccinating during the pandemic. However, the clinical setting and observational databases often differ in the types of patients (case mix), and it is a nontrivial process to identify patients with medical conditions (phenotyping) in these databases. In this study, we investigate how sensitive a model's performance is to the choice of development database, population, and outcome phenotype.

Methods: We developed > 450 different logistic regression models for nine prediction tasks across seven databases with a range of suitable population and outcome phenotypes. Performance stability within tasks was calculated by applying each model to data created by permuting the database, population, or outcome phenotype. We investigate performance (AUROC, scaled Brier, and calibration-in-the-large) stability and individual risk estimate stability.

Results: In general, changing the outcome definitions or population phenotype made little impact on the model validation discrimination. However, validation discrimination was unstable when the database changed. Calibration and Brier performance were unstable when the population, outcome definition, or database changed. This may be problematic if a model developed using observational data is implemented in a real-world setting.

Conclusions: These results highlight the importance of validating a model developed using observational data in the clinical setting prior to using it for decision-making. Calibration and Brier score should be evaluated to prevent miscalibrated risk estimates being used to aid clinical decisions.