Diagnostic and prognostic research最新文献

Background: The Cox proportional hazards regression model is frequently used to estimate an individual's probability of experiencing an outcome within a specified prediction horizon. A key assumption of this model is that of proportional hazards. An important component of validating a prediction model is assessing its discrimination. Discrimination refers to the ability of predicted risk to separate those who do and do not experience the event. The impact of violation of the proportional hazards assumption on the discrimination of risk estimates obtained from a Cox model has not been examined.

Methods: We used Monte Carlo simulations to assess the impact of the magnitude of the violation of the proportional hazards assumption on the discrimination of a Cox model as assessed using the time-varying area under the curve and on predictive accuracy as assessed using the time-varying index of predictive accuracy.

Results: Compared to settings in which the proportional hazards assumption was satisfied, discrimination and predictive accuracy decreased in settings in which the log-hazard ratio was positively associated with time. Conversely, compared to settings in which the proportional hazards assumption was satisfied, discrimination and predictive accuracy increased in settings in which the log-hazard ratio was negatively associated with time. Compared with the use of a Cox regression model, the use of accelerated failure time parametric survival models, Royston and Parmar's spline-based parametric survival models, and generalized linear models using pseudo-observations did not result in estimates with improved discrimination or predictive accuracy in settings in which the proportional hazards assumption was violated.

Conclusions: Violation of the proportional hazards assumption had an effect on the discrimination of predictions obtained using a Cox regression model.

Background: Reporting of COVID-19 prognostic models frequently falls short of established standards. The TRIPOD checklist and its 2024 AI extension (TRIPOD + AI) provide a comprehensive framework for assessing reporting quality. We therefore evaluated and compared reporting completeness in conventional versus machine-learning models.

Methods: Studies reporting the development, and internal and external validation of prognostic prediction models for COVID-19 using either conventional or machine learning-based algorithms were included. Literature searches were conducted in MEDLINE, Epistemonikos.org, and Scopus (up to July 31, 2024). Studies using conventional statistical methods were evaluated under TRIPOD, while machine learning-based studies were assessed using TRIPOD + AI. Data extraction followed TRIPOD and TRIPOD + AI checklists, measuring adherence per article and per checklist item. The protocol was prospectively registered at the Open Science Framework ( https://osf.io/kg9yw ).

Results: A total of 53 studies describing 71 prognostic models were identified. Overall, adherence to both guidelines was low, with significantly poorer compliance among machine learning-based studies (TRIPOD + AI) compared to conventional model studies (TRIPOD) (28.4% vs. 38.1%, 95% CI of difference: 4.1-15.4). No study fully adhered to abstract reporting requirements, and appropriate titles were included in only a minority of cases (29.0%, 95% CI: 16.1-46.6 for TRIPOD; 13.6%, 95% CI: 4.8-33.3 for TRIPOD + AI). Sample size calculations were not fully reported in any study. Reporting of methods and results sections was poor across both frameworks.

Conclusion: Lower adherence among machine learning studies reflects the relatively recent publication of the TRIPOD + AI guidelines (April 2024), which postdate many of the included studies. Both conventional and machine learning-based prediction models showed insufficient reporting, with major gaps in model description and performance reporting. Greater compliance with reporting guidelines is critical to improving the clarity, reproducibility, and clinical value of prediction model research.

Background: Approximately one million adults in the UK are estimated to have undiagnosed type 2 diabetes mellitus (T2DM), with a further 5.1 million adults with nondiabetic hyperglycaemia (NDH) that does not meet the threshold for a diabetes diagnosis. The Leicester Risk Assessment score (LRA) and Leicester Practice Risk score (LPR) are diagnostic risk prediction models that estimate an individual's risk of undiagnosed T2DM and NDH, developed for use in community and primary care settings respectively. The LRA is also used as a prognostic model; neither model has been updated since development. This study will systematically review all applications of these models as diagnostic and prognostic tools and any published updates to evaluate their performance in different populations. This review has been registered with PROSPERO (CRD420251005841).

Methods: We will implement a citation search strategy to search Scopus, Web of Science and Google Scholar, restricted to full text, English language papers. Eligible papers will validate, update or modify either model. Data will be extracted using a form based on the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist; missing information will be sought from authors or estimated from other available information where possible. Meta-analysis of predictive performance measures will be completed if sufficient data exist. Subgroup and sensitivity analyses will be used to explore between-study heterogeneity and risk-of-bias impact.

Discussion: This review will identify studies that have implemented, modified or validated the LRA and LPR for the risk of undiagnosed T2DM and NDH in different populations. This will allow summary measures, including level of uncertainty, of model performance to be calculated, making this highly relevant to individuals and stakeholders who recommend and implement these models. Review conclusions will also inform the potential update and recalibration of the models. This will ultimately lead to improved outcomes through earlier diagnosis and management.

Background: When developing a clinical prediction model using time-to-event data (i.e. with censoring and different lengths of follow-up), previous research focuses on the sample size needed to minimise overfitting and precisely estimating the overall risk. However, instability of individual-level risk estimates may still be large.

Methods: We propose using a decomposition of Fisher's information matrix to help examine and calculate the sample size required for developing a model that aims for precise and fair risk estimates. We propose a six-step process which can be used either before data collection or when an existing dataset is available. Steps 1 to 5 require researchers to specify the overall risk in the target population at a key time-point of interest: an assumed pragmatic 'core model' in the form of an exponential regression model, the (anticipated) joint distribution of core predictors included in that model and the distribution of censoring times. The 'core model' can be specified directly or based on a specified C-index and relative effects of (standardised) predictors. The joint distribution of predictors may be available directly in an existing dataset, in a pilot study or in a synthetic dataset provided by other researchers.

Results: We derive closed-form solutions that decompose the variance of an individual's estimated event rate into Fisher's unit information matrix, predictor values and total sample size; this allows researchers to calculate and examine uncertainty distributions around individual risk estimates and misclassification probabilities for specified sample sizes. We provide an illustrative example in breast cancer and emphasise the importance of clinical context, including any risk thresholds for decision-making, and examine fairness concerns for pre- and postmenopausal women. Lastly, in two empirical evaluations, we provide reassurance that uncertainty interval widths based on our exponential approach are close to using more flexible parametric models.

Conclusions: Our approach allows users to identify the (target) sample size required to develop a prediction model for time-to-event outcomes, via the pmstabilityss module. It aims to facilitate models with improved trust, reliability and fairness in individual-level predictions.

Background: Prediction of suboptimal growth allows early intervention that can improve outcomes for developing fetus' as well as infants and children. We investigate the risk of bias in statistical or machine learning models to predict the height or weight of a fetus, infant or child under 20 years of age to inform the current standard of research and provide insight into why equations developed over 30 years ago are still recommended for use by national professional bodies.

Methods: We systematically searched MEDLINE and EMBASE for peer reviewed original research studies published in 2022. We included studies if they developed or validated a multivariable model to predict height or weight of an individual using two or more variables, excluding studies assessing imaging or using genetics or metabolomics information. Risk of bias was assessed for all prediction models and analyses using the Prediction model Risk Of Bias ASsessment Tool (PROBAST).

Results: Sixty-four studies were included, in which we assessed the development of 180 models and validation of 61 models. Sample size was only considered in 10% of developed models and 13% of validated models. Despite height and weight being continuous variables, 77% of models developed predicted a dichotomised outcome variable.

Registration: The review was registered on PROSPERO (ID: CRD42023421146), the International prospective register of systematic reviews on 26/4/2023.

Background: High natriuretic peptide levels are associated with a poor outcome in adults with chronic heart failure (CHF). However, the incremented prediction accuracy of multivariable prognostic models after adding B-type natriuretic peptide (BNP) and/or N-terminal proBNP (NT-proBNP) remains unclear.

Methods: We carried out a systematic review narrative analysis of added-value studies of BNP and NT-proBNP in CHF prognostication. Primary clinical studies investigating prognostic model development or validation in adult participants with CHF were included. Any studies of individual factors' association with patient outcomes, treatment efficacy, or those using patients with transplant/ventricular assist devices, ≥ 10% of patients with advanced HF, or significant comorbidities, HF secondary to congenital/reversible conditions, or ≥ 33% of patients with valvular HF were excluded. The databases MEDLINE, Embase, Science Citation Index, and Cochrane Prognosis Methods Group Database were searched from January 1990 to February 2024. Predictive performance was measured in terms of discrimination and calibration, the added value in terms of the c-statistic difference before and after adding BNP and/or NT-proBNP to a base model, and the risk reclassification, namely, net reclassification index (NRI) and integrated discrimination improvement (IDI). Risk of bias assessment used the Prediction model Risk Of Bias ASsessment Tool (PROBAST).

Results: Fourteen added-value studies comprising a total of 50,949 individuals were included. Both BNP and NT-proBNP consistently improved mortality prediction performance, but studies only presented separately before and after c-statistics, without formally testing for statistically significant differences. Meta-analysis was impossible due to missing data on the change in predictive performance and data heterogeneity. All studies reported discrimination. Few reported calibration, NRI, and IDI. All studies except one were deemed to be at high risk of bias, whereas 50% showed high applicability to the review question, with only 14% scoring high for applicability concern, and the rest were unclear.

Conclusions: Improving consistency in researching and reporting the added value of natriuretic peptide testing to predict mortality in chronic heart failure patients could facilitate summarizing and interpreting the results more meaningfully.

Registration: This review is a refinement of the methods and a search update of the review of added-value biomarkers in HF prognosis (PROSPERO registration number: CRD42019086993).