Predicting abnormal C-reactive protein level for improving utilization by deep neural network model

Abstract

Background

C-reactive protein (CRP) is an inflammatory biomarker frequently used in clinical practice. However, insufficient evidence-based ordering inevitably results in its overuse or underuse. This study aims to predict its normal and abnormal levels using the deep neural network (DNN) models, helping clinicians order this item more appropriately and intelligently.

Methods

We considered complete blood count (CBC) parameters as feature vectors and 10 mg/L as a cutoff value for CRP. Several models, including linear support vector classification, logistic regression, decision trees, random forests, and DNN, were developed based on a dataset of 53834 medical records to predict binary output. We externally validated DNN models on independent 20723 samples through discrimination, calibration curve, and decision curve analysis.

Results

DNN models has the best area under the receiver operating characteristic curves (AUC). Learning curves revealed that models’ AUC, balanced accuracy, and F1 score do not significantly and continuously improve following increasing data volume. In internal validation, the AUC, balanced accuracy, and the F1 score of 10 models were 0.818 (0.95 CI: 0.812-0.824), 0.741 (0.95 CI: 0.736-0.747), and 0.649 (0.95 CI: 0.643-0.656), respectively. These metrics were 0.817 (0.95 CI: 0.816-0.817), 0.741 (0.95 CI: 0.740-0.742), and 0.641 (0.95 CI: 0.640-0.642), respectively, in external validation. AUC and balanced accuracy shown no significant difference (P-values were 0.106 and 0.339). CRP10-C2 model has the lowest Brier score of 0.154, AUC of 0.818, and calibration curve formula of y=1.001x-0.010, which was identified as a target model to deploy in the app.

Conclusions

DNN models obtained moderate performance, surpassing baseline indices in distinguishing binary CRP levels. They are good generalizations and well-calibrated. The CRP-C2 model can enhance CRP utilization by informing the orders appropriately and can contribute to inflammatory diagnostics in primary health care where CBC is available, but the CRP test is inaccessible.