Accuracy of Asthma Computable Phenotypes to Identify Pediatric Asthma at an Academic Institution.

Abstract

Objectives: Asthma is a heterogenous condition with significant diagnostic complexity, including variations in symptoms and temporal criteria. The disease can be difficult for clinicians to diagnose accurately. Properly identifying asthma patients from the electronic health record is consequently challenging as current algorithms (computable phenotypes) rely on diagnostic codes (e.g., International Classification of Disease, ICD) in addition to other criteria (e.g., inhaler medications)-but presume an accurate diagnosis. As such, there is no universally accepted or rigorously tested computable phenotype for asthma.

Methods: We compared two established asthma computable phenotypes: the Chicago Area Patient-Outcomes Research Network (CAPriCORN) and Phenotype KnowledgeBase (PheKB). We established a large-scale, consensus gold standard (n = 1,365) from the University of California, Los Angeles Health System's clinical data warehouse for patients 5 to 17 years old. Results were manually reviewed and predictive performance (positive predictive value [PPV], sensitivity/specificity, F1-score) determined. We then examined the classification errors to gain insight for future algorithm optimizations.

Results: As applied to our final cohort of 1,365 expert-defined gold standard patients, the CAPriCORN algorithms performed with a balanced PPV = 95.8% (95% CI: 94.4-97.2%), sensitivity = 85.7% (95% CI: 83.9-87.5%), and harmonized F1 = 90.4% (95% CI: 89.2-91.7%). The PheKB algorithm was performed with a balanced PPV = 83.1% (95% CI: 80.5-85.7%), sensitivity = 69.4% (95% CI: 66.3-72.5%), and F1 = 75.4% (95% CI: 73.1-77.8%). Four categories of errors were identified related to method limitations, disease definition, human error, and design implementation.

Conclusion: The performance of the CAPriCORN and PheKB algorithms was lower than previously reported as applied to pediatric data (PPV = 97.7 and 96%, respectively). There is room to improve the performance of current methods, including targeted use of natural language processing and clinical feature engineering.