NNBSVR: Neural Network-Based Semantic Vector Representations of ICD-10 codes

Abstract

Automatically predicting ICD-10 codes from clinical notes using machine learning models can reduce the burden of manual coding. However, existing methods often overlook the semantic relationships between ICD-10 codes, resulting in inaccurate evaluations when clinically similar codes are considered completely different. Traditional evaluation metrics, which rely on equality-based matching, fail to capture the clinical relevance of predicted codes. This study introduces NNBSVR (Neural Network-Based Semantic Vector Representations), a novel approach for generating semantic-based vector representations of ICD-10 codes. Unlike traditional approaches that rely on exact code matching, NNBSVR incorporates contextual and hierarchical information to enhance both prediction accuracy and evaluation methods. We validate NNBSVR using intrinsic and extrinsic evaluation methods. Intrinsic evaluation assesses the vectors’ ability to reconstruct the ICD-10 hierarchy and identify clinically meaningful clusters. Extrinsic evaluation compares our relevancy-based approach, which includes customized evaluation metrics, to traditional equality-based metrics on an ICD-10 code prediction task using a 9.57 million clinical notes corpus. NNBSVR demonstrates significant improvements over equality-based metrics, achieving a 9.81% gain in micro-F1 score on the training set and a 12.73% gain on the test set. A manual review by medical experts on a sample of 10,000 predictions confirms an accuracy of 92.58%, further validating our approach. This study makes two significant contributions: first, the development of semantic-based vector representations that encapsulate ICD-10 code relationships and context; second, the customization of evaluation metrics to incorporate clinical relevance. By addressing the limitations of traditional equality-based evaluation metrics, NNBSVR enhances the automated assignment of ICD-10 codes in clinical settings, demonstrating superior performance over existing methods.