Evaluating the quality of word representation models for unstructured clinical Text based ICU mortality prediction

In modern hospitals, the role of Clinical Decision Support Systems (CDSS) in assisting care providers is well-established. Most conventional CDSS systems are built on the availability of patient data in the form of structured Electronic Health Records. However, a significant percentage of patient data is still stored in the form of unstructured clinical text notes, especially in developing countries. These contain abundant patient-specific information, which has so far remained largely under-utilized in powering CDSS applications. In this paper, we attempt to build one such CDSS system for patient mortality prediction, using unstructured clinical records. Effectiveness of such prediction models largely depends on optimally capturing latent concept features, thus, word representation quality is of utmost importance. We experiment with three popular word embedding models - Word2Vec, FastText and GloVe for generating word embeddings of unstructured nursing notes of patients from a standard, open dataset, MIMIC-III. These word representations are used as features to train machine learning classifiers to build ICU mortality prediction models, a critical CDSS in ICUs of hospitals. Experimental validation showed that a model built on Word2Vec Skipgram based Random Forest classifier was the most optimal word embedding based mortality prediction model, that outperformed traditional severity scores like SAPS-II, SOFA, APS-III and OASIS, by a significant margin of 43-52%.