Generalization of a Deep Learning Model for Continuous Glucose Monitoring–Based Hypoglycemia Prediction: Algorithm Development and Validation Study

Abstract

Background: Predicting hypoglycemia while maintaining low false alarm rate is a challenge for wide adoption of continuous glucose monitoring (CGM) in diabetes management. One small study suggested the long short-term memory (LSTM) network deep learning model had better performance of hypoglycemia prediction than traditional machine learning algorithms in European patients with type 1 diabetes. However, given that many well-recognized deep learning models perform poorly outside the training consideration, whether the LSTM model could be generalized to different populations or patients with other diabetes subtypes are unknown. Objective: The aim of this study is to validate the LSTM hypoglycemia prediction models in more diverse populations and a wide spectrum of patients with different types of diabetes. Methods: We assembled two large datasets of patients with both type 1 diabetes and type 2 diabetes. The primary dataset containing 192 patients from Chinese were used to develop the LSTM, support vector machine (SVM) and random forest (RF) models for hypoglycemia prediction at the prediction horizon of 30 minutes. Hypoglycemia was defined as the mild (54mg/dl <= glucose < 70mg/dl) and severe (< 54mg/dl) hypoglycemic level separately. The validation dataset of 427 patients from European-Americans was used to validate the models and examine their generalizations. The predictive performance of the models was evaluated by sensitivity, specificity and area under the operating curve (AUC). Results: For the difficulty to predict mild hypoglycemia events, the LSTM model always achieved AUC greater than 97% in the primary dataset, with less than 3% AUC reduction in the validation dataset, indicating the model was robust and generalizable across populations. AUC higher than 93% was also achieved when LSTM was applied to both type 1 diabetes and type 2 diabetes in the validation dataset, further strengthening the generalizability of the model. Under different satisfactory levels of sensitivity for mild and severe hypoglycemia prediction, the LSTM model achieved higher specificity than the SVM and RF models, thereby reducing false alarms. Conclusions: Our results demonstrated that the LSTM model was robust for hypoglycemia prediction and generalizable across populations or diabetes subtypes. Given its extra advantage on false alarm reduction, the LSTM model was a strong candidate to be widely implemented by future CGM devices for hypoglycemia prediction.