Generalized reinforcement learning control algorithm for fully automated insulin delivery system

Abstract

A fully automated insulin delivery (Fully-AID) system is expected to provide the ultimate safety, comfort, and a sense of freedom for people living with diabetes (PwD). Previous studies have shown the potential of a deep reinforcement learning (DRL) model for fully-AID control algorithm in simulation environment. However, the practical implementation is still challenging due to the domain gaps between simulation and real world scenario. In this manuscript, we proposed a novel generalized control algorithm, called xgDRL, to realize a DRL-driven fully-AID system. The generalization of the proposed algorithm is achieved by our two main contributions that are introducing a novel concept of fully-AID context called total daily insulin (TDI) into the input of DRL model, and a novel training environment named type 1 diabetes (T1D) simulation-to-reality (T1Dsim2real). Here, we conduct a stepwise validation experiment to validate the performance of the proposed control algorithm, which comprises in silico, retrospective-counterfactual studies, and preclinical studies using a T1D pig model. Results from the preclinical validation demonstrate the effectiveness of the proposed algorithm, with average time in target range of 70–180 mg/dL of 72.8 %, 73.8 %, 74.5 %, and 86.9 % across breakfast, lunch, dinner, and overnight fasting time, respectively. Thus, this study represents the first preclinical validation of a DRL-driven fully-AID algorithm in PwD, confirming the efficacy of the xgDRL model in preclinical settings.