Search

Progress in continuous glucose monitoring (CGM) technologies alongside wearable electronics is driving the evolution of automated insulin delivery systems (AIDs) toward fully automated closed-loop configurations. The ultimate goal is to provide safe, individualized, and optimized management of blood glucose concentration (BGC) for patients living with diabetes. Although model predictive control offers an adaptable architecture for constructing AIDs control logic, accurate and efficient BGC regulation demands models capable of accounting for inter- and intra-patient variability as well as perturbation uncertainties. Innovations in artificial intelligence offer novel pathways for creating data-driven, fully closed-loop AIDs. Among these approaches, deep reinforcement learning (DRL) has gained substantial interest owing to its inherent robustness against disturbances. Consequently, this article presents a comprehensive literature review focused on DRL-based BGC control methodologies within AIDs. Initially, this paper systematically evaluates the advantages of deploying DRL frameworks in AIDs. Following this, it delivers an extensive analysis of diverse DRL methods and enhancements designed to surmount obstacles linked to their integration into AIDs, addressing factors such as restricted sample availability, personalization, and safety protocols. Furthermore, an application-centric assessment of DRL-based AIDs control algorithms is provided, highlighting critical hurdles encountered during practical implementation. Lastly, the paper addresses potential remedies for relevant BGC control challenges, outlines prospects for real-world application, and identifies future research trajectories.

Although Reinforcement Learning (RL) exhibits significant potential for refining intricate control and decision-making systems, Deep Reinforcement Learning (DRL) suffers from a distinct lack of explainability. This limitation restricts its deployment within highly regulated domains, including manufacturing, finance, and healthcare. The opacity of DRL decision-making introduces structural challenges that impede operational efficiency and resource management—an issue that intensifies alongside ongoing algorithmic advancements. While current trends favor transitioning from traditional Experience Replay configurations to A3C architectures, the latter imposes substantially greater resource demands. Furthermore, despite continuous efforts to refine selection mechanisms within Experience Replay, repository capacities generally remain elevated. In this study, we evaluate a Deep Convolutional Q-learning agent across 20 Atari environments while deliberately downsizing the Experience Replay capacity from 1×106 to 5×102. Our findings indicate that lowering the capacity from 1×104 to 5×103 does not meaningfully degrade performance rewards, thereby establishing a viable strategy for resource-efficient DRL frameworks. To clarify agent choices and correlate them with inherent game mechanics, we introduce a novel methodology: visualizing the Experience Replay mechanism utilizing the Deep SHAP Explainer. This framework enhances domain comprehension and delivers transparent, interpretable accounts, though capacity constraints must be managed carefully to mitigate overfitting. Ultimately, this research validates the practicality of minimizing Experience Replay storage and supports the integration of transparent explanations via the Deep SHAP Explainer to advance resource conservation.