Deploying Reinforcement Learning for Efficient Runtime Decision-Making in Autonomous Systems

Abstract

Autonomous systems need to effectively react to runtime changes in the environment and the system itself. The capability to analyze both the environment and the system is theoretically feasible through the model-based approach. How-ever, the limitations like model size are serious obstacles to autonomous decision-making process. The incremental approximation is a technique to partition the model to tackle this issue. A partition must be updated/re-verified at a reasonable cost when some change occurs. The paper suggests a policy-based analysis technique to find the optimal partitioning criteria through a set of available policies with respect to our proposed metrics, namely Balancing and Variation. Using the incremental approximation scheme, the metrics evaluate each component quantitatively according to their size and frequency. The proposed method is augmented with a reinforcement learning approach so that the autonomous system can learn how to find the best partitioning policy at runtime. Since the most time-consuming parts of this approach are done at the design time, the proposed method is efficient and meets the runtime resource requirements of the autonomous systems. We analyze the correctness of the proposed system via a few theoretical investigations and experimental results applied to a case study on energy-harvesting self-adaptive systems. The outcome illustrates the correctness of the proposed system in terms of efficiency and accuracy.