Using POMDP-based state estimation to enhance agent system survivability

Abstract

A survivable agent system depends on the incorporation of many recovery features. However, the optimal use of these recovery features requires the ability to assess the actual state of the agent system accurately at a given time. This paper describes an approach for the estimation of the state of an agent system using partially-observable Markov decision processes (POMDPs). POMDPs are dependent on a model of the agent system - components, environment, sensors, and the actuators that can correct problems. Based on this model, we define a state estimation for each component (asset) in the agent system. We model a survivable agent system as a POMDP that takes into account both environmental threats and observations from sensors. We describe the process of updating the state estimation as time passes, as sensor inputs are received, and as actuators affect changes. This state estimation process has been deployed within the Ultralog application and successfully tested using Ultralog's survivability tests on a full-scale (1000+) agent system.