Discrete-Time Partially Observable Stochastic Optimal Control Problems of McKean–Vlasov Type and Branching Particle System Approximations

Abstract

This article investigates a broad category of McKean–Vlasov type discrete-time partially observable stochastic optimal control problems. The first goal is to prove the dynamic programming principle (DPP) by means of the measurable selection argument, which provides a methodology for finding both the value function as well as the optimal control. Here, we employ the Nisio semigroup technology, which is an intrinsic characterization of the DPP. Then, we derive the recursive formula for the filter process, which enables us to clearly track the time evolution of the posterior distribution. Next, we approximate the posterior distribution utilizing branching particle systems (branching particle filters) and illustrate its convergence. Making use of branching particle system approximations and Bellman equations, we devise a numerical algorithm for addressing the optimal control problem. A numerical experiment serves as the final part of this article.