Optimal control of finite state Markov processes under counting observations

Abstract

The authors deal with the class of noisy observations of a controlled finite-state Markov process which modulates the rate of point processes. The control problems for a finite-state Markov process under partial observations are reformulated as ones for piecewise deterministic processes. In a weak sense, the value function is shown to be a viscosity solution of the corresponding Hamilton-Jacobi-Bellman equations.<>