Schrödinger's Control and Estimation Paradigm With Spatio-Temporal Distributions on Graphs

Abstract

The problem of reconciling a prior probability law on paths with data was introduced by Schrödinger in 1931 and 1932. It represents an early formulation of a maximum likelihood problem. This specific formulation can also be seen as the control problem to modify the law of a diffusion process so as to match specifications on marginal distributions at given times. Thereby, in recent years, this so-called Schrödinger's bridge problem has been at the center of the uncertainty control development. However, an understudied facet of this program has been to address uncertainty in space (state) and time, modeling the effect of tasks being completed contingent on meeting a certain condition at some random time instead of imposing specifications at fixed times. The present work is a study to extend Schrödinger's paradigm on such an issue, and herein, it is tackled in the context of random walks on directed graphs. Specifically, we study the case where one marginal is the initial probability distribution on a Markov chain, while others are marginals of stopping (first-arrival) times at absorbing states, signifying completion of tasks. We show when the prior law on paths is Markov, a Markov policy is once again optimal to satisfy those marginal constraints with respect to a likelihood cost following Schrödinger's dictum. Based on this, we present the mathematical formulation involving a Sinkhorn-type iteration to construct the optimal probability law on paths matching the spatio-temporal marginals.