Stochastic Control Liaisons: Richard Sinkhorn Meets Gaspard Monge on a Schrödinger Bridge

In 1931--1932, Erwin Schr\"odinger studied a hot gas Gedankenexperiment (an instance of large deviations of the empirical distribution). Schr\"odinger's problem represents an early example of a fundamental inference method, the so-called maximum entropy method, having roots in Boltzmann's work and being developed in subsequent years by Jaynes, Burg, Dempster, and Csisz\'ar. The problem, known as the Schr\" odinger bridge problem (SBP) with ``uniform"" prior, was more recently recognized as a regularization of the Monge-Kantorovich optimal mass transport (OMT) problem, leading to effective computational schemes for the latter. Specifically, OMT with quadratic cost may be viewed as a zerotemperature limit of the problem posed by Schr\"odinger in the early 1930s. The latter amounts to minimization of Helmholtz's free energy over probability distributions that are constrained to possess two given marginals. The problem features a delicate compromise, mediated by a temperature parameter, between minimizing the internal energy and maximizing the entropy. These concepts are central to a rapidly expanding area of modern science dealing with the so-called Sinkhorn algorithm, which appears as a special case of an algorithm first studied in the more challenging continuous space setting by the French analyst Robert Fortet in 1938--1940 specifically for Schr\"odinger bridges. Due to the constraint on end-point distributions, dynamic programming is not a suitable tool to attack these problems. Instead, Fortet's iterative algorithm and its discrete counterpart, the Sinkhorn iteration, permit computation of the optimal solution by iteratively solving the so-called Schr\" odinger system. Convergence of the iteration is guaranteed by contraction along the steps in suitable metrics, such as Hilbert's projective metric. In both the continuous as well as the discrete time and space settings, stochastic control provides a reformulation of and a context for the dynamic versions of general Schr\" odinger bridge problems and of their zero-temperature limit, the OMT problem. These problems, in turn, naturally lead to steering problems for flows of one-time marginals which represent a new paradigm for controlling uncertainty. The zero-temperature problem in the continuous-time and space setting turns out to be the celebrated Benamou--Brenier characterization of theMcCann displacement interpolation flow in OMT. The formalism and techniques behind these control problems on flows of probability distributions have attracted significant attention in recent years as they lead to a variety of new applications in spacecraft guidance, control of robot or biological swarms, sensing, active cooling, and network routing as well as in computer and data science. This multifaceted and versatile framework, intertwining SBP and OMT, provides the substrate for the historical and technical overview \ast Received by the editors May 22, 2020; accepted for publication (in revised form) October 29, 2020; published electronically May 6, 2021. https://doi.org/10.1137/20M1339982 Funding: This work was partially supported by the NSF under grants 1807664, 1839441, 1901599, and 1942523, by the AFOSR under grant FA9550-17-1-0435, and by University of Padova Research Project CPDA 140897. \dagger School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA 30332 USA (yongchen@gatech.edu). \ddagger Department of Mechanical and Aerospace Engineering, University of California, Irvine, CA 92697 USA (tryphon@uci.edu). \S Dipartimento di Matematica ``Tullio Levi-Civita,"" Universit\` a di Padova, 35121 Padova, Italy (pavon@math.unipd.it). 249 D ow nl oa de d 11 /0 9/ 21 to 1 47 .1 62 .2 13 .1 11 R ed is tr ib ut io n su bj ec t t o SI A M li ce ns e or c op yr ig ht ; s ee h ttp s: //e pu bs .s ia m .o rg /p ag e/ te rm s