On Using Spectral Graph Theory to Infer the Structure of Multiscale Markov Processes

Multiscale Markov processes are used to model and control stochastic dynamics across different scales in many applications areas such as electrical engineering, finance, and material science. A commonly used mathematical representation that captures multiscale stochastic dynamics is that of singularly perturbed Markov processes. Dimensionality reductions techniques for this class of stochastic optimal control problems have been studied for many years. However, it is typically assumed that the structure of perturbed process and its dynamics are known. In this paper, we show how to infer the structure of a singularly perturbed Markov process from data. We propose a measure of similarity for the different states of the Markov process and then use techniques from spectral graph theory to show that the perturbed structure can be obtained by looking at the spectrum of a graph defined on the proposed similarity matrix.