α-HMM and optimal decoding higher-order structures on sequential data

Decoding higher-order structure on sequential data is an indispensable task in data science. It requires models to have the capability to characterize interdependencies among hidden events that have generated observable data. However, to be able to decode arbitrary structures, such models would need to cope with the intractability arising from computing context-sensitive relations, likely compromising the quality of answers. To address this important issue, the current paper introduces the arbitrary order hidden Markov model ($\alpha$-HMM), an extension of the HMM that permits decoding of the optimal higher-order structure with an assurance of computational tractability. The advantage of the $\alpha$-HMM is made possible by an identified principle on how random variables influence each other in a stochastic process. In particular, it is shown that decoding the optimal structure with an $\alpha$-HMM can be computed in $O\left(n^3\right)$-time for any stochastic process of $n$ random variables. As an application, it is demonstrated the decoding algorithm inspires a simple yet effective algorithm for RNA secondary structure prediction.