Estimating Multivariate Conditional Models via Entropic Methods

Abstract

We introduce a new practical numerical method to estimate conditional distributions, p(y|x), where y is the value of a continuous random variable supported on R^{N_y} and x is in R^{N_x}, via the Maximum Entropy Principal. We are not aware of other practical robust methods to tackle this problem. We also introduce a new practical numerical method to estimate p(y|x), when the (multivariate) data associated with y are fat-tailed, by maximizing U-entropy, a generalization of entropy. The maximization procedures are convex programming problems and are therefore amenable to robust numerical solution. The models that result are provably robust in a certain decision-theoretic sense, and the U-entropy problem solutions are optimal with respect to Tsallis, Renyi and power f-entropy. In our approach, we do not make use of models for x or joint models of x and y. We benchmark our models against various alternative models on financial data and show that our approach produces models that outperform the benchmarks with respect to out-of-sample likelihood.