Elaboration Models with Symmetric Information Divergence.

Abstract

Various statistical methodologies embed a probability distribution in a more flexible family of distributions. The latter is called elaboration model, which is constructed by choice or a formal procedure and evaluated by asymmetric measures such as the likelihood ratio and Kullback-Leibler information. The use of asymmetric measures can be problematic for this purpose. This paper introduces two formal procedures, referred to as link functions, that embed any baseline distribution with a continuous density on the real line into model elaborations. Conditions are given for the link functions to render symmetric Kullback-Leibler divergence, Rényi divergence, and phi-divergence family. The first link function elaborates quantiles of the baseline probability distribution. This approach produces continuous counterparts of the binary probability models. Examples include the Cauchy, probit, logit, Laplace, and Student-t links. The second link function elaborates the baseline survival function. Examples include the proportional odds and change point links. The logistic distribution is characterized as the one that satisfies the conditions for both links. An application demonstrates advantages of symmetric divergence measures for assessing the efficacy of covariates.