Non-Gaussian component analysis using Density Gradient Covariance matrix

Abstract

High dimensional data are often modeled by signal plus noise where the signal belongs to a low dimensional manifold contaminated with high dimensional noise. Estimating the signal subspace when the noise is Gaussian and the signal is non-Gaussian is the main focus of this paper. We assume that the Gaussian noise variance can be high, so standard denoising approaches like Principal Component Analysis fail. The approach also differs from standard Independent Component Analysis in that no independent signal factors are assumed. This model is called non-Gaussian subspace/component analysis (NGCA). The previous approaches proposed for this subspace analysis use the fourth cumulant matrix or the Hessian of the logarithm of characteristic functions, which both have some practical and theoretical issues. We propose to use sample Density Gradient Covariances, which are similar to the Fisher information matrix for estimating the non-Gaussian subspace. Here, we use nonparametric kernel density estimator to estimate the gradients of density functions. Moreover, we extend the notion of non-Gaussian subspace analysis to a supervised version where the label or response information is present. For the supervised non-Gaussian subspace analysis, we propose to use conditional density gradient covariances which are computed by conditioning on the discretized response variable. A non-asymptotic analysis of density gradient covariance is also provided which relates the error of estimating the population DGC matrix using sample DGC to the number of dimensions and the number of samples.