Learning Convex Concepts from Gaussian Distributions with PCA

Abstract

We present a new algorithm for learning a convex set in $n$-dimensional space given labeled examples drawn from any Gaussian distribution. The complexity of the algorithm is bounded by a fixed polynomial in $n$ times a function of $k$ and $\eps$ where $k$ is the dimension of the {\em normal subspace} (the span of normal vectors to supporting hyper planes of the convex set) and the output is a hypothesis that correctly classifies at least $1-\eps$ of the unknown Gaussian distribution. For the important case when the convex set is the intersection of $k$ half spaces, the complexity is \[ \poly(n,k,1/\eps) + n \cdot \min \, k^{O(\log k/\eps^4)}, (k/\eps)^{O(k)}, \] improving substantially on the state of the art \cite{Vem04,KOS08} for Gaussian distributions. The key step of the algorithm is a Singular Value Decomposition after applying a normalization. The proof is based on a monotonicity property of Gaussian space under convex restrictions.