A gradient based technique for generating sparse representation in function approximation

Abstract

We provide an RKHS based inverse problem formulation for analytically deriving the optimal function approximation when probabilistic information about the underlying regression is available in terms of the associated correlation functions as used by Poggio and Girosi (1998) and Peney and Atick (1996). On the lines of Poggio and Girosi, we show that this solution can be sparsified using principles of SVM and provide an implementation of this sparsification using a novel, conceptually simple and robust gradient based sequential method instead of the conventional quadratic programming routines.