Online learning in L2 space with multiple Gaussian kernels

We present a novel online learning paradigm for nonlinear function estimation based on iterative orthogonal projections in an L2 space reflecting the stochastic property of input signals. An online algorithm is built upon the fact that any finite dimensional subspace has a reproducing kernel, which is given in terms of the Gram matrix of its basis. The basis used in the present study involves multiple Gaussian kernels. The sequence generated by the algorithm is expected to approach towards the best approximation, in the L2-norm sense, of the nonlinear function to be estimated. This is in sharp contrast to the conventional kernel adaptive filtering paradigm because the best approximation in the reproducing kernel Hilbert space generally differs from the minimum mean squared error estimator over the subspace (Yukawa and Müller 2016). Numerical examples show the efficacy of the proposed approach.