Nonlinear Models

Abstract

Kernel functions • Whenever a learning algorithm (such as SVMs) can be written in terms of dot-products, it can be generalized to kernels. • A kernel is any function K : R × R 7→ R which corresponds to a dot product for some feature mapping φ: K(x1, x2) = φ(x1) · φ(x2) for some φ. • Conversely, by choosing feature mapping φ, we implicitly choose a kernel function • Recall that φ(x1) · φ(x2) ∝ cos∠(φ(x1), φ(x2)) where ∠ denotes the angle between the vectors, so a kernel function can be thought of as a notion of similarity.