A Non-linear Function Approximation from Small Samples Based on Nadaraya-Watson Kernel Regression

Abstract

Solving function approximation problem is to appropriately find the relationship between dependent variable and independent variable(s). Function approximation algorithms normally require sufficient amount of samples to approximate a function. However, insufficient samples may result in unsatisfactory prediction to any function approximation algorithms. It is due to the failure of the function approximation algorithms to fill the information gap between the available and very limited samples. In this study, a function approximation algorithm which is based on Nadaraya-Watson Kernel Regression (NWKR) is proposed for approximating a non-linear function with small samples. Gaussian function is chosen as a kernel function for this study. The results show that the NWKR is effective in the case where the target function is non-linear and the given training sample is small. The performance of the NWKR is compared with other existing function approximation algorithms, such as artificial neural network.