Sparse Minimal Learning Machines Via L_1/2 Norm Regularization

Abstract

The Minimal Learning Machine (MLM) is a supervised method in which learning consists of fitting a multiresponse linear regression model between distances computed from the input and output spaces. A critical issue related to the training process in MLMs is the selection of prototypes, also called reference points (RPs), from which distances are taken. In its original formulation, the MLM selects the RPs randomly from the data. In this paper we empirically show that the original random selection may lead to a poor generalization capability. In addition, we propose a novel pruning method for selecting RPs based on L_1/2 norm regularization. Our results show that the proposed method is able to outperform the original MLM and its variants.