Joint multitask feature learning and classifier design

Abstract

The problem of classification arises in many realworld applications. Often classification of more than two classes is broken down into a group of binary classification problems using the one-versus-rest or pairwise approaches. For each binary classification problem, feature selection and classifier design are usually conducted separately. In this paper, we propose a new multitask learning approach in which feature selection and classifier design for all the binary classification tasks are carried out simultaneously. We consider probabilistic nonlinear kernel classifiers for binary classification. For each binary classifier, we give weights to the features within the kernels. We assume that the matrix consisting of all the feature weights for all the tasks has a sparse component and a low rank component. The sparse component determines the features that are relevant to each classifier, and the low rank component determines the common feature subspace that is relevant to all the classifiers. Experimental results on synthetic data demonstrate that the proposed approach achieves higher classification accuracy compared to the conventional classifiers. The proposed method accurately determines the relevant features that are important to each binary classifier.