RESEARCH OF MULTI-TASK LEARNING BASED ON EXTREME LEARNING MACHINE

Abstract

Recently, extreme learning machines (ELMs) have been a promising tool in solving a wide range of regression and classification applications. However, when modeling multiple related tasks in which only limited training data per task are available and the dimension is low, ELMs are generally hard to get impressive performance due to little help from the informative domain knowledge across tasks. To solve this problem, this paper extends ELM to the scenario of multi-task learning (MTL). First, based on the assumption that model parameters of related tasks are close to each other, a new regularization-based MTL algorithm for ELM is proposed to learn related tasks jointly via simple matrix inversion. For improving the learning performance, the algorithm proposed above is further formulated as a mixed integer programming in order to identify the grouping structure in which parameters are closer than others, and finally an alternating minimization method is presented to solve this optimization. Experiments conducted on a toy problem as well as real-life data set demonstrate the effectiveness of the proposed MTL algorithm compared to the classical ELM and the standard MTL algorithm.