OFSF-BC: Online feature selection framework for binary classification

Abstract

Feature selection is a very important technique in machine learning and pattern classification. Feature selection studies using batch learning methods are inefficient when handling big data in real world, especially when data arrives sequentially. Online Feature Selection is a new paradigm which is more efficient than batch feature selection methods but it still very challenging in large-scale ultra-high dimensional sparse domains. In this paper, we propose a framework of online feature selection for binary classification exploiting first-order and second-order information. This framework is designed to assume an efficient and scalable online feature selection process. In the experimental studies, we adopt first-order and second-order online learning based online feature selection methods FOOL-OFS and SOOL-OFS. We conduct extensive experiments to evaluate the learning accuracy and time cost of different algorithms on several benchmarks and some real-world datasets.