A new approach for binary feature selection and combining classifiers

Abstract

This paper explores feature selection and combining classifiers when binary features are used. The concept of Non-Reducible Descriptors (NRDs) for binary features is introduced. NRDs are descriptors of patterns that do not contain any redundant information. The underlying mathematical model for the present approach is based on learning Boolean formulas which are used to represent NRDs as conjunctions. Starting with a description of a computational procedure for the construction of all NRDs for a pattern, a two-step solution method is presented for the feature selection problem. The method computes weights of features during the construction of NRDs in the first step. The second step in the method then updates these weights based on repeated occurrences of features in the constructed NRDs. The paper then proceeds to present a new procedure for combining classifiers based on the votes computed for different classifiers. This procedure uses three different approaches for obtaining the single combined classifier, using majority, averaging, and randomized vote.