Robust Feature Selection by Removing Noise Entropy Within Mutual Information for Limited-Sample Industrial Data

Abstract

Feature selection is challenging in high-dimensional and small-sample data, particularly in industrial informatics with diverse noise sources. The information entropy of feature noise is included in mutual information of a label and noise-corrupted features, which can be removed to increase classification accuracy. In this article, we propose a robust feature selection method by eliminating feature noise in the relevance measure. Feature noise is modeled as a zero-mean censored normal distribution, so its entropy is determined by solving the variance equation based on the maximum entropy principle. Then, a noisy channel for feature transmission is proposed to extract class-relevant noise component. Furthermore, a noise-free mutual information metric is developed by removing noise entropy within mutual information. Eventually, a novel criterion is proposed by maximizing relevance based on noise-free mutual information while minimizing redundancy. Experimental results confirm the effectiveness of our approach on datasets from various industrial sectors.