FeatureX: An explainable feature selection for deep learning

Abstract

Feature selection is critical for the performance of deep learning models by reducing the dimensionality of feature sets to understand the features’ importance. Existing techniques focus on the statistical characteristics of different features, which makes them hard to understand due to complicated mathematical reasoning. Furthermore, feature selection can be impacted by model preferences, resulting in a lack of explainability. To this end, this paper proposes an effective method called FeatureX to obtain the optimal feature subset and enhance the explainability of the feature selection process through quantitative evaluation. Firstly, FeatureX proposes importance analysis to quantify the contribution of each feature to the deep learning model by leveraging feature perturbation. Secondly, to mitigate the multicollinearity, FeatureX employs statistical analysis to calculate the correlation coefficients of these features and removes redundant features based on the magnitude of the correlation coefficients. Finally, with the feature contribution and correlation coefficients, FeatureX screens these features automatically to identify the most relevant and high-contribution features. Based on existing research and prior knowledge of the data, FeatureX presets the values of relevant parameters and demonstrates their effectiveness through parameter sensitivity analysis. FeatureX is evaluated on 17 public datasets with 5 fundamental deep learning models. Experimental results show that FeatureX can reduce the number of features by an average of 47.83% and the accuracy of 63.33% deep learning models are improved. Furthermore, when comparing against the existing feature selection techniques, FeatureX improves the F-measure by an average of 1.61%, demonstrating its effectiveness.