Context-Aware Feature Selection using Denoising Auto-Encoder for Fault Detection in Cloud Environments

Abstract

Machine learning is expected to play an instrumental role in automating the detection of faults in next-generation cloud networks. The existing machine-learning-based fault detection methods suffer from the following drawbacks: (i) ignoring the issue of missing feature values, (ii) ignoring the impact of each feature on output prediction over other features (measurement of feature importance), and (iii) lack of calculation of the proper number of features for fault detection. To address the above challenges, in this paper, we propose a context-aware feature selection method to improve the performance of fault detection methods in the cloud environment, aiming at maximizing the $F_{1}$-score. Our proposed solution comprises Denoising Auto-Encoder (DAE) stacked with a Discriminative Model (DM). The DAE is applied to handle the missing feature values and encoding features while the DM is responsible for making predictions of system status based on the encoded features. Then, the sensitivity analysis of output prediction with respect to each input feature value is used to measure the feature importance. We compare our work with existing solutions from the literature. Our results reveal that the proposed solution can improve the $F_{1}$-score up to 47 % and 76 % in the scenario where all feature values are known and in the scenario where only 25 % of feature values are known, respectively.