Attention Based Echo State Network: A Novel Approach for Fault Prognosis

Abstract

Recurrent neural networks (RNNs) are widely studied in recent years, since RNNs are capable of modeling the significant nonlinear dynamical systems. Echo state network (ESN) is a novel type of RNN with an interconnected reservoir to model temporal dynamics of complex sequential information. In this paper, a novel ESN structure is developed and employed to conduct fault prognosis. Fault prognosis is vital in predictive maintenance, which is a prevalent research area that mainly concentrates on predicting the remaining useful life of a machine and reducing the machine's downtime. Attention model is integrated to a typical ESN and thus different importance levels of different input elements can be adaptively treated. To further enhance the generalization of the prediction model, genetic algorithm is applied to adaptively optimize the parameters of the attention-based ESN. The proposed prognostic approach is verified on the NASA's turbofan benchmark dataset. Experimental results show that the attention-based ESN can not only achieve superior prediction accuracy but also obtain substantial improvement on stability.