An Intelligent Bearing Fault Diagnosis based on Modified Probabilistic Knowledge Distillation

Abstract

Knowledge distillation (KD) is one of popular algorithms for compressing deep neural networks because it generates a compact but still powerful deep neural network for the cases of complicated situations and limited computation resources. In this study, an intelligent fault diagnosis method is developed based on the probabilistic knowledge distillation (PKD) and deep convolutional neural network (CNN) to determine the health states of bearings. First, the one-dimensional vibration signal is reshaped as a two-dimensional matrix to input the teacher or student network. Then, a deeper neural network and small network are trained as the teacher and student networks, respectively. The probability distribution (PD) is learned by minimizing the difference of the joint density probability estimation between the teacher and student networks, that is, the lightweight network learns to integrate the PD of the deeper neural network in the high-dimensional feature space and realizes the knowledge transfer from training samples to test samples. The results of experimental bearings indicate that the proposed diagnosis method has higher diagnosis accuracy than the other two popular knowledge distillation methods and its student network only has about one 700-th parameter of the teacher network. Therefore, the proposed method achieves a good balance between the classification accuracy and network compression, and demonstrates potential application to intelligent fault diagnosis of bearings under varying working conditions.