A limited annotated sample fault diagnosis algorithm based on nonlinear coupling self-attention mechanism

Abstract

Deep learning-based intelligent diagnostic algorithms are regarded as a technology with significant industrial application prospects. However, acquiring sufficient annotated samples for training remains challenging in practice application, rendering the model susceptible to overfitting. To tackle the issue, a semi-supervised learning algorithm based on nonlinear coupling self-attention mechanism (NCSAM) is proposed for fault diagnosis with scarce annotated samples. Specifically, the method combines a pre-training model using multi-scale convolutional autoencoder (MSCAE) with a novel pre-training approach based on signal transformation to extract generic features from an ample number of unlabeled samples. On this basis, a nonlinear coupling self-attention mechanism is designed to adaptively explore both linear and nonlinear information in input data, achieving the integration of multi-scale features. Finally, the fault classification is completed using a linear classifier. The effectiveness of the proposed method has been validated on two public datasets. The results demonstrate that even with extremely limited annotated samples, the method achieves an accuracy of 97.83%, a 4.74% improvement over the baseline. Additionally, extensive comparative experiments with both semi-supervised and supervised algorithms have been designed to confirm the advantages of the proposed approach. In contrast, the diagnostic performance of the proposed method surpasses that of other methods.