A Transformer-based self-supervised learning model for fault diagnosis of air-conditioning systems with limited labeled data

Abstract

Despite the great successes of supervised learning-based fault diagnosis techniques for heating, ventilation and air-conditioning (HVAC) systems, their applications are severely limited due to insufficient labeled data accompanied with massive unlabeled data. To address this drawback, a Transformer-based self-supervised representation learning model (TSSRL) is proposed in this study for HVAC fault diagnosis with limited labeled data. Specifically, a customized Transformer model is developed as the feature encoder by embedding a context-attention module on the self-attention module, which enables TSSRL to mine the contextual representations among input data. In addition, a joint data augmentation strategy is designed to improve the diversity of inputs, promoting the pretext tasks to learn more extensive representations from unlabeled data. Meanwhile, two cooperative pretext tasks, namely contrastive similarity matching and data reconstruction, are formulated to extract discriminative representations from unlabeled data. The diagnosis-beneficial representations learned from unlabeled data are used for downstream classification modeling tasks with limited labeled data. Experiments on two benchmark HVAC fault datasets demonstrate the superiority of the proposed TSSRL model over other state-of-the-art HVAC fault diagnosis methods.