Self-reduction multi-head attention module for defect recognition of power equipment in substation

Abstract

Safety maintenance of power equipment is of great importance in power grids, in which image-processing-based defect recognition is supposed to classify abnormal conditions during daily inspection. However, owing to the blurred features of defect images, the current defect recognition algorithm has poor fine-grained recognition ability. Visual attention can achieve fine-grained recognition with its ability to model long-range dependencies while introducing extra computational complexity, especially for multi-head attention in vision transformer structures. Under these circumstances, this paper proposes a self-reduction multi-head attention module that can reduce computational complexity and be easily combined with a Convolutional Neural Network (CNN). In this manner, local and global features can be calculated simultaneously in our proposed structure, aiming to improve the defect recognition performance. Specifically, the proposed self-reduction multi-head attention can reduce redundant parameters, thereby solving the problem of limited computational resources. Experimental results were obtained based on the defect dataset collected from the substation. The results demonstrated the efficiency and superiority of the proposed method over other advanced algorithms.