Residual-like multi-kernel block and dynamic attention for deep neural networks

Abstract

Traditional network architectures struggled with a uniform approach to receptive field (RF) sizes, leading to suboptimal performance across scales. Although recent advances have addressed the problem by utilizing different RF sizes, a balance between accuracy and complexity remains elusive. In addition, the existing group attention mechanism that simply uses the squeeze-and-excitation method neglects the spatial position information in the feature selection and fusion process. Therefore, this research introduces a lightweight and efficient architecture named Split-Dense Adaptive Network (SDANet) to cope with these limitations. In the proposed network, a residual-like multi-kernel method is implemented to enable better feature extraction under diverse RF sizes. Next, a new grouped attention module processes features dynamically and highlight the location information. Also, the constructed feature augmentation structure strengthens the model's representation. Furthermore, a new channel split and merge strategy is utilized for computation reduction. Compared with state-of-the-art methods, our model achieved better generalization ability, less computational complexity, and superior precision based on various public datasets. The introduced network shows a promising general applicability in the field of computer vision, and further inspires research on supervised deep learning.