H-SGANet: Hybrid sparse graph attention network for deformable medical image registration

Abstract

The integration of Convolutional Neural Networks (ConvNets) and Transformers has become a strong candidate for image registration, combining the strengths of both models and utilizing a large parameter space. However, this hybrid model, which treats brain MRI volumes as grid or sequence structures, struggles to accurately represent anatomical connectivity, diverse brain regions, and critical connections within the brain’s architecture. There are also concerns about the computational expense and GPU memory usage of this model. To address these issues, we propose a lightweight hybrid sparse graph attention network (H-SGANet). The network includes Sparse Graph Attention (SGA), a core mechanism based on Vision Graph Neural Networks (ViG) with predefined anatomical connections. The SGA module expands the model’s receptive field and integrates seamlessly into the network. To further enhance the hybrid network, Separable Self-Attention (SSA) is used as an advanced token mixer, combined with depth-wise convolution to form SSAFormer. This strategic integration is designed to more effectively extract long-range dependencies. As a hybrid ConvNet-ViG-Transformer model, H-SGANet offers three key benefits for volumetric medical image registration. It optimizes fixed and moving images simultaneously through a hybrid feature fusion layer and an end-to-end learning framework. Compared to VoxelMorph, a model with a similar parameter count, H-SGANet demonstrates significant performance enhancements of 3.5% and 1.5% in Dice score on the OASIS dataset and LPBA40 dataset, respectively. The code is publicly available at https://github.com/2250432015/H-SGANet/.