DMCMFuse: A dual-phase model via multi-dimensional cross-scanning state space model for multi-modality medical image fusion

Abstract

Multi-modality medical image fusion is crucial for improving diagnostic accuracy by combining complementary information from different imaging modalities. However, a key challenge is effectively balancing the abundant modality-specific features (e.g., soft tissue details in MRI and bone structure in CT) with the relatively fewer modality-shared features, often leading to suboptimal fusion outcomes. To address this, we propose DMCMFuse, a dual-phase model for multi-modality medical image fusion that leverages a multi-dimensional cross-scanning state-space model. The model first decomposes multi-modality images into distinct frequency components to maintain spatial and channel coherence. In the fusion phase, we apply Mamba for the first time in medical image fusion and develop a fusion method that integrates spatial scanning, spatial interaction, and channel scanning. This multi-dimensional cross-scanning approach effectively combines features from each modality, ensuring the retention of both global and local information. Comprehensive experimental results demonstrate that DMCMFuse surpasses the state-of-the-art methods, generating fused images of superior quality with enhanced structure consistency and richer feature representation, making it highly effective for medical image analysis and diagnosis.