3D ShiftBTS: Shift Operation for 3D Multimodal Brain Tumor Segmentation

Abstract

Recently, ShiftViT and its variants have attracted much attention for their simple and efficient shift operation, showing excellent efficacy in several tasks on natural images, surpassing Swin Transformer. However, considering the complexity of 3D multimodal images, which have higher dimensions than natural images, and the relative stability of the human tissue structure in medical images, the applicability of shift operation on 3D multimodal medical data has yet to be determined. This paper demonstrates that ShiftViT has enormous potential in 3D multimodal medical image analysis. Using 3D medical image segmentation as a representative downstream task, we investigate how shift operation can improve model performance. First, applying ShiftViT to 3D multimodal medical images not only effectively extracts global information but also significantly enhances the model’s performance. Second, as a plug-and-play strategy, the shift operation can be integrated with other modules without adding additional computational burden, proving its flexibility in the overall system. Finally, we further investigate the generalizability of the shift operation by introducing a cascaded attention module, which provides useful insights to improve the generalizability of 3D medical image segmentation models. Through this study, we extend the application scope of ShiftViT and bring new exploration directions to the field of 3D multimodal medical image analysis. Our research results prove the feasibility of applying ShiftViT in 3D multimodal medical images and provide an effective and scalable model, which is expected to further promote the development of medical image processing technology.