A multi-step interaction network for multi-class classification based on OCT and OCTA images

Abstract

OCT and OCTA images are important basis for diagnosing multiple ophthalmic diseases. However, it is a challenge to fuse these two modalities with high redundancy and simultaneous projection of 3D data for multi-class classification. This paper proposes a novel Multi-step Interaction Network (MINet) for projection and feature fusion as a unified framework, where OCT and OCTA images deeply interact for a seven-class classification task of ophthalmic diseases. Firstly, we design a Multi-modal Interaction Projection Module to iterate projection and shallow information interaction for effective feature selection. Secondly, a Feature Redundancy Removal Module compares the feature difference information between the two modalities to eliminate redundancy. Thirdly, the Feature Interaction Fusion Module utilizes the differential modal information from the backbone CNN to perform respective modal attention and achieve interactive fusion. Finally, a classifier module generates multi-class classification results. Experimental results show that our method achieved Accuracy of 0.8690, Precision of 0.6921, Recall of 0.7250, and F1 score of 0.7081 on the OCTA-500 dataset. Comparative experiments with other state-of-the-art methods for OCT image classification, along with ablation experiments, demonstrate the superior performance of MINet.