Multiscale attention network for retinal vein occlusion classification with multicolor image

Recently, automatic diagnostic approaches widely use various retinal images to classify ocular diseases. And retinal vein occlusion (RVO) is the second most common retinal vascular disease after diabetic retinopathy. In clinical practice, ophthalmologists are usually accustomed to resorting to images of one modality. But single‐modality images often ignore other modality‐specific information. To solve this problem, this paper uses a novel retinal imaging, the multicolor (MC) imaging, for RVO recognition. It can obtain four multiple modal images with different wavelengths to provide much richer information about retinal features. Since the MC images contain local and global pathologies at multiple scales, a multiscale attention structure is proposed to recognize RVO. In simple terms, this structure uses Resnet as the backbone network for feature extraction, with simultaneous input of images in four modalities. Then, the feature maps at different scales are fed into an attention module to fuse the global and local features, which combines two attention mechanisms, the channel attention mechanism and the spatial attention mechanism. The extensive experimental results demonstrate that our proposed framework achieves quite promising classification performance on the fundus diseases and normal images.