A Local-Global Attention Fusion Framework with Tensor Decomposition for Medical Diagnosis

Abstract

Dear Editor, In this letter, a novel hierarchical fusion framework is proposed to address the imperfect data property in complex medical image analysis (MIA) scenes. In particular, by combining the strengths of convolutional neural networks (CNNs) and transformers, the enhanced feature extraction, spatial modeling, and sequential context learning are realized to provide comprehensive insights on the complex data patterns. Integration of information in different level is enabled via a multi-attention fusion mechanism, and the tensor decomposition methods are adopted so that compact and distinctive representation of the underlying and high-dimensional medical image features can be accomplished [1]. It is shown from the evaluation results that the proposed framework is competitive and superior as compared with some other advanced algorithms, which effectively handles the imperfect property of inter-class similarity and intra-class differences in diseases, and meanwhile, the model complexity is reduced within an acceptable level, which benefits the deployment in clinic practice.