Investigating Vision Transformer Models for Low-Resolution Medical Image Recognition

Abstract

Vision Transformers use self-attention techniques to learn long-range spatial relations to focus on the relevant parts of an image. They have achieved state-of-the-art results in many computer vision tasks. Recently, some methods have to leverage Vision Transformer-based models to tackle tasks in medical imaging. However, Vision Transformer emphasizes the low-resolution features due to the repetitive downsamplings, which result in a loss or lack of detailed localization information, making it highly unfit for low-level image recognition. In this paper, we investigate the performance of Vision Transformer on low-level medical images and contrast it with convolutional neural networks. The experimental results show that Convolutional Neural Network outperforms the Vision Transformer-based models on all four datasets.