A reconstruction and convolution operations enabled variant vision transformer with gastroscopic images for automatic locating of polyps in Internet of Medical Things

Abstract

Gastric polyps are an important cause of gastric disease. At present, the computer-aided diagnosis technology based on convolutional neural network (CNN) can automatically locate the position of polyps from the gastroscopic image, which improves the efficiency of doctors. However, due to the small polyp area in the gastroscopic image, the CNN-based method has a high rate of missed detection. To solve the above problems, in this work, we propose a reconstruction and convolution operations enabled variant vision transformer (RCVViT) to automatically locate the position of polyps in gastroscopic images. The RCVViT model uses the vision transformer model as a benchmark model. By using the self-attention mechanism, contextual information can be considered, and irregularly shaped polyps or polyps with small areas can be effectively detected. The feedforward neural network (FNN) and CNN are used to flatten each image patch data into a one-dimensional vector. The advantage of combining the FNN and CNN is that the local feature information and structural information of the polyp area are considered. In addition, we use an Internet of Medical Things (IoMT) platform to collect and analyze patients’ medical data to make timely diagnosis of patients’ diseases. Finally, our multiple experimental results on real gastroscopic datasets demonstrate the superiority of the RCVViT model.