ResNet50 in remote sensing and agriculture: evaluating image captioning performance for high spectral data

Abstract

Remote sensing image captioning is crucial as it enables the automatic interpretation and description of complex images captured from satellite or aerial sensors, facilitating the efficient analysis and understanding of vast amounts of geospatial data. This capability is essential for various applications, including environmental monitoring, disaster management, urban planning, and agricultural assessment, where accurate and timely information is vital for decision-making and response. This paper aims to evaluate deep learning models for image captioning in the context of remote sensing data and specifically compares Vision Transformer (ViT) and ResNet50 architectures. Utilizing the BLEU score to evaluate the quality of generated captions, the research explores the models' capabilities across varying sample sizes: The amount of samples included 25, 50, 75, and 100 samples. As it is shown in the tables above, the Vision Transformer outperforms the ResNet50 model in most cases, with the highest BLEU score of 0. 5507 at 50 samples, which indicates the superiority in learning global dependencies for image understanding and text generation. Nonetheless, the performance of ViT decreases slightly when the number of samples is greater than 50, which might be attributed to overfitting or scalability. On the other hand, ResNet50 shows a gradual increase in BLEU score with the increase in sample size and attains the maximum BLEU score of 0. 4783 at 100 samples, meaning that it is most effective with large data sets where it can fully take advantage of the learning algorithm. This work also discusses the advantages and disadvantages of the two models and makes suggestions on when it is suitable to use which model for image captioning tasks in remote sensing, thus helps to advance the discussion on model selection and improvement for image captioning tasks.