Mirjana Voelsen, Franz Rottensteiner, Christian Heipke
{"title":"Transformer models for Land Cover Classification with Satellite Image Time Series","authors":"Mirjana Voelsen, Franz Rottensteiner, Christian Heipke","doi":"10.1007/s41064-024-00299-7","DOIUrl":null,"url":null,"abstract":"<p>In this paper we address the task of pixel-wise land cover (LC) classification using satellite image time series (SITS). For that purpose, we use a supervised deep learning model and focus on combining spatial and temporal features. Our method is based on the Swin Transformer and captures global temporal features by using self-attention and local spatial features by convolutions. We extend the architecture to receive multi-temporal input to generate one output label map for every input image. In our experiments we focus on the application of pixel-wise LC classification from Sentinel‑2 SITS over the whole area of Lower Saxony (Germany). The experiments with our new model show that by using convolutions for spatial feature extraction or a temporal weighting module in the skip connections the performance improves and is more stable. The combined usage of both adaptations results in the overall best performance although this improvement is only minimal. Compared to a fully convolutional neural network without any self-attention layers our model improves the results by 2.1% in the mean F1-Score on a corrected test dataset. Additionally, we investigate different types of temporal position encoding, which do not have a significant impact on the performance.</p>","PeriodicalId":56035,"journal":{"name":"PFG-Journal of Photogrammetry Remote Sensing and Geoinformation Science","volume":null,"pages":null},"PeriodicalIF":2.1000,"publicationDate":"2024-08-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"PFG-Journal of Photogrammetry Remote Sensing and Geoinformation Science","FirstCategoryId":"89","ListUrlMain":"https://doi.org/10.1007/s41064-024-00299-7","RegionNum":4,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"IMAGING SCIENCE & PHOTOGRAPHIC TECHNOLOGY","Score":null,"Total":0}
引用次数: 0
Abstract
In this paper we address the task of pixel-wise land cover (LC) classification using satellite image time series (SITS). For that purpose, we use a supervised deep learning model and focus on combining spatial and temporal features. Our method is based on the Swin Transformer and captures global temporal features by using self-attention and local spatial features by convolutions. We extend the architecture to receive multi-temporal input to generate one output label map for every input image. In our experiments we focus on the application of pixel-wise LC classification from Sentinel‑2 SITS over the whole area of Lower Saxony (Germany). The experiments with our new model show that by using convolutions for spatial feature extraction or a temporal weighting module in the skip connections the performance improves and is more stable. The combined usage of both adaptations results in the overall best performance although this improvement is only minimal. Compared to a fully convolutional neural network without any self-attention layers our model improves the results by 2.1% in the mean F1-Score on a corrected test dataset. Additionally, we investigate different types of temporal position encoding, which do not have a significant impact on the performance.
期刊介绍:
PFG is an international scholarly journal covering the progress and application of photogrammetric methods, remote sensing technology and the interconnected field of geoinformation science. It places special editorial emphasis on the communication of new methodologies in data acquisition and new approaches to optimized processing and interpretation of all types of data which were acquired by photogrammetric methods, remote sensing, image processing and the computer-aided interpretation of such data in general. The journal hence addresses both researchers and students of these disciplines at academic institutions and universities as well as the downstream users in both the private sector and public administration.
Founded in 1926 under the former name Bildmessung und Luftbildwesen, PFG is worldwide the oldest journal on photogrammetry. It is the official journal of the German Society for Photogrammetry, Remote Sensing and Geoinformation (DGPF).