Transformer models for Land Cover Classification with Satellite Image Time Series

IF 2.1 4区 地球科学 Q3 IMAGING SCIENCE & PHOTOGRAPHIC TECHNOLOGY PFG-Journal of Photogrammetry Remote Sensing and Geoinformation Science Pub Date : 2024-08-06 DOI:10.1007/s41064-024-00299-7
Mirjana Voelsen, Franz Rottensteiner, Christian Heipke
{"title":"Transformer models for Land Cover Classification with Satellite Image Time Series","authors":"Mirjana Voelsen, Franz Rottensteiner, Christian Heipke","doi":"10.1007/s41064-024-00299-7","DOIUrl":null,"url":null,"abstract":"<p>In this paper we address the task of pixel-wise land cover (LC) classification using satellite image time series (SITS). For that purpose, we use a supervised deep learning model and focus on combining spatial and temporal features. Our method is based on the Swin Transformer and captures global temporal features by using self-attention and local spatial features by convolutions. We extend the architecture to receive multi-temporal input to generate one output label map for every input image. In our experiments we focus on the application of pixel-wise LC classification from Sentinel‑2 SITS over the whole area of Lower Saxony (Germany). The experiments with our new model show that by using convolutions for spatial feature extraction or a temporal weighting module in the skip connections the performance improves and is more stable. The combined usage of both adaptations results in the overall best performance although this improvement is only minimal. Compared to a fully convolutional neural network without any self-attention layers our model improves the results by 2.1% in the mean F1-Score on a corrected test dataset. Additionally, we investigate different types of temporal position encoding, which do not have a significant impact on the performance.</p>","PeriodicalId":56035,"journal":{"name":"PFG-Journal of Photogrammetry Remote Sensing and Geoinformation Science","volume":"23 1","pages":""},"PeriodicalIF":2.1000,"publicationDate":"2024-08-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"PFG-Journal of Photogrammetry Remote Sensing and Geoinformation Science","FirstCategoryId":"89","ListUrlMain":"https://doi.org/10.1007/s41064-024-00299-7","RegionNum":4,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"IMAGING SCIENCE & PHOTOGRAPHIC TECHNOLOGY","Score":null,"Total":0}
引用次数: 0

Abstract

In this paper we address the task of pixel-wise land cover (LC) classification using satellite image time series (SITS). For that purpose, we use a supervised deep learning model and focus on combining spatial and temporal features. Our method is based on the Swin Transformer and captures global temporal features by using self-attention and local spatial features by convolutions. We extend the architecture to receive multi-temporal input to generate one output label map for every input image. In our experiments we focus on the application of pixel-wise LC classification from Sentinel‑2 SITS over the whole area of Lower Saxony (Germany). The experiments with our new model show that by using convolutions for spatial feature extraction or a temporal weighting module in the skip connections the performance improves and is more stable. The combined usage of both adaptations results in the overall best performance although this improvement is only minimal. Compared to a fully convolutional neural network without any self-attention layers our model improves the results by 2.1% in the mean F1-Score on a corrected test dataset. Additionally, we investigate different types of temporal position encoding, which do not have a significant impact on the performance.

Abstract Image

查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
利用卫星图像时间序列进行土地覆盖分类的变换器模型
在本文中,我们利用卫星图像时间序列(SITS)解决了像素级土地覆盖(LC)分类任务。为此,我们使用了一个有监督的深度学习模型,并侧重于结合空间和时间特征。我们的方法以 Swin 变换器为基础,通过自我关注捕捉全局时间特征,并通过卷积捕捉局部空间特征。我们对架构进行了扩展,以接收多时态输入,为每张输入图像生成一个输出标签图。在实验中,我们重点应用了下萨克森州(德国)整个地区哨兵-2 SITS 的像素级 LC 分类。使用我们的新模型进行的实验表明,通过使用卷积进行空间特征提取或在跳转连接中使用时间加权模块,可以提高性能并使其更加稳定。综合使用这两种适配方法可获得最佳的整体性能,尽管这种改进微乎其微。与没有任何自我注意层的完全卷积神经网络相比,我们的模型在校正测试数据集上的平均 F1 分数提高了 2.1%。此外,我们还研究了不同类型的时间位置编码,但这些编码对性能并无显著影响。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
CiteScore
8.20
自引率
2.40%
发文量
38
期刊介绍: PFG is an international scholarly journal covering the progress and application of photogrammetric methods, remote sensing technology and the interconnected field of geoinformation science. It places special editorial emphasis on the communication of new methodologies in data acquisition and new approaches to optimized processing and interpretation of all types of data which were acquired by photogrammetric methods, remote sensing, image processing and the computer-aided interpretation of such data in general. The journal hence addresses both researchers and students of these disciplines at academic institutions and universities as well as the downstream users in both the private sector and public administration. Founded in 1926 under the former name Bildmessung und Luftbildwesen, PFG is worldwide the oldest journal on photogrammetry. It is the official journal of the German Society for Photogrammetry, Remote Sensing and Geoinformation (DGPF).
期刊最新文献
Self-Supervised 3D Semantic Occupancy Prediction from Multi-View 2D Surround Images Characterization of transient movements within the Joshimath hillslope complex: Results from multi-sensor InSAR observations Monocular Pose and Shape Reconstruction of Vehicles in UAV imagery using a Multi-task CNN Assessing the Impact of Data-resolution On Ocean Frontal Characteristics Challenges and Opportunities of Sentinel-1 InSAR for Transport Infrastructure Monitoring
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1