Yousef Yassin;Anthony Fuller;Keyvan Ranjbar;Kenny Bala;Leonid Nichman;James R. Green
{"title":"Soft Contrastive Representation Learning for Cloud-Particle Images Captured In-Flight by the New HVPS-4 Airborne Probe","authors":"Yousef Yassin;Anthony Fuller;Keyvan Ranjbar;Kenny Bala;Leonid Nichman;James R. Green","doi":"10.1109/LGRS.2024.3506483","DOIUrl":null,"url":null,"abstract":"Cloud properties underpin accurate climate modeling and are often derived from the individual particles comprising a cloud. Studying these cloud particles is challenging due to their intricate shapes, called “habits,” and manual classification via probe-generated images is time-consuming and subjective. We propose a novel method for habit representation learning that uses minimal labeled data by leveraging self-supervised learning (SSL) with Vision Transformers (ViTs) on a newly acquired dataset of 124000 images captured by the novel high-volume precipitation spectrometer ver. 4 (HVPS-4) probe. Our approach significantly outperforms ImageNet pretraining by 48% on a 293-sample annotated dataset. Notably, we present the first SSL scheme for learning habit representations, leveraging data collected in flight from the probe. Our results demonstrate that self-supervised pretraining significantly improves habit classification even when using single-channel HVPS-4 data. We achieve further gains using sequential views and a soft contrastive objective tailored for sequential, in-flight measurements. Our work paves the way for applying SSL to multiview and multiscale data from advanced cloud-particle imaging probes, enabling comprehensive characterization of the flight environment. We publicly release data, code, and models associated with this study.","PeriodicalId":91017,"journal":{"name":"IEEE geoscience and remote sensing letters : a publication of the IEEE Geoscience and Remote Sensing Society","volume":"22 ","pages":"1-5"},"PeriodicalIF":0.0000,"publicationDate":"2024-12-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE geoscience and remote sensing letters : a publication of the IEEE Geoscience and Remote Sensing Society","FirstCategoryId":"1085","ListUrlMain":"https://ieeexplore.ieee.org/document/10783433/","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0
Abstract
Cloud properties underpin accurate climate modeling and are often derived from the individual particles comprising a cloud. Studying these cloud particles is challenging due to their intricate shapes, called “habits,” and manual classification via probe-generated images is time-consuming and subjective. We propose a novel method for habit representation learning that uses minimal labeled data by leveraging self-supervised learning (SSL) with Vision Transformers (ViTs) on a newly acquired dataset of 124000 images captured by the novel high-volume precipitation spectrometer ver. 4 (HVPS-4) probe. Our approach significantly outperforms ImageNet pretraining by 48% on a 293-sample annotated dataset. Notably, we present the first SSL scheme for learning habit representations, leveraging data collected in flight from the probe. Our results demonstrate that self-supervised pretraining significantly improves habit classification even when using single-channel HVPS-4 data. We achieve further gains using sequential views and a soft contrastive objective tailored for sequential, in-flight measurements. Our work paves the way for applying SSL to multiview and multiscale data from advanced cloud-particle imaging probes, enabling comprehensive characterization of the flight environment. We publicly release data, code, and models associated with this study.