{"title":"Robust semi-supervised learning with reciprocal weighted mixing distribution alignment","authors":"","doi":"10.1016/j.engappai.2024.109185","DOIUrl":null,"url":null,"abstract":"<div><p>Recent semi-supervised learning(SSL) methods have achieved great success owing to the impressive performances brought by the combination of pseudo-labeling and consistency regularization. These methods often use pre-defined constant thresholds or dynamical thresholds to select unlabeled samples that contribute to training. However, many correct/incorrect pseudo-labels may be ignored/selected. Especially in distribution mismatched scenario, threshold-adjusted strategy is often complex and ineffective. To alleviate this issue, we develop a simple yet powerful framework whose idea is to abandon this strategy and utilize distribution alignment to adjust the predictions generated from a biased model softly. Specifically, first, we create two classifiers to predict pseudo-label(i.e., the sample belongs to a specific category) and complementary pseudo-label(i.e., the sample does not belong to a specific category), respectively. Second, by maintaining the distributions of pseudo-labels, complementary pseudo-labels and their reverse versions from past iterations, we enforce a reciprocal weighted mixing according to the predicted category weights. Third, a reciprocal distribution alignment is applied to the mixed distributions to adjust the predicted distributions. Finally, we propose Implication Alignment Loss , which keeps consistency between the predictions of the same implications but from different versions. We empirically demonstrate the effectiveness of our proposed method in comparison with state-of-the-art benchmarks. Especially, our method achieves a 1.18% error rate reduction over the latest state-of-the-art method MutexMatch on CIFAR-10 with 2 labels per class and exhibits robustness in the scenario of mismatched distribution.</p></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":null,"pages":null},"PeriodicalIF":7.5000,"publicationDate":"2024-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Engineering Applications of Artificial Intelligence","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0952197624013435","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}
引用次数: 0
Abstract
Recent semi-supervised learning(SSL) methods have achieved great success owing to the impressive performances brought by the combination of pseudo-labeling and consistency regularization. These methods often use pre-defined constant thresholds or dynamical thresholds to select unlabeled samples that contribute to training. However, many correct/incorrect pseudo-labels may be ignored/selected. Especially in distribution mismatched scenario, threshold-adjusted strategy is often complex and ineffective. To alleviate this issue, we develop a simple yet powerful framework whose idea is to abandon this strategy and utilize distribution alignment to adjust the predictions generated from a biased model softly. Specifically, first, we create two classifiers to predict pseudo-label(i.e., the sample belongs to a specific category) and complementary pseudo-label(i.e., the sample does not belong to a specific category), respectively. Second, by maintaining the distributions of pseudo-labels, complementary pseudo-labels and their reverse versions from past iterations, we enforce a reciprocal weighted mixing according to the predicted category weights. Third, a reciprocal distribution alignment is applied to the mixed distributions to adjust the predicted distributions. Finally, we propose Implication Alignment Loss , which keeps consistency between the predictions of the same implications but from different versions. We empirically demonstrate the effectiveness of our proposed method in comparison with state-of-the-art benchmarks. Especially, our method achieves a 1.18% error rate reduction over the latest state-of-the-art method MutexMatch on CIFAR-10 with 2 labels per class and exhibits robustness in the scenario of mismatched distribution.
期刊介绍:
Artificial Intelligence (AI) is pivotal in driving the fourth industrial revolution, witnessing remarkable advancements across various machine learning methodologies. AI techniques have become indispensable tools for practicing engineers, enabling them to tackle previously insurmountable challenges. Engineering Applications of Artificial Intelligence serves as a global platform for the swift dissemination of research elucidating the practical application of AI methods across all engineering disciplines. Submitted papers are expected to present novel aspects of AI utilized in real-world engineering applications, validated using publicly available datasets to ensure the replicability of research outcomes. Join us in exploring the transformative potential of AI in engineering.