ParetoSSL: Pareto Semi-Supervised Learning With Bias-Aware Gradient Preferences for Fruit Yield Estimation

Fruit counting is a fundamental task for fruit yield estimation. Though semi-supervised counting methods have received increased attention in recent years, due to the high data utilization of unlabeled data, they suffer from two limitations. Firstly, difficult weight selection is a limitation, as these methods rely on manually selected fixed weights for both the supervised learning loss and the consistency learning loss, resulting in limited performance. Secondly, biased pseudo-labeling is another limitation, as they may predict biased pseudo-labels that result in small consistency learning losses, leading to training being dominated by supervised learning with large losses. To tackle these two limitations, in this paper, we propose a novel method named ParetoSSL to automatically derive weights of losses from the perspective of multi-task learning. Specifically, ParetoSSL formulates a multi-objective optimization problem for weight derivation by maximizing the similarity between weighted gradients of losses and a customized gradient preference vector, in which, the vector can guide weight derivation. Moreover, to relieve the effect of pseudo-label biases on consistency learning, we propose a bias-aware gradient preference vector. This vector considers gradient biases brought by the pseudo-label biases, which will down-weight the supervised learning loss while high-weighting the consistency learning loss. Meanwhile, to improve the robustness of ParetoSSL, an inequality equation regarding the norm of the gradients of the consistency learning loss is designed to control the range of gradient biases. Extensive experiments are conducted on the Clustered-Fruit dataset and Fruit-2019 dataset to evaluate the effectiveness of ParetoSSL on semi-supervised counting. Experimental results show that our ParetoSSL is superior to state-of-the-art methods. Note to Practitioners—This work is motivated by the emerging need for semi-supervised counting methods in fruit yield estimation. The difficulty of selecting loss weights for training semi-supervised counting algorithms is exacerbated by the pseudo-label bias issue that pseudo-label biases mislead the weight derivation while maximizing the similarity between the weight gradient and gradient preference vectors. The proposed bias-aware gradient preference vectors help users derive loss weights automatically and save the time of choosing loss weights for model fine-tuning. The proposed ParetoSSL is generic as it can be employed as a fruit yield estimation component of crop management support systems, while at the same time being applied to counting frameworks of other objects.