Dynamic mutual training semi-supervised semantic segmentation algorithm with adaptive capability (AD-DMT) for choy sum stem segmentation and 3D positioning of cutting points

Abstract

Choy sum (Brassica rapa var. parachinensis) is a commonly grown leafy vegetable, primarily harvested for its stem. Accurate stem segmentation is crucial for accurate harvesting, yet the visual similarity between choy sum stems and leaves poses challenges for traditional supervised learning methods, making data labeling costly and affecting segmentation accuracy. This study introduces AD-DMT, an enhanced Dynamic Mutual Training (DMT) algorithm for semi-supervised segmentation, which improves on the original framework by incorporating: 1) Introduction of data augmentation strategies such as CutMix, brightness, and contrast adjustments to alleviate model generalization difficulties caused by data homogeneity; 2) The design of adaptive loss weights re-scaled factor (γ₁ and γ₂) dynamically adjusts the balance between mutual learning and entropy minimization based on training epochs; 3) A dynamic temperature coefficient is incorporated to enhance divergent learning in training by modulating Softmax output. For validation, images of field-grown choy sum were captured to evaluate AD-DMT’s performance under different labeled data ratios (1/2, 1/4, 1/8, 1/20). The results demonstrate efficient segmentation across all conditions, with mIoU values exceeding 84.0 %. Notably, even with minimal labeled data (1/20 ratio), AD-DMT achieved a 4.04 % improvement in mIoU over the baseline. Building on these segmentation results, we further determined the optimal cutting points of choy sum stems by using skeleton extraction and corner detection algorithms, calculating the three-dimensional coordinates of these points with depth images, achieving an average vertical offset error (VOE) within 6.29 mm.