Research and experimentation on an automatic teat cupattachment system for dairy cows based on visual perception

Abstract

The milking process is the most labour-intensive activity in dairy farming. Teat cup attachment is the primary challenge in realizing automatic milking. The existing AMS (automatic milking systems) primarily utilize laser/vision technology for their sensing systems, leading to slow teat sensing process and struggles with attaching teat cups to angled teats. Moreover, current depth sensing technologies fail to achieve real-time, high-accuracy 6D pose estimation (3D position and orientation) for deformable biological targets like cow teats. To address these issues, we first expand the cow udder image dataset and propose an arbitrary-oriented teat detection and target endpoint positioning method based on YOLOv8-obb. The AP (average precision) of the method is 97.82%, the average orientation error is 1.83°, the average positioning error of target endpoint is 2.04 pixels, and the detection time is 11 ms per image. Secondly, based on the morphological characteristics of the teats, combined with the teats rotated bounding boxes and binocular vision, we develop an efficient method for spatial positions and orientations calculation of teats’ target endpoints, with an average time consumption of 24.62 ms. Finally, a lightweight automatic teat cup attachment system is built to perform experiments. Among the 25 groups experiments, the average spatial positioning errors of the X, Y, and Z axes are 2.54 mm, 2.08 mm, and 3.90 mm, respectively, while the average spatial orientation errors are 1.16°, 1.28°, and 1.73°, respectively. The average time consumption for the entire process is 116.71 ms. The results prove the feasibility and accuracy of the proposed methods.