Field test and evaluation of an innovative vision-guided robotic cotton harvester

Conventional cotton harvesters are efficient but heavy causing soil compaction. They normally perform one harvesting pass, but since cotton bolls mature over two months, the early opened bolls must wait for later ones to be harvested, exposing their fiber to weather and degrading fiber quality. A swarm of small, lightweight robotic cotton harvesters can address these issues. This study presents field tests and evaluations of an innovative robotic cotton harvester prototype. A stereovision camera in conjunction with the YOLOv4-tiny algorithm was used for cotton boll detection and localization. The picking system included a 3-DOF (degree of freedom) linear robotic arm, a three-finger end-effector, and an agile control algorithm. The performance rates of detection, localization, and picking systems were 78.1 %, 70.0 %, and 83.1 %, respectively, with an average cycle time of 8.8 s. Collecting cotton bolls orientation data proved that they tend to stay their faces upward causing difficulty in picking the rear part of the bolls in 40.5 % of cases. Controlling the illumination, developing more robust detection and localization systems, increasing the arm’s DOF, enhancing the end-effector’s operating speed, and its adaptability to different boll orientations can improve the robot’s performance in terms of the picking ratio of the seed cotton and speed. The dataset, including field images, annotations of cotton bolls, and the best training weights, is publicly available at: https://github.com/hussein-pasha/Robotic-Cotton-Harvester. A video demonstration of the harvester being tested in the field is available at: https://youtu.be/IztKk3E7zSc.