Optimizing pick-place operations: Leveraging k-means for visual object localization and decision-making in collaborative robots

Abstract

This article presents an approach to object localization algorithms for pick-place operations in collaborative robots by utilizing conventional color segmentation in computer vision and k-means clustering. Adding the k-means clustering algorithm complements the color segmentation by distinguishing and grouping the sections of similar pixels; hence, object localization is more accurate. The order of pick-place operations of each cluster acquired from the proposed algorithm is prioritized based on  norm. Integrating the proposed framework provides a well-structured depiction of the localized objects, which is fundamental for successful pick-place operations. The TCP/IP communication framework via socket communication is established to facilitate data transmission between the robot and the host computer. The objective is to ensure that the robot's end effector performs as directed by the host computer by obtaining information on the pick-and-place operation, including the localized coordinates, dimensions, the order of operations, and the pose of the objects of interest to the robot. In this experiment, a cobot arm is employed to autonomously pick and place objects with different shapes and colors in a workspace filled with diverse objects, requiring the robot to choose the closest objects to operate based on the data from the host computer. Our results demonstrate the effectiveness of this integration, showcasing the enhanced adaptability and efficiency of pick-place operations in collaborative robots. This study indicates 98% accuracy in pick-and-place operations with an average latency of 0.52 ± 0.1 s, indicating an improvement compared to the traditional algorithm without k-means clustering, which achieves an accuracy of 88%. Additional studies reveal that when incorporating pose estimation into the pick-place operations, the proposed algorithm's accuracy is 94%. The demonstration highlights the potential of leveraging machine learning algorithms and computer vision from the camera to perform flexible pick-place operations via socket communication.