Design and implementation of shoes arrangement robot system for unknown environment

TANG Xiaolong and HUANG Hui College of Computer Science and Software Engineering, Shenzhen University, Shenzhen 518060, Guangdong Province, P. R. China Abstract: In recent years, the robot and artificial intelligence technology have developed rapidly, but there still remains a challenging problem to realize the contact operation of robot in home environment. In order to make the robot arrange shoes automatically, we design an autonomous shoes arrangement robot system based on the 3D vision. In this system, the instance segmentation network and minimum enclosing rectangle are used to recognize the shoes and their orientation, and the grasping pose and placing pose of robot are estimated accurately by means of the point cloud information of depth camera. In addition, the convolution neural network and cosine similarity are adopted to match with a pair of shoes. Afterwards, we evaluate the accuracy of shoe orientation recognition and shoe matching in the system, and then carry out a real robot arrangement experiment. The result show that this method could assure 96.2% accuracy of shoe orientation recognition, and the matching accuracy of shoes increases from 62.6% to 87.4% when the VGG16 network is added to the shoes matching algorithm. In conclusion, this method can accurately recognize the shoes and their orientation, and then match with a pair of shoes, meanwhile, improves the stability of the robot shoes arrangement.