Customized Object Recognition and Segmentation by One Shot Learning with Human Robot Interaction

There are two difficulties to utilize state-of-the-art object recognition/detection/segmentation methods to robotic applications. First, most of the deep learning models heavily depend on large amounts of labeled training data, which are expensive to obtain for each individual application. Second, the object categories must be pre-defined in the dataset, thus not practical to scenarios with varying object categories. To alleviate the reliance on pre-defined big data, this paper proposes a customized object recognition and segmentation method. It aims to recognize and segment any object defined by the user, given only one annotation. There are three steps in the proposed method. First, the user takes an exemplar video of the target object with the robot, defines its name, and mask its boundary on only one frame. Then the robot automatically propagates the annotation through the exemplar video based on a proposed data generation method. In the meantime, a segmentation model continuously updates itself on the generated data. Finally, only a lightweight segmentation net is required at testing stage, to recognize and segment the user-defined object in any scenes.