Planar Pushing of Unknown Objects Using a Large-Scale Simulation Dataset and Few-Shot Learning

Abstract

Contact-rich object manipulation skills challenge the recent success of learning-based methods. It is even more difficult to predict the state of motion of novel objects due to the unknown physical properties and generalization issues of the learning-based model. In this work, we aim to predict the dynamics of novel objects in order to facilitate model-based control methods in planar pushing. We deal with this problem in two aspects. First, we present a large-scale planar pushing simulation dataset called SimPush. It is characterized by a large number of pushes and a variety of object physical properties, providing a wide avenue for exploring the object responses to the pusher action. Secondly, we propose a novel task-aware representation for pushes. This method keeps the spatial relation between the object and pusher and emphasizes the local contact features. Finally, we propose an encoder-decoder structured model possessing a cascaded residual attention mechanism to integrate prior knowledge to infer novel object motions. We experimentally show that the proposed model purely trained by SimPush attains good performance and robust prediction of novel object motions.