Bilateral transformer 3D planar recovery

Abstract

In recent years, deep learning based methods for single image 3D planar recovery have made significant progress, but most of the research has focused on overall plane segmentation performance rather than the accuracy of small scale plane segmentation. In order to solve the problem of feature loss in the feature extraction process of small target object features, a three dimensional planar recovery method based on bilateral transformer was proposed. The two sided network branches capture rich small object target features through different scale sampling, and are used for detecting planar and non-planar regions respectively. In addition, the loss of variational information is used to share the parameters of the bilateral network, which achieves the output consistency of the bilateral network and alleviates the problem of feature loss of small target objects. The method is verified on Scannet and Nyu V2 datasets, and a variety of evaluation indexes are superior to the current popular algorithms, proving the effectiveness of the method in three dimensional planar recovery.