3D scene graph prediction from point clouds

Abstract

Background

In this study, we propose a novel 3D scene graph prediction approach for scene understanding from point clouds.

Methods

It can automatically organize the entities of a scene in a graph, where objects are nodes and their relationships are modeled as edges. More specifically, we employ the DGCNN to capture the features of objects and their relationships in the scene. A Graph Attention Network (GAT) is introduced to exploit latent features obtained from the initial estimation to further refine the object arrangement in the graph structure. A one loss function modified from cross entropy with a variable weight is proposed to solve the multi-category problem in the prediction of object and predicate.

Results

Experiments reveal that the proposed approach performs favorably against the state-of-the-art methods in terms of predicate classification and relationship prediction and achieves comparable performance on object classification prediction.

Conclusions

The 3D scene graph prediction approach can form an abstract description of the scene space from point clouds.