A deep architecture for log-Euclidean Fisher vector end-to-end learning with application to 3D point cloud classification

Abstract

Point clouds are a widely used form of 3D data, which can be produced by depth sensors, such as RGB-D cameras. The classification of common elements of 3D point clouds remains an open research problem.

We propose a new deep network approach for the end-to-end training of log-Euclidean Fisher vectors (LE-FVs), applied to the classification of 3D point clouds. Our method uses a log-Euclidean (LE) metric in order to extend the concept of Fisher vectors (FVs) to LE-FV encoding. The LE-FV was computed on covariance matrices of local 3D point cloud descriptors, representing multiple features. Our architecture is composed of two blocks. The first one aims to map the covariance matrices representing the 3D point cloud descriptors to the Euclidean space. The second block allows for joint and simultaneous learning of LE-FV Gaussian Mixture Model (GMM) parameters, LE-FV dimensionality reduction, and multi-label classification.

Our LE-FV deep learning model is more accurate than the FV deep learning architecture. Additionally, the introduction of joint learning of 3D point cloud features in the log-Euclidean space, including LE-FV GMM parameters, LE-FV dimensionality reduction, and multi-label classification greatly improves the accuracy of classification. Our method has also been compared with the most popular methods in the literature for 3D point cloud classification, and it achieved good performance. The quantitative evidence will be shown through different experiments.