Comparing Vision-based to Sonar-based 3D Reconstruction

Abstract

Our understanding of sonar based sensing is very limited in comparison to light based imaging. In this work, we synthesize a ShapeNet variant in which echolocation replaces the role of vision. A new hypernetwork method is presented for 3D reconstruction from a single echolocation view. The success of the method demonstrates the ability to reconstruct a 3D shape from bat-like sonar, and not just obtain the relative position of the bat with respect to obstacles. In addition, it is shown that integrating information from multiple orientations around the same view point helps performance. The sonar-based method we develop is analog to the state-of-the-art single image reconstruction method, which allows us to directly compare the two imaging modalities. Based on this analysis, we learn that while 3D can be reliably reconstructed form sonar, as far as the current technology shows, the accuracy is lower than the one obtained based on vision, that the performance in sonar and in vision are highly correlated, that both modalities favor shapes that are not round, and that while the current vision method is able to better reconstruct the 3D shape, its advantage with respect to estimating the normal's direction is much lower.