360° structured light with learned metasurfaces

Abstract

Structured light has proven instrumental in three-dimensional imaging, LiDAR and holographic light projection. Metasurfaces, comprising subwavelength-sized nanostructures, facilitate 180°-field-of-view structured light, circumventing the restricted field of view inherent in traditional optics like diffractive optical elements. However, extant-metasurface-facilitated structured light exhibits sub-optimal performance in downstream tasks, due to heuristic design patterns such as periodic dots that do not consider the objectives of the end application. Here we present 360° structured light, driven by learned metasurfaces. We propose a differentiable framework that encompasses a computationally efficient 180° wave propagation model and a task-specific reconstructor, and exploits both transmission and reflection channels of the metasurface. Leveraging a first-order optimizer within our differentiable framework, we optimize the metasurface design, thereby realizing 360° structured light. We have utilized 360° structured light for holographic light projection and three-dimensional imaging. Specifically, we demonstrate the first 360° light projection of complex patterns, enabled by our propagation model that can be computationally evaluated 50,000× faster than the Rayleigh–Sommerfeld propagation. For three-dimensional imaging, we improve the depth-estimation accuracy by 5.09× in root-mean-square error compared with heuristically designed structured light. Such 360° structured light promises robust 360° imaging and display for robotics, extended-reality systems and human–computer interactions. A single-metasurface-based holographic light projection covering the whole 360° field of view is realized by optimizing the metasurface design through a neural network and applying 360° structured light for holographic light projection and three-dimensional imaging.