A Calibration Algorithm for Real-time Scene-aware Portable Augmented Reality

Abstract

In order to solve the problem of accurate holographic projection of the point cloud collected by an external high-precision depth camera in HoloLens, we propose an augmented reality calibration algorithm for real-time scene perception. Firstly, we build a portable high-precision real-time sensing system, using external RealSense to collect point cloud data, and the portable host processes and returns the data to HoloLens via a local area network. Secondly, it calibrates the internal parameters of HoloLens' webcam and RealSense depth cameras, then fixed the two cameras for dual purpose calibration, so as to obtain the internal rotation and translation matrix. Finally, the calculated posture computed by the matrix transformation transforms of the virtual object from the RealSense coordinate system displayed in OSG (Open Scene Graph) to HoloLens unified. The Direct X coordinate system is then transformed into the HoloLens Webcam coordinate system, and then the HoloLens API is used to acquire the fixed coordinate system established during the acquisition. At the same time, the virtual object of the holographic projection is accurately merged with the real object, and the spatial anchor is fixed in the real scene, so that the system realizes an accurate and real-time aware augmented reality capability.