Digital-twin deep dynamic camera position optimisation for the V-STARS photogrammetry system based on 3D reconstruction

Abstract

Photogrammetry systems are widely used in industrial manufacturing applications as an assistance measurement tool. Not only does it provide high-precision feedback for assembly process inspection and product quality assessment, but also it can improve the flexibility and robustness of manufacturing systems and production lines. However, with growing global competition and demands, companies are forced to enhance production efficiency, shorten production lifecycle and increase product variety by incorporating reconfigurable factory design that can meet challenging timeline and requirements. Although dynamic facility layout is widely investigated, the position selection for the photogrammetry system in dynamic manufacturing environment is usually overlooked. In this paper, dynamic layout of the V-STARS photogrammetry system is investigated and optimised in a digital-twin environment using deep reinforcement learning. The learning objectives are derived from the field of view (FoV) evaluation from point clouds 3D reconstruction, and collision detection from the digital twin simulated in Visual Components. The application feasibility of the proposed dynamic layout optimisation of the V-STARS photogrammetry system is verified with a real world industrial application.