Equipment Assembly Recognition for Augmented Reality Guidance

Abstract

Equipment maintenance is a challenging task, particularly for complex equipment with many parts. Augmented Reality (AR) technology can assist technicians by providing real-time, on-site guidance. A fundamental requirement for this guidance is recognizing the current pose and assembly state of the equipment. This work addresses the problem of recognizing the pose and assembly state of equipment from multiple visible light images, specifically aiming to handle real-world equipment with hundreds of parts and millions of possible assembly states. We propose a novel two-stage method that first estimates a coarse pose and assembly state, then refines these estimates by leveraging multi-view integration of 2D features in a 3D voxel grid. Our approach is validated on two real assemblies with hundreds of parts: a small engine and a 3D printer. Experimental results demonstrate the effectiveness of our method, with refinement improving both pose and assembly state estimates. This work contributes a new perspective to AR-guided equipment maintenance, highlighting the importance of valid assembly states in training and the benefits of multi-view feature integration for assembly recognition.