SLAM Augmented Reality Based Interaction Method for Hand Rehabilitation Training

Abstract

The current hand rehabilitation training implemented through augmented reality technology has shown the potential to enhance user engagement. However, it falls short in realizing the virtual and realistic occlusion effect, or in producing realistic edge processing of virtual and realistic occlusion. This fragmentation between virtual objects and the real world poses a significant challenge. In light of this, this paper proposes a method that utilizes SLAM to achieve more accurate and realistic voxel occlusion and interaction effects during hand rehabilitation training. The method begins with acquiring dense point cloud data, which is voxelized by SLAM with dynamic removal. Corresponding objects are then segmented using OPTICS clustering for voxel edge constraint. Next, a second thread is initiated to reconstruct the dense point cloud prediction, leveraging local map semantic information combined with the edge SDF algorithm for the input 3D point cloud. By combining the predicted point cloud with the voxel segmented edges, the edge points are updated, and new edge surfaces are fitted to improve the accuracy and shapeliness of the 3D segmented object edges. The feasibility and accuracy of the algorithm are verified using datasets and real-time experiments. Finally, the paper presents an interactive system for hand rehabilitation training with virtual and real occlusion.