Imperceptible projection blanking for reliable segmentation within mixed reality applications

Abstract

Applications that run in a projection-based mixed reality environment frequently require simultaneous image acquisition of the user by a camera. For later processing, very often a segmentation of the user's image is required, but along with the user, the camera captures the projected dynamic image in the background of the scene. This dynamic background complicates the later segmentation of the user in real-time. To ease segmentation, we introduce methods allowing to imperceptible blank the projected image for the camera. Four different methods are tested and compared. Acquired video streams are recorded and a successful segmentation of a user in front of a dynamic background is demonstrated. The paper concludes with an outlook on possible applications.