Texture overlay onto deformable surface using HMD

Abstract

We propose a system that overlays textures onto the deformable surface of an object in real time using HMD. We assume that the surface projected onto an HMD image consists of curved surfaces which can be approximated by 2D geometric curved surface, so that we can deform textures using the matrix of 2D geometric transformation and the deformed textures are overlaid onto the HMD image. The system computes the transformation matrix in each frame, the textures are overlaid in real time even if an observer with HMD moves or deforms the surface. In the system, we select a book as the object with deformable shape and documents as the textures. Therefore, the observer can read digitized documents as if he reads real books.