Avatar motion adaptation for AR based 3D tele-conference

With the advent of inexpensive depth sensors and more viable methods for human tracking, traditional 2D tele-conference systems are evolving into one that is AR and 3D teleportation based. Compared to the traditional tele-conference systems which offer only flat 2D upper body imageries and mostly a fixed view point (and inconsistent gaze directions), an AR tele-conference with 3D teleported avatars would be more natural and realistic, and can give an enhanced and immersive communication experience. This paper presents an AR based 3D tele-conference prototype with a method to adapt the motion of the teleported avatar to the physical configuration of the other site. The adaptation is needed due to the differences in the physical environments between two sites where the human controller is interacting at one (e.g. sitting on a low chair) and the avatar is being displayed at the other (e.g. augmented on a high chair). The adaptation technique is based on preserving a particular spatial property among the avatar and its interaction objects between the two sites. The spatial relationship is pre-established between the important joint positions of the user/avatar and carefully selected points on the environment interaction objects. The motions of the user transmitted to the other site are then modified in real time considering the “changed” environment object and by preserving the spatial relationship as much as possible. We have developed a test prototype to demonstrate our approach using the Kinect-based human tracking and a video see-through head-mounted display.