Augmenting the world with interactive virtual studio

Abstract

In this paper, a system named IVS (Interactive Virtual Studio), which integrates low-level (feature extraction and ordering), mid-level (camera parameters estimation and evaluation), high-level (object recognition and reconstruction), and graphics techniques, is presented. The purpose of IVS is to integrate computer generated animation with real image sequence seamlessly. With minimum human intervention, IVS is an off-line postprocessing system that operates on a real world sequence in which a known object is tracked. The camera pose in each frame is recovered from which the camera trajectory is reconstructed. Firstly, object features are extracted from the environment and correspondence between the extracted shape and a shape from a database is solved by an SVD (Singular Value Decomposition) based technique. Secondly, SFSM (Structure from Simulated Motion) is used for object recognition and the information is used for reconstruction of the 3D object contained in the image. An evaluation mechanism is introduced to provide feedback and reduce the registration error. In the last stage of the processing, virtual graphics objects are imported to interact with the real image sequence. While the concept of developing such a system is not new, the various novel techniques we incorporated into IVS make it different.