Content-based 3D mosaic representation for video of dynamic 3D scenes

Abstract

We propose a content-based 3D mosaic representation for long video sequences of 3D and dynamic scenes captured by a camera on a mobile platform. The motion of the camera has a dominant direction of motion (as on an airplane or ground vehicle), but 6 degrees-of-freedom (DOF) motion is allowed. In the first step, a pair of generalized parallel-perspective (pushbroom) stereo mosaics is generated that captured both the 3D and dynamic aspects of the scene under the camera coverage. In the second step, a segmentation-based stereo matching algorithm is applied to extract parametric representation of the color, structure and motion of the dynamic and/or 3D objects in urban scenes where a lot of planar surfaces exist. Based on these results, the content-based 3D mosaic (CB3M) representation is created, which is a highly compressed visual representation for very long video sequences of dynamic 3D scenes. Experimental results are given