Locale-based multiple cue algorithm for object segmentation

Abstract

This paper proposes a Locale-based Multiple Cue (LMC) algorithm to solve the problem of segmenting foregroundmoving objects from the background scene. The major cue used for object segmentation is the motion information obtained from a novel locale-based motion estimation and clustering algorithm. At first, locales are classified according to color and intensity. The motion estimation is applied on the tiles of 16 x 16 pixels, which are the building blocks of locales. After motion estimation, camera motions are detected using a 2D affine motion model. Then the locales are grown from the tile level to the frame level in a pyramidal way using motion, color, and centroid variance constraint. The resulting locales, with tiles homogeneous in motion and color, are post-processed to recover the object boundary. Experimental results show that LMC combines temporal and spatial information in a graceful way, which enables it to segment the moving objects under different camera motions. Future work includes object tracking over multiple frames and utilization of texture information.