Semantic stereo: Integrating piecewise planar stereo with segmentation and classification

Piecewise planar model for stereo matching can overcome the challenges presented by poorly textured surfaces. Lots of works employ color segmentation cues to build piecewise planar model. However, segmentation is not sufficient to represent the content-consistency, since segments are usually too small or too messy to ensure pixels from the same object surface to be assigned to the same disparity layer. To obtain more compact piecewise planar models for urban scenes, in this work, a two-layer (pixel-wise and semantic-piecewise) graph structure, which incorporates cues from image segmentation and semantic classification, is proposed. One of the graph layers models pixel color-consistency in image pairs, and the other models the disparity smoothness between image segments of the same object according to semantic classification. Experiments on different urban scenes justify the efficiency of our method.