Modalities consensus for multi-modal constraint propagation

Abstract

This paper presents a novel modalities consensus framework for multi-modal pairwise constraint propagation (MCP). We first combine multiple single-modal constraint propagation (SCP) problems together, and then explicitly introduce a new modalities consensus regularizer to force the propagation results on different modalities to be consistent with each other. With a separable consensus regularizer, the proposed approach can be effectively solved using an alternating optimization way. More importantly, based on our modalities consensus framework, two single-modal constraint propagation algorithms can be directly reformulated as two well-defined multi-modal solutions. Experimental results on constrained clustering tasks have shown that the proposed framework can achieve significant improvements with respect to the state of the arts.