Multimodal weighted dictionary learning

Abstract

Classical dictionary learning algorithms that rely on a single source of information have been successfully used for the discriminative tasks. However, exploiting multiple sources has demonstrated its effectiveness in solving challenging real-world situations. We propose a new framework for feature fusion to achieve better classification performance as compared to the case where individual sources are utilized. In the context of multimodal data analysis, the modality configuration induces a strong group/coupling structure. The proposed method models the coupling between different modalities in space of sparse codes while at the same time within each modality a discriminative dictionary is learned in an all-vs-all scheme whose class-specific sub-parts are non-correlated. The proposed dictionary learning scheme is referred to as the multimodal weighted dictionary learning (MWDL). We demonstrate that MWDL outperforms state-of-the-art dictionary learning approaches in various experiments.