SOMMA: Cortically inspired paradigms for multimodal processing

Abstract

SOMMA (Self Organizing Maps for Multimodal Association) consists on cortically inspired paradigms for multimodal data processing. SOMMA defines generic cortical maps - one for each modality - composed of 3-layers cortical columns. Each column learns a discrimination to a stimulus of the input flow with the BCMu learning rule [26]. These discriminations are self-organized in each map thanks to the coupling with neural fields used as a neighborhood function [25]. Learning and computation in each map is influenced by other modalities thanks to bidirectional topographic connections between all maps. This multimodal influence drives a joint self-organization of maps and multimodal perceptions of stimuli. This work takes place after the design of a self-organizing map [25] and of a modulation mechanism for influencing its self-organization [26] oriented towards a multimodal purpose. In this paper, we introduce a way to connect these self-organizing maps to obtain a multimap multimodal processing, completing our previous work. We also give an overview of the architectural and functional properties of the resulting paradigm SOMMA.