A template-projection approach to decode higher-order vision in realtime and at the perceptual threshold

Abstract

The link between object perception and neural activity in visual cortical areas is a problem of fundamental importance in neuroscience. We measured brain surface physiology with implanted electrocorticography (ECoG) electrodes in humans. Physiological responses to visual stimuli in object-specific ventral temporal loci are highly polymorphic in different cortical loci, for both broadband and raw potential trace changes. To address this, we developed a template-projection method, where averaged responses from a localizer task are projected into the continuous datastream recorded from the brain. These projections are used to build a feature space. A classifier for decoding visual perception is applied to this feature space during training periods, and is applied to plain images, as well as noise masked images. This enables robust classification of visual perceptual state.