Cross-form facilitation effects from simultaneous gesture/word combinations with ERP analysis

Abstract

The aim of this study was to investigate if simultaneous redundant forms of stimuli involving gesture and word combinations would produce a facilitation effect in reaction time as well as produce an neurological semantic incongruity effect called an N400. Event-related potentials (ERPs) were recorded from 14 scalp electrode sites while subject performed reaction-time decision tasks. For each trial, subjects had to indicate if the single form or cross-form stimuli were oriented upwards or downwards, or if they contradicted each other (only applicable in cross-form stimuli were there were two stimuli). The uniform stimuli were defined by a picture of a man pointing up or down, a picture of just a hand pointing up or down, or various word combinations such as up/down, high/low, above/below, and top/bottom. The cross-form stimuli were combinations of the words and pictures. The results show that while there was no N400 effect, there was however an facilitation effect regarding the positioning of the cross-form stimuli indicating that people have a tendency to pay more attention to gestures than words for information.