Does Task-Irrelevant Brightness Modulation Affect Auditory Contrast Processing? Exploring the Interplay Between Temporal Synchrony and Stimulus Salience

Abstract

Stimulus factors such as timing, spatial location, and stimulus effectiveness affect whether and how information across the senses is integrated. Extending recent work highlighting interactions between stimulus factors, here we investigated the influence of visual information on auditory processing, complementing previous studies on the influence of auditory information on visual processing. We hypothesized that task-irrelevant and spatially non-informative visual information would enhance auditory contrast processing, when visual information was at an optimal salience level and changed synchronously with the sound. We asked human observers to indicate the location of an amplitude-modulated white-noise sound, while its loudness against a constant white-noise background varied across trials. To test for the influence of task-irrelevant visual information, we modulated screen brightness smoothly (Experiment 1) or transiently (Experiment 2) in phase or out of phase with the amplitude modulation of the target sound. In addition, to test for the interaction between temporal synchrony and stimulus salience, maximum brightness varied systematically across trials. Auditory contrast thresholds were compared across conditions. Results showed that task-irrelevant visual information did not alter auditory contrast thresholds regardless of the nature of modulation of brightness, contrary to our expectations. Nonetheless, task-irrelevant visual information modulated in phase with the target sound reduced auditory contrast thresholds if we accounted for individual differences in the optimal salience required for the largest multisensory effects. Our results are discussed in light of several stimulus factors that might be critical in modulating multisensory enhancement.