Audio-visual discrepancy and the influence on vertical sound source localization

Abstract

Human audio perception is influenced by vision and vice versa. The effect and thresholds of perceptual fusion, for example the ventriloquism-effect, are well investigated for natural listening conditions in the horizontal plane. Modern reproduction approaches for realistic spatial audio, e.g. binaural reproduction, promise more realistic sound reproduction, though, including proper perception of direction, distance, and elevation. This raises the question if the thresholds of perceptual fusion in audio reproduction systems that consider elevation are the same as in natural listening conditions. To estimate the influence of audiovisual discrepancy on vertical sound source localization via binaural headphones, two experiments were conducted. Results show an effect of similar magnitude for the vertical and horizontal plane.