Sensory modality and information domain contribute jointly to dual-task interference between working memory and perceptual processing

Abstract

Abstract Making sense of our environment requires us to extract temporal and spatial information from multiple sensory modalities, particularly audition and vision. Often, we must hold this sensory information in working memory (WM) to guide future actions, while simultaneously processing new sensory inputs as they arise. However, these processes of WM maintenance and perceptual processing can interfere with one another when the tasks rely on similar cognitive resources. fMRI studies have uncovered attention and WM networks that are specialized for either auditory-temporal or visual-spatial processing; the functional specialization of these networks makes specific predictions about patterns of interference between perceptual processing and WM. Specifically, we hypothesized that dual-task interference should increase when the tasks share a common sensory modality, a common information domain (temporal vs. spatial processing), or both. To test these predictions, we asked participants to store temporal or spatial information about auditory or visual stimuli in WM. On some trials, participants also performed an intervening auditory task, which was either temporal or spatial, during WM retention. Errors on WM recall and perceptual judgment tasks both generally increased when the tasks relied on shared modality- and domain-biased resources, with maximal interference when both tasks were auditory-temporal. Pupil dilations were also larger and started earlier when both tasks were auditory-temporal, indicating an increase in cognitive effort to overcome the interference. Event-related potentials (ERPs) and alpha-band oscillatory activity revealed neural signatures of domain-based interference even when the tasks were presented in different sensory modalities, when behavioral differences were masked by ceiling effects. These results demonstrate that sensory modality and information domain jointly affect how task information is represented in WM, consistent with past work demonstrating how tasks engage complementary auditory-temporal and visual-spatial cognitive control networks.