Sound source localization and detection based on densely connected network and attention mechanism

Abstract

Sound Source Localization and Detection (SSLD) is a joint task of detecting sound event activity and sound source localization (SSL). This paper proposes an SSLD method based on dense connection and attention mechanism. We propose a densely connected block parallel to the gated linear unit (GLU) to enhance feature propagation and alleviate vanishing gradient problems. We introduce multi-headed self-attention (MHSA) to aggregate contextual information and model long-term dependencies. The output adopts activity-coupled Cartesian direction-of-arrival (ACCDOA) representation. We decomposed the loss function based on Euclidean distance and proposed a loss function based on vector length loss and vector angle loss. These two types of losses correspond to sound activity detection (SAD) and direction-of-arrival (DOA) estimation in the SSLD task, respectively. The experimental results indicate that the proposed model achieves state-of-the-art performance compared to the baseline models. The proposed loss function also achieved performance improvement compared to the general Euclidean distance loss. In addition, we propose a system for remote SSL. In a real-world environment, we used two Laser Doppler vibrometers (LDVs) to remotely capture sound signals from different types of target objects for remote SSL. The results of real-world experiments have validated the effectiveness of the proposed system and demonstrated its potential application value.