Accurate speaker counting, diarization and separation for advanced recognition of multichannel multispeaker conversations

Abstract

This paper aims at highlighting the recent trends and effective solutions in the field of automatic distant speech recognition in a multispeaker and multichannel recording scenario. The paper is mainly based on the results and experience obtained during our extensive research for CHiME-8 Challenge Task 1 (DASR) aimed at automatic distant speech transcription and diarization with multiple recording devices. Our main attention was paid to the carefully trained and tuned diarization pipeline and the speaker counting. This allowed us to significantly reduce the diarization error rate (DER) and obtain more reliable segments for speech separation and recognition. To improve source separation, we designed a Guided Target Speaker Extraction (G-TSE) model and used it in conjunction with the traditional Guided Source Separation (GSS) method. To train various parts of our pipeline, we investigated several data augmentation and generation techniques, which helped us improve the overall system quality.