A Lightweight Learning Framework for Packet Loss Concealment and Speech Enhancement

Abstract

Voice-related online communication applications are vulnerable to disruptions from complex environments, such as packet loss in IP-switch channels and ambient noise, which hinder communication efficiency. Various packet loss concealment (PLC) and speech enhancement (SE) techniques have been developed to enhance speech quality and to improve user experience. However, generating high-quality speech under noisy conditions with low computational cost remains a significant challenge. This study introduces a lightweight FCN-IPF approach that integrates a fully convolutional network (FCN) with an interpolation-based post-filter (IPF) to reduce signal processing time while improving sound quality and intelligibility. The proposed FCN-IPF is evaluated on system robustness, computational cost, and performance under packet loss and ambient noise conditions. Results show that FCN-IPF-processed speech achieves improvements of 17.65% in sound quality and 12.00% in intelligibility compared to the packet loss input. Under challenging conditions, sound quality and intelligibility are improved by 12.84% and 10.45%, respectively. Additionally, compared to the conventional CRN_FC method, FCN-IPF reduces model processing time per frame from 60.52 ms to 42.69 ms and decreases memory usage from 162 MB to 6.51 MB. These enhancements lower communication latency and boost noise reduction capabilities, making FCN-IPF well-suited for communication in challenging environments.