LungHeart-AtMe: Adventitious Cardiopulmonary Sounds Classification Using MMoE with STFT and MFCCs Spectrograms

Adventitious cardiopulmonary (lung and heart) sound detection and classification through a digital stethoscope plays a vital role in early diagnosis and telehealth services. However, automatically detecting the adventitious sounds is challenging since they are easily susceptible to each other’s influence and noises. In this paper, for the first time, we simultaneously classify adventitious lung and heart sounds using our proposed LungHeart-AtMe model based on a mixed dataset of the ICBHI 2017 lung sounds dataset and the PhysioNet 2016 heart sounds dataset. Based on characteristics of lung and heart sounds, Wavelet Decomposition is applied first to perform noise reduction, then two time-frequency feature extraction techniques, which are Short Time Fourier Transform (STFT) and Mel Frequency Cepstral Coefficients (MFCCs), are chosen to extract preliminary features of sounds and transform sounds data to spectrograms that are easy to analyze. Our LungHeart-AtMe model is improved by introducing MMoE structure and by using the attention mechanism-based CNN model to extend its global feature extraction capability. From our experimental result, LungHeart-AtMe has achieved a Sensitivity of 71.55% and a Specificity of 28.06% for cardiopulmonary sounds classification.