Video-based heart rate estimation with spectrogram signal quality ranking and fusion

Abstract

Remote photoplethysmography (rPPG) enables non-contact measurement of heart rate (HR). However, the stability of rPPG extraction is a bottleneck limiting its application. To address this issue, a signal quality ranking and fusion (SQRF) approach based on HR continuity in the time–frequency domain is introduced. Firstly, the facial region is divided into multiple regions of interest (ROIs), and the raw blood volume pulse (BVP) signal is extracted from each ROI separately using a conventional rPPG method such as the plane orthogonal to skin (POS) method. Then, wavelet synchrosqueezed transform (WSST) is employed to convert the raw pulse signals into spectrograms, which are further ranked according to the HR instantaneous continuity. The selected spectrograms with high-quality HR continuity are then fused using a weighted average to predict the final HR. The proposed SQRF algorithm is verified on three public datasets DDPM, UBFC-Phys and PURE with real scenarios. The obtained mean absolute error (MAE) was reduced by 58.7%, 47.5%, and 16.0% respectively, compared to the original single-ROI method. The results prove that SORF with spectrogram-based HR continuity can consistently boost the stability of POS.