PhysKANNet: A KAN-based model for multiscale feature extraction and contextual fusion in remote physiological measurement

Physiological indicator reflects the health status of the human body, and remote photoplethysmography (rPPG) is a highly promising technology for contactless measurement of these indicators through facial video. However, current deep learning methods mainly rely on traditional neural networks with limited spatiotemporal receptive fields, overlooking the importance of multi-scale features and noise resistance in rPPG signal modeling. This results in challenges when addressing subtle color changes and noise interference. To overcome these limitations, we leverage the advantages of the Kolmogorov-Arnold Network (KAN) in handling sparse data and propose PhysKANNet, a novel KAN-based encoder–decoder architecture that integrates multi-scale feature extraction and contextual information fusion to enhance rPPG signal extraction. We introduce three new plug-and-play modules for PhysKANNet: the rPPG-Aware Convolutional Attention Block, which extracts features at different scales through a multi-branch structure and enhances multi-scale representation using KAN’s nonlinear modeling capabilities; the Multi-Dimensional Feature Fusion Block, which combines high-dimensional features from the encoder with low-dimensional features from the decoder; and the rPPG Edge Sampling Block, which fuses edge and semantic information to further optimize signal extraction accuracy. We employ unsupervised learning for training PhysKANNet and conducted comprehensive experiments on multiple benchmark datasets. The results show that PhysKANNet significantly improves feature learning from unlabeled data, achieving excellent performance across various testing scenarios.