Multi-task learning (MTL) has demonstrated significant advantages in the field of physiological signal measurement. This approach enhances the model's generalization ability by sharing parameters and features between similar tasks, even in data-scarce environments. However, traditional multi-task physiological signal measurement methods face challenges such as feature conflicts between tasks, task imbalance, and excessive model complexity, which limit their application in complex environments. To address these issues, this paper proposes an enhanced multi-scale spatiotemporal network (EMSTN) based on Eulerian video magnification (EVM), super-resolution reconstruction and convolutional multilayer perceptron. First, EVM is introduced in the input stage of the network to amplify subtle color and motion changes in the video, significantly improving the model's ability to capture pulse and respiratory signals. Additionally, a super-resolution reconstruction module is integrated into the network to enhance the image resolution, thereby improving detail capture and increasing the accuracy of facial action unit (AU) tasks. Then, convolutional multilayer perceptron is employed to replace traditional 2D convolutions, improving feature extraction efficiency and flexibility, which significantly boosts the performance of heart rate and respiratory rate measurements. Finally, comprehensive experiments on the Binghamton-Pittsburgh 4D Spontaneous Facial Expression Database (BP4D+) fully validate the effectiveness and superiority of the proposed method in multi-task physiological signal measurement.
Heart rate is a crucial indicator of human health with significant physiological importance. Traditional contact methods for measuring heart rate, such as electrocardiograph or wristbands, may not always meet the need for convenient health monitoring. Remote photoplethysmography (rPPG) provides a non-contact method for measuring heart rate and other physiological indicators by analyzing blood volume pulse signals. This approach is non-invasive, does not require direct contact, and allows for long-term healthcare monitoring. Deep learning has emerged as a powerful tool for processing complex image and video data, and has been increasingly employed to extract heart rate signals remotely. This article reviewed the latest research advancements in rPPG-based heart rate measurement using deep learning, summarized available public datasets, and explored future research directions and potential advancements in non-contact heart rate measurement.
Remote photoplethysmography is susceptible to motion artifacts and individual physiological variations in complex environments. This paper proposes a remote heart rate estimation method based on frequency regulation and multi-scale spatio-temporal modeling. To address artifact noise issues, a frequency-regulated normalization module is designed to emphasize the dominant heart rate frequency while suppressing noise. To address the issue of individual physiological variations, the proposed method introduces a multi-level spatio-temporal feature fusion module to comprehensively capture physiological information through multi-scale convolutions and cross-layer integration. Subsequently, a dynamic weighting spatio-temporal feature module is introduced during spatio-temporal modeling to enhance long-term dependency modeling. Experimental results demonstrate that the proposed method achieves superior performance in cross-dataset evaluation. When trained on the PURE dataset and tested on the UBFC-rPPG dataset, the mean absolute error decreases from 1.31 to 1.28. Conversely, when trained on the UBFC-rPPG dataset and tested on the PURE dataset, the mean absolute error further decreases from 0.97 to 0.82. These results significantly outperform existing state-of-the-art methods, demonstrating the strong generalization capability and outstanding performance of our model across datasets. From the perspectives of frequency-regulated and multi-scale spatio-temporal modeling, this work enriches the modeling methodology for remote photoplethysmography pulse wave-based heart rate estimation, enhancing the stability and usability of remote heart rate estimation under complex interference and cross-scenario conditions.
