Toward Motion Robustness: A Masked Attention Regularization Framework in Remote Photoplethysmography

Pengfei Zhao,Qigong Sun,Xiaolin Tian,Yige Yang,Shuo Tao,Jie Cheng,Jiantong Chen; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 7829-7838

There has been growing interest in facial video-based remote photoplethysmography (rPPG) measurement recently with a focus on assessing various vital signs such as heart rate and heart rate variability. Despite previous efforts on static datasets their approaches have been hindered by inaccurate region of interest (ROI) localization and motion issues and have shown limited generalization in real-world scenarios. To address these challenges we propose a novel masked attention regularization (MAR-rPPG) framework that mitigates the impact of ROI localization and complex motion artifacts. Specifically our approach first integrates a masked attention regularization mechanism into the rPPG field to capture the visual semantic consistency of facial clips while it also employs a masking technique to prevent the model from overfitting on inaccurate ROIs and subsequently degrading its performance. Furthermore we propose an enhanced rPPG expert aggregation (EREA) network as the backbone to obtain rPPG signals and attention maps simultaneously. Our EREA network is capable of discriminating divergent attentions from different facial areas and retaining the consistency of spatiotemporal attention maps. For motion robustness a simple open source detector MediaPipe for data preprocessing is sufficient for our framework due to its superior capability of rPPG signal extraction and attention regularization. Exhaustive experiments on three benchmark datasets (UBFC-rPPG PURE and MMPD) substantiate the superiority of our proposed method outperforming recent state-of-the-art works by a considerable margin.