RTrPPG: An Ultra Light 3DCNN for Real-Time Remote Photoplethysmography

Deivid Botina-Monsalve, Yannick Benezeth, Johel Miteran; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2022, pp. 2146-2154

The acquisition of remote photoplethysmography (rPPG) signals is important in multiple applications. Recently, deep-learning-based approaches such as 3D convolutional networks (3DCNNs) have outperformed traditional hand-crafted methods. However, despite their robust modeling ability, it is well known that large 3DCNN models have high computational costs and may be unsuitable for real-time applications. In this paper, we propose a study of the 3DCNN architecture, finding the best compromise between heart rate measurement precision and inference time. The fast inference is obtained decreasing the input size while the precision performance is obtained introducing a new time and frequency-based loss function by adding the signal-to-noise-ratio component to the regular Pearson's correlation loss function. In addition, changing the input color space from RGB to YUV also improved heart rate measurement precision. Using the VIPL-HR database, we retained the HR mean absolute error at 3.99 bpm which is comparable to 3.87 bpm of the state-of-the-art, while the GPU and CPU inference process improved around 88% from 51.77 ms to 2.32 ms in GPU and from 241.57 ms to 28.65 ms in CPU. The resulting network is called Real-Time rPPG (RTrPPG). We release the RTrPPG source code to encourage reproducibility.