Weakly Supervised rPPG Estimation for Respiratory Rate Estimation

Jingda Du, Si-Qi Liu, Bochao Zhang, Pong C. Yuen; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2021, pp. 2391-2397

Recent studies demonstrate that respiratory rate can be estimated from skin videos through analyzing the frequency domain attributes of their remote photoplethysmography (rPPG). However, respiration is not always periodic so the frequency attributes of rPPG may not accurately estimate the respiratory rate. In this paper, we proposed an end-to-end network to estimate both rPPG signals and respiratory rates from facial videos. Since only breathing waves are available in the Remote Physiological Signal Sensing track2 competition, to preserve the respiratory pattern in rPPG estimation, rPPG signals pre-estimated by chrominace-based methods and modulated by breathing waves are used as weak labels for supervision. To adapt to the large differences between training and testing data, in terms of recording environment and subjects behavior, we also involved customized adversarial training on feature extractor to minimize the domain gap. In the competition, our model achieved 7.56 bpm MAE and ranked the second place.