Respiratory Rate Estimation Based on Detected Mask Area in Thermal Images

Natalia Kowalczyk, Jacek Ruminski; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2023, pp. 6042-6051

The popularity of non-contact methods of measuring vital signs, particularly respiratory rate, has increased during the SARS-COV-2 pandemic. Breathing parameters can be estimated by analysis of temperature changes observed in thermal images of nostrils or mouth regions. However, wearing virus-protection face masks prevents direct detection of such face regions. In this work, we propose to use an automatic mask detection approach to select pixels within a mask region as a source of respiration information allowing efficient estimation of respiratory signals. We performed experiments with two important types of virus protection masks, i.e., FFP2 (N95) and surgical masks, for subjects while sitting, slowly walking from a short distance toward a camera, and slowly walking with moderate head movements. Experiments conducted with the adapted YOLO model have shown that detection of the mask area on the face allows for higher SNR values and reduces error in respiratory rate estimation in all analyzed scenarios. The Mean Absolute Error for respiratory rate estimation was below 1 bpm for sitting subjects for all types of masks. The error for walking subjects was 1.21 bpm for an FFP2 mask and about 2.1 bpm for a surgical mask. In the presence of head movements, while walking, the MAE was below 1.39 bpm and less than 1 bpm when only one outlier was removed.