Real-Time Physiological Measurement and Visualization Using a Synchronized Multi-Camera System

Otkrist Gupta, Dan McDuff, Ramesh Raskar; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2016, pp. 46-53

Remote physiological measurement has widespread implications in healthcare and affective computing. This paper presents an efficient system for remotely measuring heart rate and heart rate variability using multiple low-cost digital cameras in real-time. We combine an RGB camera, monochrome camera with color filter and a thermal camera to recover the blood volume pulse (BVP). We show that using multiple cameras in synchrony yields the most accurate recovery of the BVP signal. The RGB combination is not optimal. We show that the thermal camera improves performance of measurement under dynamic ambient lighting but thermal camera alone is not enough and accuracy can be improved by adding more spectral channels. We present a real-time prototype that allows accurate physiological measurement combined with a novel user interface to visualize changes in heart rate and heart rate variability. Finally, we propose how this system might be used for applications such as patient monitoring.