Space-Time-Brightness Sampling Using an Adaptive Pixel-Wise Coded Exposure

Hajime Nagahara, Toshiki Sonoda, Dengyu Liu, Jinwei Gu; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2018, pp. 1834-1842

Most conventional digital video cameras face a fundamental trade-off between spatial resolution, temporal resolution and dynamic range (i.e., brightness resolution) because of a limited bandwidth for data transmission. A few recent studies have shown that with non-uniform space-time sampling, such as that implemented with pixel-wise coded exposure, one can go beyond this trade-off and achieve high efficiency for scene capture. However, in these studies, the sampling schemes were pre-defined and independent of the target scene content. In this paper, we propose an adaptive space-time-brightness sampling method to further improve the efficiency of video capture. The proposed method adaptively updates a pixel-wise coded exposure pattern using the information analyzed from previously captured frames. We built a prototype camera that enables adaptive coding of patterns online to show the feasibility of the proposed adaptive coded exposure method. Simulation and experimental results show that the adaptive space-time-brightness sampling scheme achieves more accurate video reconstruction results and high dynamic range with less computational cost, than previous method. To the best of our knowledge, our prototype is the first implementation of an adaptive pixel-wise coded exposure camera.