Coded Two-Bucket Cameras for Computer Vision

Mian Wei, Navid Sarhangnejad, Zhengfan Xia, Nikita Gusev, Nikola Katic, Roman Genov, Kiriakos N. Kutulakos; Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 54-71

We introduce coded two-bucket (C2B) imaging, a new operating principle for computational sensors with applications in active 3D shape estimation and coded-exposure imaging. A C2B sensor modulates the light arriving at each pixel by controlling which of the pixel's two "buckets" should integrate it. C2B sensors output two images per video frame---one per bucket---and allow rapid, fully-programmable, per-pixel control of the active bucket. Using these properties as a starting point, we (1) develop an image formation model for these sensors, (2) couple them with programmable light sources to acquire illumination mosaics, i.e., images of a scene under many different illumination conditions whose pixels have been multiplexed onto the sensor plane and acquired in one shot, and (3) show how to process illumination mosaics to acquire time-varying depth or normal maps of dynamic scenes at the sensor's native resolution. We present the first experimental demonstration of these capabilities, using a fully functional C2B camera prototype. Key to this prototype is a C2B sensor that was designed by us, fabricated in a standard CMOS imaging technology, and demonstrated for the first time in this paper.