-
[pdf]
[supp]
[bibtex]@InProceedings{Gutierrez-Barragan_2023_ICCV, author = {Gutierrez-Barragan, Felipe and Mu, Fangzhou and Ardelean, Andrei and Ingle, Atul and Bruschini, Claudio and Charbon, Edoardo and Li, Yin and Gupta, Mohit and Velten, Andreas}, title = {Learned Compressive Representations for Single-Photon 3D Imaging}, booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)}, month = {October}, year = {2023}, pages = {10756-10766} }
Learned Compressive Representations for Single-Photon 3D Imaging
Abstract
Single-photon 3D cameras can record the time-of-arrival of billions of photons per second with picosecond accuracy. One common approach to summarize the photon data stream is to build a per-pixel timestamp histogram, resulting in a 3D histogram tensor that encodes distances along the time axis. As the spatio-temporal resolution of the histogram tensor increases, the in-pixel memory requirements and output data rates can quickly become impractical. To overcome this limitation, we propose a family of linear compressive representations of histogram tensors that can be computed efficiently, in an online fashion, as a matrix operation. We design practical lightweight compressive representations that are amenable to an in-pixel implementation and consider the spatio-temporal information of each timestamp. Furthermore, we implement our proposed framework as the first layer of a neural network, which enables the joint end-to-end optimization of the compressive representations and a downstream SPAD data processing model. We find that a well-designed compressive representation can reduce in-sensor memory and data rates up to 2 orders of magnitude without significantly reducing 3D imaging quality. Finally, we analyze the power consumption implications through an on-chip implementation.
Related Material