GANESH: Generalizable NeRF for Lensless Imaging

Rakesh Raj Madhavan, Akshat Kaimal, Badhrinarayanan K.V, Vinayak Gupta, Rohit Choudhary, Chandrakala Shanmuganathan, Kaushik Mitra; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 9481-9490

Lensless imaging offers a significant opportunity to develop ultra-compact cameras by removing the conventional bulky lens system. However without a focusing element the sensor's output is no longer a direct image but a complex multiplexed scene representation. Traditional methods have attempted to address this challenge by employing learnable inversions and refinement models but these methods are primarily designed for 2D reconstruction and do not generalize well to 3D reconstruction. We introduce GANESH a novel framework designed to enable simultaneous refinement and novel view synthesis from multi-view lensless images. Unlike existing methods that require scene-specific training our approach supports on-the-fly inference without retraining on each scene. Moreover our framework allows us to tune our model to specific scenes enhancing the rendering and refinement quality. To facilitate research in this area we also present the first multi-view lens- less dataset LenslessScenes. Extensive experiments demonstrate that our method outperforms current approaches in reconstruction accuracy and refinement quality. Code and video results are available here.