Jittered Exposures for Image Super-Resolution

Nianyi Li, Scott McCloskey, Jingyi Yu; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2018, pp. 1852-1859

Camera design involves tradeoffs between spatial and temporal resolution. For instance, traditional cameras provide either high spatial resolution (e.g., DSLRs) or high frame rate, but not both. Our approach exploits the optical stabilization hardware already present in commercial cameras and increasingly available in smartphones. Whereas single image super-resolution (SR) methods can produce convincing-looking images and have recently been shown to improve the performance of certain vision tasks, they are still limited in their ability to fully recover information lost due to under-sampling. In this paper, we present a new imaging technique that efficiently trades temporal resolution for spatial resolution in excess of the sensor's pixel count without attenuating light or adding additional hardware. On the hardware front, we demonstrate that the consumer-grade optical stabilization hardware provides enough precision in its static position to enable physically-correct SR. On the algorithm front, we elaborately design the Image Stabilization (IS) lens position pattern so that the SR can be efficiently conducted via image deconvolution. Compared with state-of-the-art solutions, our approach significantly reduces the computation complexity of the processing step while still providing the same level of optical fidelity, especially on quantifiable performance metrics from optical character recognition (OCR) and barcode decoding.