An Efficient Hybrid Model for Low-Light Image Enhancement in Mobile Devices

Zhicheng Fu, Miao Song, Chao Ma, Joseph Nasti, Vivek Tyagi, Grant Lloyd, Wei Tang; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2022, pp. 3057-3066

With the help of continuous optimizations in hardware and software, smartphones can now capture vivid, detailed macro pictures as well as high-resolution videos. However, taking photos/videos in a low-light environment with smartphones would still result in underexposed and bad-quality photos/videos due to their physical limitations -- small sensor size, compact lenses, and the lack of specific hardware and software. A variety of low-light enhancement techniques have been proposed, but their effectiveness is limited by their high complexity and the limited computational resources of smartphones. In this paper, we present an efficient hybrid solution, named as LLNet, to generate a high-resolution enhanced image given the corresponding high-resolution low-light image on mobile devices. LLNet consists of two main parts: 1) a lightweight convolutional neural network for features restoration that takes a low-resolution low-light image scaled down from the high-resolution input and predicts an enhanced low-resolution output; 2) a non-trainable transformation estimation model that approximates a linear transformation between the low-resolution input and predicted low-resolution output. By applying the estimated transformation on a high-resolution low-light image, the corresponding enhanced image can be predicted efficiently. To support the development of this learning-based solution, we introduce a dataset of normal-exposure low-light images, with corresponding long-exposure reference images, and all the images were captured by smartphones under real-world low-light scenes. Experiments demonstrate that LLNet can provide a real-time (around 32ms) smartphone preview (1440*1080 resolution) with outstanding image enhancement under low-light environments with affordable resources consumption. One real viewfinder video demo is attached as supplementary material to indicate the practicality of LLNet on real smartphones.