-
[pdf]
[arXiv]
[bibtex]@InProceedings{Morawski_2024_CVPR, author = {Morawski, Igor and He, Kai and Dangi, Shusil and Hsu, Winston H.}, title = {Unsupervised Image Prior via Prompt Learning and CLIP Semantic Guidance for Low-Light Image Enhancement}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops}, month = {June}, year = {2024}, pages = {5971-5981} }
Unsupervised Image Prior via Prompt Learning and CLIP Semantic Guidance for Low-Light Image Enhancement
Abstract
Currently low-light conditions present a significant challenge for machine cognition. In this paper rather than optimizing models by assuming that human and machine cognition are correlated we use zero-reference low-light enhancement to improve the performance of downstream task models. We propose to improve the zero-reference low-light enhancement method by leveraging the rich visual-linguistic CLIP prior without any need for paired or unpaired normal-light data which is laborious and difficult to collect. We propose a simple but effective strategy to learn prompts that help guide the enhancement method and experimentally show that the prompts learned without any need for normal-light data improve image contrast reduce over-enhancement and reduce noise over-amplification. Next we propose to reuse the CLIP model for semantic guidance via zero-shot open vocabulary classification to optimize low-light enhancement for task-based performance rather than human visual perception. We conduct extensive experimental results showing that the proposed method leads to consistent improvements across various datasets regarding task-based performance and compare our method against state-of-the-art methods showing favorable results across various low-light datasets.
Related Material