Adherent Raindrop Removal with Self-Supervised Attention Maps and Spatio-Temporal Generative Adversarial Networks

Stefano Alletto, Casey Carlin, Luca Rigazio, Yasunori Ishii, Sotaro Tsukizawa; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 0-0

With the rapid increase of outdoor computer vision applications requiring robustness to adverse weather conditions such as automotive and robotics, the loss in image quality that is due to raindrops adherent to the camera lenses is becoming a major concern. In this paper we propose to remove raindrops and improve image quality in the spatio-temporal domain by leveraging the inherent robustness of adopting motion cues and the restorative capabilities of conditional generative adversarial networks. We first propose a competitive single-image baseline capable of estimating the raindrop locations in a self-supervised manner, and then use it to bootstrap our novel spatio-temporal architecture. This shows encouraging performance when compared to both state of the art single-image de-raining methods, and recent video-to-video translation approaches.