RainGAN: Unsupervised Raindrop Removal via Decomposition and Composition

Xu Yan, Yuan Ren Loke; Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Workshops, 2022, pp. 14-23

Adherent raindrops on windshield or camera lens may distort and occlude vision, causing issues for downstream machine vision perception. Most of the existing raindrop removal methods focus on learning the mapping from a raindrop image to its clean content by the paired raindrop-clean images. However, the paired real-world data is difficult to collect in practice. This paper presents a novel framework for raindrop removal that eliminates the need for paired training samples. Based on the assumption that a raindrop image is a composition of a clean image and raindrop style, the proposed framework decomposes a raindrop image into a clean content image and a raindrop-style latent code. Inversely, it composes a clean content image and a raindrop style code to a raindrop image for data augmentation. The proposed framework introduces a domain-invariant residual block to facilitate the identity mapping for the clean portion of the raindrop image. Extensive experiments on real-world raindrop datasets show that our network can achieve superior performance in raindrop removal to other unpaired image-to-image translation methods, even with comparable performance with state-of-the-art methods that require paired data.