Self-Supervised Training for Blind Multi-Frame Video Denoising

Valery Dewil, Jeremy Anger, Axel Davy, Thibaud Ehret, Gabriele Facciolo, Pablo Arias; Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2021, pp. 2724-2734

Abstract


We propose a self-supervised approach for training multi-frame video denoising networks. These networks predict frame t from a window of frames around t. Our self-supervised approach benefits from the video temporal consistency by penalizing a loss between the predicted frame t and a neighboring target frame, which are aligned using an optical flow. We use the proposed strategy for online internal learning, where a pre-trained network is fine-tuned to denoise a new unknown noise type from a single video. After a few frames, the proposed fine-tuning reaches and sometimes surpasses the performance of a state-of-the-art network trained with supervision. In addition, for a wide range of noise types, it can be applied blindly without knowing the noise distribution. We demonstrate this by showing results on blind denoising of different synthetic and real noises.

Related Material


[pdf] [supp] [arXiv]
[bibtex]
@InProceedings{Dewil_2021_WACV, author = {Dewil, Valery and Anger, Jeremy and Davy, Axel and Ehret, Thibaud and Facciolo, Gabriele and Arias, Pablo}, title = {Self-Supervised Training for Blind Multi-Frame Video Denoising}, booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)}, month = {January}, year = {2021}, pages = {2724-2734} }