Towards Real-world Video Face Restoration: A New Benchmark

Ziyan Chen, Jingwen He, Xinqi Lin, Yu Qiao, Chao Dong; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 5929-5939

Blind image face restoration (BFR) has significantly progressed over the last several years while the real-world video face restoration (VFR) which is more challenging for more complex face motions such as moving gaze directions and facial orientations involved remains unsolved. Typical BFR methods are evaluated on privately synthesized datasets or self-collected real-world low-quality face images which are limited in their coverage of real-world video frames. In this work we introduce a new benchmark dataset named FOS with a taxonomy of Full Occluded and Side faces from both video frames and images with not only synthetic and real-world degradations covered but also rich facial data contents such as various expressions poses and ethnicity involved. Given the established datasets we benchmark both the state-of-the-art BFR methods and the video super resolution (VSR) methods to comprehensively study current approaches revealing their potential and limitations in VFR tasks. For performance assessment we not only employ the commonly used general image quality assessment (IQA) metrics but also explore face IQA metrics by leveraging an elaborately designed user study. With extensive experimental results and detailed analysis provided we gain insights from the successes and failures of current methods. In addition these studies also pose challenges in current BFR approaches and shed light on future VFR research.