A Video Compression Framework Using an Overfitted Restoration Neural Network

Gang He, Chang Wu, Lei Li, Jinjia Zhou, Xianglin Wang, Yunfei Zheng, Bing Yu, Weiying Xie; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2020, pp. 148-149

Many existing deep learning based video compression approaches apply deep neural networks (DNNs) to enhance the decoded video by learning the mapping between de- coded video and raw video (ground truth). The big chal- lenge is to train one well-fitted model (one mapping) for various video sequences. Different with the other applica- tions such as image enhancement whose ground truth can only be obtained in the training process, the video encoder can always get the ground truth which is the raw video. It means we can train the model together with video com- pression and use one model for each sequence or even for each frame. The main idea of our approach is building a video compression framework (VCOR) using overfitted restoration neural network (ORNN). A lightweight ORNN is trained for a group of consecutive frames, so that it is overfitted to this group and achieves a strong restoration ability. After that, parameters of ORNN are transmitted to the decoder as a part of the encoded bitstream. At the de- coder side, ORNN can perform the same strong restoration operation to the reconstructed frames. We participate in the CLIC2020 challenge on P-frame track as the team "WestWorld".