Multi-Level Adaptive Separable Convolution for Large-Motion Video Frame Interpolation

Ruth Wijma, Shaodi You, Yu Li; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2021, pp. 1127-1135

Current state-of-the-art methods within Video Frame In-terpolation (VI) fail at synthesizing interpolated frames incertain problem areas, such as when the video containslarge motion. This work aims at improving performanceon frame sequences containing large displacements by ex-tending the Adaptive Separable Convolution model in twoways. First of all, we increase the receptive field of themodel by utilizing spatial pyramids, which efficiently in-crease the interpolation kernel size. We additionally adaptthe network to accommodate for four frames, as opposedto just two, which should give it the ability to learn morecomplex motion patterns. This work also introduces theLarge-Motion Video Interpolation Dataset (LMD), whichcontains extracted frames from videos containing large dis-placements and highly non-linear movements. Our analy-sis shows that applying the model changes, together withthe use of our new dataset, does indeed result in improvedperformance on large displacement videos. We also showthat the increase in performance generalizes to frame se-quences of all sorts by outperforming other models in ourbenchmark on most tasks, and almost setting the new state-of-the-art on the Vimeo-90K dataset.