RefVSR++: Exploiting Reference Inputs for Reference-Based Video Super-Resolution

Han Zou, Masanori Suganuma, Takayuki Okatani; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 2756-2765

Smartphones with multi-camera systems featuring cameras with varying field-of-views (FoVs) are increasingly common. This variation in FoVs results in content differences across videos paving the way for an innovative approach to video super-resolution (VSR). This method enhances the VSR performance of lower resolution (LR) videos by leveraging higher resolution reference (Ref) videos. Previous works which operate on this principle generally expand on traditional VSR models by combining LR and Ref inputs over time into a unified stream. However we can expect that better results are obtained by independently aggregating these Ref image sequences temporally. Therefore we introduce an improved method RefVSR++ which performs the parallel aggregation of LR and Ref images in the temporal direction aiming to optimize the use of the available data. RefVSR++ also incorporates improved mechanisms for aligning image features over time crucial for effective VSR. Our experiments demonstrate that RefVSR++ outperforms previous works by over 1dB in PSNR setting a new benchmark in the field.