-
[pdf]
[supp]
[arXiv]
[bibtex]@InProceedings{Liu_2025_WACV, author = {Liu, Kuan-Hung and Yang, Cheng-Kun and Chen, Min-Hung and Liu, Yu-Lun and Lin, Yen-Yu}, title = {CorrFill: Enhancing Faithfulness in Reference-Based Inpainting with Correspondence Guidance in Diffusion Models}, booktitle = {Proceedings of the Winter Conference on Applications of Computer Vision (WACV)}, month = {February}, year = {2025}, pages = {1618-1627} }
CorrFill: Enhancing Faithfulness in Reference-Based Inpainting with Correspondence Guidance in Diffusion Models
Abstract
In the task of reference-based image inpainting an additional reference image is provided to restore a damaged target image to its original state. The advancement of diffusion models particularly Stable Diffusion allows for simple formulations in this task. However existing diffusion-based methods often lack explicit constraints on the correlation between the reference and damaged images resulting in lower faithfulness to the reference images in the inpainting results. In this work we propose CorrFill a training-free module designed to enhance the awareness of geometric correlations between the reference and target images. This enhancement is achieved by guiding the inpainting process with correspondence constraints estimated during inpainting utilizing attention masking in self-attention layers and an objective function to update the input tensor according to the constraints. Experimental results demonstrate that CorrFill significantly enhances the performance of multiple baseline diffusion-based methods including state-of-the-art approaches by emphasizing faithfulness to the reference images.
Related Material