TransInpaint: Transformer-Based Image Inpainting with Context Adaptation

Pourya Shamsolmoali, Masoumeh Zareapoor, Eric Granger; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2023, pp. 849-858

Image inpainting aims to generate realistic content for missing regions of an image. Existing methods often struggle to produce visually coherent content for missing regions of an image, which results in blurry or distorted structures around the damaged areas. These methods rely on surrounding texture information and have difficulty in generating content that harmonizes well with the broader context of the image. To address this limitation, we propose a novel model that generates plausible content for missing regions while ensuring that the generated content is consistent with the overall context of the original image. In particular, we introduce a novel context-adaptive transformer for image inpainting (TransInpaint) that relies on the visible content and the position of the missing regions. Additionally, we design a texture enhancement network that combines skip features from the encoder with the coarse features produced by the generator, yielding a more comprehensive and robust representation of image content. Based on extensive evaluations on challenging datasets, our proposed TransInpaint outperforms the cutting-edge generative models for image inpainting in terms of quality, textures, and structures.