Layout-guided Indoor Panorama Inpainting with Plane-aware Normalization

Chao-Chen Gao, Cheng-Hsiu Chen, Jheng-Wei Su, Hung-Kuo Chu; Proceedings of the Asian Conference on Computer Vision (ACCV), 2022, pp. 2337-2353

We present an end-to-end deep learning framework for indoor panoramic image inpainting. Although previous inpainting methods have shown impressive performance on natural perspective images, most fail to handle panoramic images, particularly the indoor scenes, which usually contain complex structure and texture content. To achieve better inpainting quality, we propose to exploit both the global and local context of indoor panorama during the inpainting process. Specifically, we take the low-level layout edges estimated from input panorama as a prior to guide the inpainting model for recovering the global indoor structure. A plane-aware normalization module is employed to embed plane-wise style features derived from the layout into the generator, encouraging local texture restoration from adjacent room structures (i.e. ceiling, floor, and walls). Experimental results show that our work outperforms the current state-of-the-art methods on a public panoramic dataset quantitatively and qualitatively.