CleftLipGAN : Interactive GAN-Inpainting for Post-Operative Cleft Lip Reconstruction

Daniel Anojan Atputharuban, Christoph Theopold, Aonghus Lawlor; Proceedings of the Asian Conference on Computer Vision (ACCV) Workshops, 2024, pp. 175-192

Synthetic generation of post-surgical outcomes holds significant value in the clinical domain, particularly for Cleft lip and Palate surgery. These synthetic images can be utilized for surgical planning, serve as reference points to evaluate surgical success and assist in educating patients and caretakers about potential outcomes. Image inpainting is effective for selectively generating Cleft-affected regions, making it a promising technique for this task. However, due to the lack of publicly available Cleft-specific datasets, Cleft inpainting models are typically trained on healthy data and applied to Cleft conditions to generate post-surgical lip appearances. Existing Cleft inpainting methods often struggle to capture the complexities of Cleft deformities, leading to implausible outcomes that fail to reflect the unique structural characteristics of Cleft-affected regions. To address this, we propose a Structural Guided Pluralistic Inpainting model, trained on healthy images, which allows for real-time, interactive adjustments to synthesize Cleft-specific images. We demonstrate the model's effectiveness by generating images that closely resemble Cleft conditions and benchmarking it against existing GAN-Inpainting methods. Additionally, we provide a user-friendly interface designed as a tool for post-surgical visualization of Cleft conditions. The source code is available at https: //github.com/danielanojan/CleftLipGAN.git