Seamless-Through-Breaking: Rethinking Image Stitching for Optimal Alignment

KuanYan Chen, Atik Garg, Yu-Shuen Wang; Proceedings of the Asian Conference on Computer Vision (ACCV), 2024, pp. 4352-4367

In this paper, we introduce a novel concept called seamless-through-breaking to tackle the challenges that arise in image stitching. Conventional methods attempt to maintain warping continuity while stitching two images together to avoid visible breaks in the final output. However, we propose that content alignment and warping continuity are mutually exclusive, especially when a significant depth gap exists between the foreground and the background. To solve this issue, we use optical flow to warp the source image into the target image's domain, which allows the creation of holes in the source image. Considering that optical flow estimators are trained on synthetic data, we fine-tune the estimator using real-world data to improve its accuracy in practical applications. Once the images are aligned within the same domain, we fill these holes with content from the target image. Additionally, as no optical flow estimators are perfect, directly copying pixels from the target image to fill the holes may create visual artifacts. To avoid this issue, we apply an image inpainting technique around the edges of the holes to smooth out alignment discrepancies, ensuring that the stitched image looks as natural as if it were captured in one shot.