Asymmetric Color Transfer With Consistent Modality Learning

Kaiwen Zheng, Jie Huang, Man Zhou, Feng Zhao; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2023, pp. 2815-2823

The mono-color dual-lens system widely exists in the smartphone that captures asymmetric stereo image pairs, including high-resolution (HR) monochrome images and low-resolution (LR) color images. Asymmetric color transfer aims to reconstruct an HR color image by transferring the color information of the LR color image to the HR monochrome image. However, the inconsistency of spectral resolution and spatial resolution between stereo image pairs poses a challenge for establishing reliable stereo correspondence for precise color transfer. Previous works have not adequately addressed this issue. In this paper, we propose a dual-modality consistency learning framework to assist the establishment of reliable stereo correspondence. According to the complementarity of color and frequency information between stereo images, a dual-branch Stereo Information Complementary Module (SICM) is devised to perform the consistent modality learning in feature domain. Specifically, we meticulously design the stereo frequency and color modulation mechanism equipped in the SICM for capturing the information complementarity between dual-modal features. Furthermore, a parallax attention distillation is proposed to drive consistent modality learning for better stero matching. Extensive experiments demonstrate that our model outperforms the state-of-the-art methods in the Flickr1024 dataset and has superior generalization ability over the KITTI dataset and real-world scenarios. The code is available at https://github.com/keviner1/SICNet.