TexDC: Text-Driven Disease-Aware 4D Cardiac Cine MRI Images Generation

Cong Liu, Xiaohan Yuan, ZhiPeng Yu, Yangang Wang; Proceedings of the Asian Conference on Computer Vision (ACCV), 2024, pp. 3005-3021

Generating disease-aware cardiac cine magnetic resonance imaging (cine MRI) images has immense potential in medical research, with recent advancements in text-driven image generation technology offering a viable solution. However, establishing clear correlations between textual descriptions and subtle disease regions, especially in capturing their dynamic complexities within cardiac contexts, remains a challenge. To tackle this, our approach emphasizes pre-aligning textual and cardiac cine MRI image features to highlight critical disease areas, establishing interactive relationships between disease text features and spatiotemporal image features during generation. We propose a text-driven framework for synthesizing disease-aware cardiac cine MRI images. Initially, knowledge is transferred from large language models, refining input semantics by updating learnable contexts. By introducing disease-aware pre-alignment, we emphasize and align key disease features across textual and spatiotemporal dimensions, effectively guiding image generation while maintaining spatiotemporal coherence. To our knowledge, this represents the first application of text-driven medical image generation in 4D modalities. We evaluate the superiority of our method on multi-center cardiac cine MRI datasets. Our code is available at: xxx.